When Policy meets Execution
Community post by Amir Montazery, Managing Director, Open Source Technology Improvement Fund
In this blog post, we present an overview of independent audits conducted at the end of 2021 and first half of 2022. This is based on CNCF’s strong commitment to improving security of projects, a sound guiding policy and project maturity model, and a repeatable process for executing audits with the help of strategic partner Open Source Technology Improvement Fund (OSTIF). The post will begin with a brief introduction to security audits, highlight the guiding policy and CNCF projects audited, share some project feedback, and, finally, discuss associated and future work and conclude.
Security Audit Introduction
Here we will give a brief introduction to security audits. This is not meant to be an exhaustive or detailed description, but rather just enough to understand what a security audit is, the goals and benefits of security audits, and in particular a brief argument as to why they are absolutely critical for securing projects.
The Basics
OSTIF defines a Security Audit as a collaborative effort between project contributors/maintainers and security experts to independently analyze risks, examine code, build tooling, and find and fix vulnerabilities to improve the overall security posture of a project. It is critical that adherence to best practices, such as Independence & Objectivity, Due Professional Care, and Technical Acumen be followed in order to execute audits effectively.
Goals and Benefits of a Security Audit
The goal of a security audit is to have independent experts examine a project’s code, review tooling and practices, and collaborate with project maintainers on findings and fixes to improve security posture holistically. Every open source project is different, so it is important to tailor the audit to project needs and scope the audit accordingly.
The benefits of a security audit are two-fold; immediate and long-term. The immediate benefits come in the form of security focus, threat modeling, fixed vulnerabilities, and an audit report documenting the process. Project maintainers get the opportunity to think about their project solely from a security perspective. They collaborate with the audit team to document threat vectors and their impact. Auditors typically have findings, in the form of vulnerabilities, bugs, or miscellaneous improvements, that get fixed in collaboration with project maintainers and respective communities. The long-term benefits of a security audit come in the form of improved tooling and closed classes of bugs. Fuzzers and associated tools built as part of an audit continuously scan the code base for issues, and can be improved upon. Furthermore, when auditors find common themes or design issues as part of the audit, fixing them results in closed classes of bugs, reducing the risk or new problems in future development.
Why Security Audits are Critical for Securing Projects
Independent audits, third-party reviews, internal and IT audits are all examples of a best practice: peer review from a qualified external party. A fact remains that code is written by humans and humans make mistakes, and that’s okay! Security audits have shown to be an effective tool for improving software and finding and fixing problems. OSTIF is a trusted partner for facilitating security audits effectively, as demonstrated by its track record and this report.
Guiding Policy
CNCF provides a strong platform for open source projects to incubate and grow. Per the CNCF website: “CNCF projects have a maturity level of sandbox, incubating, or graduated, which corresponds to the Innovators, Early Adopters, and Early Majority tiers of the Crossing the Chasm diagram. The maturity level is a signal by CNCF as to what sorts of enterprises should be adopting different projects. Projects increase their maturity by demonstrating their sustainability to CNCF’s Technical Oversight Committee: that they have adoption, a healthy rate of changes, and committers from multiple organizations; have adopted the CNCF Code of Conduct; and have achieved and maintained the Core Infrastructure Initiative Best Practices Badge.” This is followed by the Graduation Criteria, which includes a list of criteria. The important criteria to note is for projects to have completed an independent and third party security audit with results published.
The Projects
The following CNCF projects have completed security audits or associated work. As of July 2022, a total of 7 projects worked with OSTIF to improve their security posture.
Feedback from Projects
“The maintainers and community are very grateful for the work put into this by everyone and the opportunity to grow and improve as a project.”
Project Community Manager
“This is a big progress to the whole Community.”
Lead Project Maintainer
“We had waited almost two years for a security review, which was frustrating because the bureaucratic process was opaque to us. Once OSTIF was brought on board, it happened in a couple of months. The team we worked with was professional, capable, and thorough. The whole process was a breeze, and I certainly would recommend it to any project looking for a similar audit.”
Lead Security Engineer
“Our project, despite having a dedicated team of maintainers, needed help navigating the complexities of getting security help. Once OSTIF got involved the process became significantly easier and we are pleased with the results. Our testing suite is more effective thanks to the team mobilized by OSTIF.”
Product Manager
“OSTIF helps ensure right focus on priorities by taking away the painstaking task of finding the right partners, project management responsibilities and ascertaining mutually agreeable modus operandi between parties involved.”
Product Team Lead
Associated and Future Work
As the CNCF ecosystem grows and matures, OSTIF looks to more collaborations and opportunities to improve security posture. OSTIF and CNCF are collaborating on a similar amount of audits for projects for the remaining half of the year with more to come. Furthermore, CNCF has supported a significant amount of fuzzing improvements among other efforts to improve security posture.
OSTIF is auditing more projects than ever before and working with multiple organizations and foundations with their security needs. An example would be OSTIF’s collaboration with Google and their Open Source Security Team. Recent audit results, such as the Simple Logging Facade for Java, slf4j, which was identified in the Harvard Census II results as one of the most widely-deployed logging frameworks, are a result of this collaboration.
Conclusion
A security audit requires three main parts to be successful: A qualified team of experts, strong scoping and project management, and engagement from project maintainers and respective communities. OSTIF has been able to methodize the first two parts, and a strong guiding policy and commitment to security from CNCF supports the third. Proactive security is both cost-effective and impactful, as these audits have been able to demonstrate. More funding means more security audits, which means better security in the open source ecosystem. More info on OSTIF and the CNCF projects audited (along with their respective audit reports) can be found at the links below.