How OpenSSF Scorecards can help to evaluate open-source software risks

Everyone knows the phrase “software is eating the world” by Marc Andreessen from over a decade ago. Software powers and touches nearly every aspect of modern society, both personally and professionally, and is critical to the modern economy and national security.

It can also be said that open-source software (OSS) has eaten the software industry. The Linux Foundation and other groups have estimated that free and open-source software (FOSS) constitutes 70% to 90% of any modern software product. Not only is modern software largely composed of OSS components, but IT leaders are more likely to work with vendors who also contribute to the OSS community.

OSS use is rampant because of its flexibility, cost savings, innovation through community enabled projects, and arguably better security through more eyeballs on the code, especially for large OSS projects. That said, OSS comes with its own concerns, including Common Vulnerabilities and Exposures (CVEs) for affected code.

CVE is a project by MITRE that strives to “identify, define and catalog publicly disclosed cybersecurity vulnerabilities.” However, as the Cloud Native Computing Foundation (CNCF) Software Supply Chain Best Practices whitepaper notes, CVEs are a “trailing metric,” meaning they enumerations vulnerabilities that have been publicly disclosed. They are also just one type of risk associated with software.

For this reason, organizations should use other methods to evaluate the state of security for OSS projects they consume. One of the most notable is the Open Source Security Foundation’s (OpenSSF’s) Scorecards project.

What is OpenSSF Scorecards for open-source projects?

Announced in late 2020, Scorecards aims to auto-generate a security score for OSS projects to help consumers and organizations make risk-informed decisions about their OSS consumption. Organizations are making overwhelming use of OSS dependencies but determining the risk of consuming those dependencies remains a largely manual activity, particularly at scale across the software ecosystem. The Scorecards project seeks to alleviate some of that burden using automated heuristics and security checks on a scoring scale of 0 to 10. It does this while assessing OSS projects for security concerns that align with best practices such as signing or SAST, already advocated for by both public and private security leaders.

Chris Hughes

The Scorecards project isn’t aiming low either, they scan the one million most critical OSS projects based on direct dependencies and publish the results to a public dataset on a weekly basis. In addition to leveraging this publicly available dataset, organizations can also run Scorecards against their own GitHub projects by using GitHub actions. When there is a change in the repository, the GitHub Action runs and provides alerts to the maintainers of those projects.

The Scorecards project uses a scoring scale of critical, high, medium and low, which are severity levels that most security practitioners are familiar with. It also uses a standard list of checks that it runs against all the projects for which you target it, whether public projects or, if you’re using it natively, your own GitHub repositories.

You can dive into what some of those checks are. They include fundamental security practices such as using branch protection, cryptographically signing releases, and the presence of unfixed vulnerabilities. To detect the presence of unfixed vulnerabilities, the Scorecards project uses the OSV Vulnerability Database. This is a distributed vulnerability database for OSS that uses OpenSSF OSV format. OSV, at its core, is an aggregation of other vulnerability databases using the OSV schema, such as GitHub Security Advisories and the Global Security Database among others. OSC also supports both APIs and command line interface (CLI) tools for scanning SBOMs in either CycloneDX or SPDX formats.

The Scorecards project meets bi-weekly and has an active Slack channel. It is led by facilitators from companies such as Google, Datto and Cisco. Since its inception, Scorecards has grown in popularity and is listed as having over 3,000 Stargazers, or users who have essentially bookmarked the project. As organizations continue their push to mature their OSS consumption governance practices, the project will inevitably grow in popularity.

How can organizations use OpenSSF Scorecards?

Organizations’ ability to conduct due diligence, governance, and risk management of their OSS consumption is largely still in its infancy. We’re seeing a big push to bolster the software supply chain’s resiliency and mature organization’s software supply chain security practices. We’ve had NIST’s Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations Rev.1, Secure Software Development Framework (SSDF), the OpenSSF OSS Security Mobilization Plan, Supply Chain Levels for Software Artifacts (SLSA), and other best practices and sources of guidance emerge. All touch on the need to govern an organization’s consumption of OSS and to ensure that consumption aligns with the organization’s risk tolerance.

While that may sound straightforward, the idea of doing that across the entire robust ecosystem of OSS projects and components organizations are consuming isn’t so trivial. OpenSSF’s Scorecards project provides an automated way to get security and risk insights into over a million leading OSS projects as well as use the project natively for their own software and projects internal to the organization.

Organizations can use Scorecards via the CLI for projects they do not own as well as use a package manager for projects such as npm, PyPi or RubyGems. Scorecards is also available as a Docker container.

Organizations and individual contributors may participate in the project, including submitting need checks to be considered for the scoring assessment. Organizations can also customize their use of Scorecards and run only specific checks, potentially aligned with their organizational or industry-specific security requirements.

For more specific guidance, organizations can reference NIST’s recommended practices for Open Source Software Controls, which were published in response to the Cybersecurity Executive Order (EO) 14028, Improving the Nation’s Cybersecurity. These include tiered capabilities based on the maturity of the organization from three levels: foundational, sustaining and enhancing. Within those tiers are capabilities such as using software composition analysis (SCA) source code reviews to identify vulnerable components, prioritizing the use of programming languages with built-in guardrails, and automating the process of collecting, storing and scanning OSS components into hardened internal repositories prior to introducing them to production environments.