Aram Hovespyan, co-founder and CEO of security biz Codific, says that the rating systems for identifying security vulnerabilities and assessing threat risk need to be overhauled.

Having examined the CVE (Common Vulnerabilities and Exposures) vulnerability identification numbering system, Hovespyan argues that about a third of CVEs are meaningless.

His analysis cites academic research published in August as part of the USENIX Security Symposium. The paper, “Confusing Value with Enumeration: Studying the Use of CVEs in Academia,” (Moritz Schloegel et al.), reports that 34 percent of 1,803 CVEs cited in research papers over the past five years either have not been publicly confirmed or have been disputed by maintainers of the supposedly vulnerable software projects. The authors argue that CVEs should not be taken as a proxy for the real-world impact of claimed vulnerabilities.

CVEs begin when a security researcher discloses a vulnerability to a CVE Numbering Authority (CNA). The CNA – initially MITRE, which subsequently extended CNA status to other organizations (e.g. Microsoft) – is supposed to review the submission, verify it, assign a CVE number, and eventually publish the details. 

CNAs may be companies, open source maintainers, foundations, service providers, vulnerability researchers, national computer security incident response teams (CSIRTs), and others. CNAs may also delegate CVE assignment to a CNA-LR – that is, a CVE Numbering Authority of Last Resort – such as Red Hat, which can assign CVEs and publish details outside of its scope, on behalf of a CNA.

Hovespyan says that the CVE assignment system is subject to misaligned incentives. 

“Vulnerability researchers often aim to publish as many CVEs as possible to build their reputations,” Hovespyan wrote in his post. “Product CNAs, on the other hand, have little motivation to create CVEs that expose flaws in their own software. Meanwhile, CNA Last Resorts typically lack the technical context for thorough validation and are more inclined to publish quickly rather than accurately.”

Developers then have to deal with these reports, which are difficult to dispute and may not be accurate or valid.

Hovespyan also looked at the related CVSS (Common Vulnerability Scoring System) for ranking vulnerability severity. He contends that these scores are inconsistent, noting that “studies have found that more than 40 percent of CVEs receive different scores when re-evaluated by the same person just nine months later.”

He also argues that running calculations on CVSS scores is mathematically unsound. The ordinal CVSS numbers assign the vulnerability’s position in a list but are inappropriately treated as quantitative values for security tool calculations and algorithms.

As an example of CVE system problems, he cites how Florian Hantke, a PhD student from Germany, created a CVE for a deprecated system no one used. The vulnerability received a 9.1 CVSS score before being downgraded. Hantke documented his experience in a blog post that concludes, “we need to recalibrate how we perceive and value CVEs.”

Hovespyan also cites a problematic curl vulnerability report that received a CVSS score of 9.8 out of 10, before being downgraded to 3.3.

Daniel Stenberg, the creator and maintainer of curl, a popular command line tool, told The Register in an email that Hovespyan’s criticism describes a real problem. It’s particularly an issue, he said, for products and projects that can be used in diverse environments where a single score can’t be easily applied to accurately reflect every usage scenario.

“CVSS is meant to give a base score and then everyone should apply their own environment and risk judgement on top, but in reality that is not how the numbers are used,” Stenberg explained.

“This is one of the reasons why we in curl project actually don’t provide CVSS scores at all. We don’t think we can reliably set a single score for the world to (ab)use. I often argue for this point in unison with Greg [Kroah-Hartman] who is head of the Linux kernel CNA who also never sets a CVSS for the CVEs they create – at considerably larger volumes than we do for curl.”

Stenberg in fact penned a blog post on the subject earlier this year. It’s titled, “CVSS is dead to us.”

While Hovespyan argues for procedural improvements among those assessing vulnerability reports, he acknowledges that CVEs and CVSS scores still have some value.

“CVEs and CVSS aren’t useless,” he said in a statement provided to The Register. “They’re valuable inputs. But they should never be the foundation of an entire AppSec strategy. We need to start with a shared understanding of risk, grounded in threat modeling and contextual triage. Vulnerability dashboards can help, but only when interpreted through a scientific lens.” ®