Skip to main content

How Your Audit Score is Calculated

Understanding your AuditAgent score is key to gauging your project's security posture. While the precise algorithm involves multiple factors and weightings, this page provides a transparent overview of the core components that contribute to your final score, which is presented on a scale of 0-100 (where 100 is the best possible score).

Security Score

Core Principles​

The scoring mechanism is designed to reflect the density and severity of issues found within your codebase, adjusted by the confidence we have in the detectors that found them, and slightly adjusted with the overall code quality.

warning

Despite our best efforts to deliver the most accurate and reliable results, large language models (LLMs) are still prone to hallucinations and may sometimes return false positives. Review all findings carefully.

Calculation Stages​

The final score is derived through a multi-stage process:

1. Base Security Score (0-100)​

This is the primary component of your overall score.

  • Weighted Issues: Each finding is assigned a weight based on its severity (High, Medium, Low). Informational findings and best practices typically do not negatively impact the score directly through weights.
  • Detector Confidence Adjustment: The weight of a finding is then adjusted based on the confidence level associated with the detector that identified it. For example, findings from more deterministic static analyzers might carry more weight than those from detectors that are inherently more probabilistic, like some LLM-based scans. This helps in normalizing the impact of different types of findings.
  • Issue Density: The total adjusted weighted issues are then considered in relation to the size of your codebase (specifically, the number of actual code lines, excluding comments). A project with many issues in a small codebase will generally have a higher density of findings leading to a lower score than a project with the same number of issues in a much larger codebase.
  • Security Penalty: An issue density metric is calculated, and a penalty is derived from this. This penalty is scaled (using a logarithmic function) to ensure that scores are more granular, especially at lower to moderate issue densities, rather than dropping drastically with only a few findings.
  • The Base Security Score is then calculated as 100 - Security Penalty.

2. Quality Penalty​

A small penalty may be applied based on the overall code quality detected and if the code has a very low ratio of comments to actual code lines.

  • Comment Ratio: We calculate the ratio of comment lines to code lines.
  • Penalty Application: If this ratio falls below a predefined target, a small penalty is applied. This penalty is capped to avoid moving the overall score too much.
  • This is intended to gently encourage best practices and basic code documentation. No bonus is given for exceeding the target comment ratio, to avoid incentivizing excessive or unhelpful comments.
tip

Leverage the Summary and the Diagram that we generate from you codebase to create/improve your documentation!

3. Final Score​

  • The Final Score is determined by subtracting the Quality Penalty (if any) from the Base Security Score.
  • Final Score = Base Security Score - Quality Penalty
  • The result is then rounded up to the nearest integer and clamped between 0 and 100.

Important Considerations​

  • Lines of Code: The score normalizes for project size by considering lines of actual code. This means a larger project isn't unfairly penalized simply for having more code, as the density of issues is what matters.
  • Severity Matters: High-severity issues will impact your score more significantly than low-severity ones.
  • Experimental Nature: As mentioned in the overview, the scoring system, particularly aspects influenced by LLM-based analysis, is continuously being refined. LLMs can occasionally produce false positives, which might affect the score.
  • Focus on Actionable Insights: While the score provides a useful benchmark, the primary goal of AuditAgent is to provide actionable insights. Focus on addressing the identified vulnerabilities, especially those with higher severity.

This overview should give you a clearer understanding of what influences your audit score. We believe in transparency and are continuously working to improve the accuracy and utility of our scoring algorithm.