Anthropic opens OSS Scanner to open source: free model-generated vulnerability reports, with no human triage
Synopsis
Anthropic launched OSS Scanner, an opt-in vulnerability scanner that periodically scans participating open-source projects for free using its strongest models. Maintainers previously received mostly low-quality reports from LLMs, and human validation now caps disclosure: over the past six months the company found more than 29,000 candidate vulnerabilities but manually reviewed only about 6,000. Scanner output is fully model-generated with no human triage, so reports may be wrong; in early validation, expert reviewers checked 97 critical and high-severity findings across 48 projects and 85, or 88%, met the bar for its coordinated disclosure process.
Interpretation
Open-source projects can now opt in to have Anthropic's strongest language models scan their code on a periodic basis for free and send vulnerability reports directly, without waiting for human-validated disclosure. Maintainers used to receive mostly low-quality content from LLMs and now receive high-quality bug reports, while human validation has become the bottleneck: of more than 29,000 candidate vulnerabilities found in six months, only about 6,000 were manually reviewed and triaged. On CyberGym, an academic vulnerability-finding benchmark, the share of vulnerabilities LLMs find rose from under 20% at the beginning of last year to over 85% this year, the capability background the service rests on.
The service packages scanning, reproduction and fix suggestions into a report a maintainer can act on immediately. Each report contains a self-contained reproducer, an explanation of the vulnerability including, where possible, a bisection of when the bug was introduced, and a candidate patch when available. In several weeks of pipeline validation with dozens of open-source projects, the initial disclosures contained hundreds of bug reports, including multiple vulnerabilities that could be chained into unauthenticated remote code execution exploits affecting those projects.
Scanner output is fully model-generated, with no human review or triage, in exchange for faster and more frequent scanning. This runs alongside Anthropic's continued manual disclosure of human-verified reports through its existing coordinated vulnerability disclosure process, especially for projects without the resources to triage reports themselves. Expert penetration testers reviewed 97 critical and high-severity vulnerabilities from the scanner across 48 projects; 85, or 88%, met the bar for the coordinated disclosure process, and of the remaining 12, 11 were real but duplicated and only one was a false positive.
Perspective
The service targets open-source projects with a critical impact on infrastructure and user security; core maintainers enroll by submitting a pull request following the project template, and eligibility is decided case by case. Reports are fully model-generated, without human review or triage.
Reports may be incorrect or invalid; some maintainers said severity ratings can be inflated or that the scanner misunderstood the project's threat model. Anthropic says it cannot guarantee the scanner will be perfect and will keep refining it based on maintainer feedback and as models improve.
