VulnCheck researcher Patrick Garrity challenged Anthropic’s Project Glasswing results in a November 9, 2026 analysis, arguing that duplicate findings and slow remediation undermine its vulnerability-discovery claims. His comparison says Anthropic’s ledger added 11,813 findings between September 8 and November 2 while recording only 15 additional fixes. VulnCheck’s analysis also questions how much validation work reaches open-source maintainers.
Anthropic’s own dashboard complicates that account. Its November 2 summary lists 629 findings patched upstream, substantially above the 217 marked fixed in Garrity’s analysis. It also reports a 94.2% true-positive rate, with qualifications about what that number measures.
The strongest conclusion supported by these sources concerns measurement: finding volume, report validity and completed remediation are different outcomes. Glasswing’s headline figures need those distinctions to be useful.
What VulnCheck’s Counts Actually Show
Garrity says his tracking identified 442 CVEs attributable to Anthropic over seven months. His collection draws on CVE records, GitHub Security Advisories, Linux kernel commits and Anthropic’s ledger, supplemented by manual advisory checks. It is therefore broader than a count of CVEs displayed on Anthropic’s dashboard.
He also reports that two of those 442 CVEs appeared in VulnCheck’s catalog of vulnerabilities known to be exploited. That measures exploitation identified by VulnCheck, not the potential severity or future usefulness of the remaining vulnerabilities. Treating a low observed-exploitation count as proof of low security value would go beyond the evidence. VulnCheck
The central comparisons look like this:
| Measure | VulnCheck’s account | Anthropic’s November 2 dashboard | How to interpret it |
|---|---|---|---|
| Findings discovered | 37,966 | 37,966 | Both use the same headline total. |
| CVEs | 442 attributed to Anthropic | 230 CVE records listed | Broader attribution tracking and dashboard records have different coverage. |
| Remediation | 217 marked fixed | 629 patched upstream | These counts cannot safely be treated as interchangeable. |
| Duplicates | 1,670 in its comparison table | No corresponding headline total in the opened dashboard summary | This is VulnCheck’s reading of ledger statuses. |
| Withdrawn findings | 419 | No corresponding headline total in the opened dashboard summary | Withdrawals need reasons before they can be classified as errors. |
Sources: VulnCheck’s November update and Anthropic’s disclosure dashboard.
VulnCheck’s post has an internal discrepancy: its prose gives 1,654 duplicates, while its comparison table gives 1,670. Both support its broader point about duplicate volume, but the exact count needs that qualification.
More consequentially, the available pages do not reconcile 217 fixed with 629 patched upstream. Anthropic also lists 503 GitHub Security Advisories alongside its 230 CVE records, explaining that one finding can carry both identifiers. Adding advisory records is therefore not a way to count unique vulnerabilities.
I would read Garrity’s “15 additional fixes” figure as a change in the ledger category he tracked, not as a demonstrated count of every fix produced during those two months. The dashboard’s larger patch total makes that distinction essential.
The 94.2% Rate Applies to a Reviewed Subset
Anthropic’s dashboard describes a workflow involving six external security research firms. Those firms reproduce candidate issues, assess validity and severity, and prepare reports for confirmed bugs.
As of November 2, the dashboard showed:
- 37,966 discovered findings.
- 6,376 candidates reviewed by external firms.
- 6,009 confirmed valid, yielding the reported 94.2% true-positive rate.
- 5,087 findings disclosed to maintainers across 680 projects.
- 629 findings known to have been patched upstream.
These are Anthropic’s published figures, not an independently audited measurement of Glasswing’s effectiveness. Anthropic’s dashboard
The company explicitly includes real bugs later found to be duplicates in its validity rate. It also includes issues maintainers will not fix, such as bugs outside a project’s threat model or affecting code that is not typically reachable. Anthropic acknowledges that external reviewers can make classification errors and describes the rate as “one proxy for impact.”
That definition explains how a high true-positive rate can coexist with duplicates and relatively few patches. A finding can describe a real bug without giving a maintainer a new, actionable security problem.
VulnCheck questions whether duplicates and withdrawals are feeding back into the reported rate appropriately. But duplicate status alone cannot disprove Anthropic’s rate under its stated definition. A withdrawn report is also difficult to interpret without knowing why it was withdrawn.
My view is that the missing measurement is maintainer usefulness: the share of reports that identify unique, relevant issues worth addressing. Validity matters, but it answers a narrower question.
Most Disclosed Reports Took the Direct Route
Anthropic’s dashboard contains a particularly consequential detail: not every disclosed report receives the external review described above.
It separates its 5,087 disclosures into:
- 1,862 findings reported following external review.
- 3,225 findings reported directly by Anthropic, without the same independent check.
The direct route can include cases where maintainers specifically request untriaged findings. Anthropic warns that this route may contain false positives. Anthropic’s dashboard
Dividing 3,225 by 5,087 shows that approximately 63.4% of disclosed findings used the direct route. That is a calculation from Anthropic’s published counts, not a separate benchmark.
This is the clearest reason readers should not apply the 94.2% rate to every report arriving in a maintainer’s inbox. The rate describes externally reviewed candidates; most disclosed findings followed a different path.
Direct disclosure is not inherently a bad arrangement. A maintainer may prefer early access and have the expertise to assess a report quickly. But it changes where validation happens. An opt-in stream of raw candidates is a different service from a stream of reproduced, externally assessed vulnerabilities.
Anthropic says human triage is a rate-limiting step and that confirmed bugs can remain undisclosed because of capacity limits. Faster discovery therefore does not automatically produce faster remediation.
Maintainer Accounts Point in Different Directions
VulnCheck cites a curl sample of 34 published reports: 19 rejected as vulnerabilities, six confirmed as low severity and nine awaiting assessment. That sample supports scrutiny of report usefulness, but it was incomplete at publication and cannot establish results across other projects. The counts are presented in Garrity’s analysis.
The backlash also attracted substantial discussion. The opened Hacker News thread showed 391 points and 180 comments, a snapshot of attention rather than a representative survey.
One participant, posting as pizlonator, described receiving nine Anthropic reports concerning Fil-C and wrote: “All of them were real bugs.” The commenter praised the reports and test cases, while describing the proposed fixes as useful to varying degrees.
Frequently Asked Questions
3 questions
1Do Duplicate Reports Count as False Positives?
Not necessarily. A duplicate can describe a real vulnerability that was already reported. Anthropic includes such bugs in its true-positive rate, although they may add little value for maintainers.
