CWE-bench v2 (CWE-bench)
We show this table for reference; we do not rank on it.
Defensive vulnerability patching on 250 audit-and-patch tasks spanning 185 weakness types, 100 public and 150 held out.
Benchmark score on CWE-bench — November 11, 2026
We compile the CWE-bench rows from benchmark-owner or independent runs. Claude Opus 5.5 leads the table at 67.1%, followed by Muse Wonder (66.1%) and Gemini 4 Argon (65.0%). We do not use these results to rank models overall.
Claude Opus 5.5
Anthropic
Muse Wonder
Meta
Gemini 4 Argon
11 modelsAgenticCurrentDisplay onlyUpdated November 11, 2026
Current releaseCWE-bench v2 · Released · 250 tasks (100 public, 150 held out) · Official leaderboard · Not comparable with v1
Benchmark score table (11 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 5.5Anthropic | 67.1% | Not reported | Closed |
| 2 | Muse WonderMeta | 66.1% | Not reported | Closed |
| 3 | Gemini 4 ArgonGoogle | 65.0% | Not reported | Closed |
| 4 | GPT-6 AstraOpenAI | 64.2% | Not reported | Closed |
| 5 | DeepSeek V4.1 Flash 1017DeepSeek | 57.6% | Not reported | Open |
| 6 | Grok 4.7xAI | 54.7% | Not reported | Closed |
| 7 | Hy4 previewTencent | 47.3% | Not reported | Open |
| 8 | Mistral Large 4Mistral | 45.6% | Not reported | Open |
| 9 | Qwen3.8 MaxAlibaba | 45.4% | Not reported | Open |
| 10 | Kimi K3.1 ProMoonshot AI | 45.3% | Not reported | Open |
| 11 | GLM-5.3Z.AI | 42.0% | Not reported | Open |
CWE-bench version history
Every CWE-bench version BenchLM has recorded, newest first. Version 2 changes the task set (250 tasks, 185 weakness types, a public set of 100), so its pass@1 scores are not comparable with v1; ranks are the fairer way to follow a model across versions.
| Version | Released | Tasks | What changed | Leader | Models |
|---|---|---|---|---|---|
| CWE-bench v2Current | Nov 2, 2026 | 250 tasks (100 public, 150 held out) | Adds a public set of 100 tasks to 150 held-out ones and raises coverage from 73 to 185 weakness types. Pass@1 is over all 250 tasks. | LeaderClaude Opus 5.5 67.1%Official board | 11 models |
| CWE-bench v1 | Sep 25, 2026 | 120 held-out tasks | 120 held-out tasks across 73 weakness types, grouped by the OWASP Top 10. The source no longer shows this table; scores are as last captured on October 28, 2026. | LeaderGemini 4 Carbon 71%Official board | 12 models |
Rankings across CWE-bench versions
Each cell is the model’s rank among catalog models on that version’s leaderboard, with its pass@1 below. A dash means the model has no row on that version.
| Model | CWE-bench v211 models | CWE-bench v112 models |
|---|---|---|
| Claude Opus 5.5 | #167.1% | #567% |
| Muse Wonder | #266.1% | Not on this board |
| Gemini 4 Argon | #365% | #268% |
| GPT-6 Astra | #464.2% | #268% |
| DeepSeek V4.1 Flash 1017 | #557.6% | #855% |
| Grok 4.7 | #654.7% | #268% |
| Hy4 preview | #747.3% | #1053% |
| Mistral Large 4 | #845.6% | Not on this board |
| Qwen3.8 Max | #945.4% | Not on this board |
| Kimi K3.1 Pro | #1045.3% | Not on this board |
| GLM-5.3 | #1142% | Not on this board |
| Gemini 4 Carbon | Not on this board | #171% |
| Claude Fable 5.1 | Not on this board | #658% |
| Grok 4.6 | Not on this board | #757% |
| Muse Spark 1.3 | Not on this board | #855% |
| GPT-6 Sol | Not on this board | #1152% |
| Inkling | Not on this board | #1237% |
Previous releaseReleased · 120 held-out tasks · Official leaderboard · Not comparable with v2
CWE-bench v1
The deterministic programmatic pass@1 score requires blocking the exploit while preserving existing tests. Models use high reasoning, four rollouts per task, and a one-hour limit per rollout. Version 1 uses 120 tasks and stays separate from v0, pass@4, and judge-panel scores.
| Source rank | Source model or configuration | Score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Gemini 4 CarbonGoogle | 71.0% | Collinear AI · Original result | Not reported | Closed |
| 2 | GPT-6 AstraOpenAI | 68.0% | Collinear AI · Original result | Not reported | Closed |
| 3 | Gemini 4 ArgonGoogle | 68.0% | Collinear AI · Original result | Not reported | Closed |
| 4 | Grok 4.7xAI | 68.0% | Collinear AI · Original result | Not reported | Closed |
| 5 | Claude Opus 5.5Anthropic | 67.0% | Collinear AI · Original result | Not reported | Closed |
| 6 | Claude Fable 5.1Anthropic | 58.0% | Collinear AI · Original result | Not reported | Closed |
| 7 | Grok 4.6xAI | 57.0% | Collinear AI · Original result | Not reported | Closed |
| 8 | DeepSeek V4.1 Flash 1017DeepSeek | 55.0% | Collinear AI · Original result | Not reported | Open |
| 9 | Muse Spark 1.3Meta | 55.0% | Collinear AI · Original result | Not reported | Closed |
| 10 | Hy4 previewTencent | 53.0% | Collinear AI · Original result | Not reported | Open |
| 11 | GPT-6 SolOpenAI | 52.0% | Collinear AI · Original result | Not reported | Closed |
| 12 | InklingThinking Machines Lab | 37.0% | Collinear AI · Original result | Not reported | Open |
Among the reported CWE-bench rows, Claude Opus 5.5 is first at 67.1%. The third row is 2.1 points behind. The broader top-10 range is 21.8 points, so the table still separates the published systems.
11 models have been evaluated on CWE-bench. The benchmark falls in the Agentic category. CWE-bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About CWE-bench
Year
2026
Tasks
Audit and patch C/C++, Go, Java, TypeScript/JavaScript, Python, and Rust repositories; 100 public and 150 held-out tasks
Format
Pass@1
Difficulty
Frontier agent evaluation
The deterministic programmatic pass@1 score requires blocking the exploit while preserving existing tests, over all 250 tasks. Models use high reasoning, four rollouts per task, and a one-hour limit per rollout. Version 2 (November 2, 2026) adds a public set of 100 tasks and raises coverage from 73 to 185 weakness types, so its scores are not comparable with v1, whose last scores stay on the same page. Pass@4 and judge-panel scores are separate metrics.
Freshness and provenance
Version
CWE-bench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does CWE-bench measure?
Defensive vulnerability patching on 250 audit-and-patch tasks spanning 185 weakness types, 100 public and 150 held out.
Which model scores highest on CWE-bench?
Claude Opus 5.5 by Anthropic currently leads with a score of 67.1% on CWE-bench.
How many models are evaluated on CWE-bench?
11 AI models have published results on CWE-bench in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.