CWE-bench v2 (CWE-bench)
We show this table for reference; we do not rank on it.
Defensive vulnerability patching on 250 audit-and-patch tasks spanning 185 weakness types, 100 public and 150 held out.
Benchmark score on CWE-bench — November 13, 2026
We compile the CWE-bench rows from benchmark-owner or independent runs. Claude Fable 5.2 leads the table at 75.0%, followed by Gemini 4 Hydrogen (68.1%) and Claude Opus 5.5 (67.1%). We do not use these results to rank models overall.
Claude Fable 5.2
Anthropic
Run by CWE-benchGemini 4 Hydrogen
Claude Opus 5.5
Anthropic
Run by CWE-bench14 modelsAgenticCurrentDisplay onlyUpdated November 13, 2026
Current releaseCWE-bench v2 · Released · 250 tasks (100 public, 150 held out) · Official leaderboard · Not comparable with v1
Benchmark score table (14 models)
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Fable 5.2AnthropicRun by CWE-bench | 75.0% | Run by CWE-bench | Not reported | Closed |
| 2 | Gemini 4 HydrogenGoogleRun by CWE-bench | 68.1% | Run by CWE-bench | Not reported | Closed |
| 3 | Claude Opus 5.5AnthropicRun by CWE-bench | 67.1% | Run by CWE-bench | Not reported | Closed |
| 4 | Muse WonderMetaRun by CWE-bench | 66.1% | Run by CWE-bench | Not reported | Closed |
| 5 | Gemini 4 ArgonGoogleRun by CWE-bench | 65.0% | Run by CWE-bench | Not reported | Closed |
| 6 | GPT-6 AstraOpenAIRun by CWE-bench | 64.2% | Run by CWE-bench | Not reported | Closed |
| 7 | Hy4TencentRun by CWE-bench | 64.1% | Run by CWE-bench | Not reported | Pending |
| 8 | DeepSeek V4.1 Flash 1017DeepSeekRun by CWE-bench | 57.6% | Run by CWE-bench | Not reported | Open |
| 9 | Grok 4.7xAIRun by CWE-bench | 54.7% | Run by CWE-bench | Not reported | Closed |
| 10 | Hy4 previewTencentRun by CWE-bench | 47.3% | Run by CWE-bench | Not reported | Open |
| 11 | Mistral Large 4MistralRun by CWE-bench | 45.6% | Run by CWE-bench | Not reported | Open |
| 12 | Qwen3.8 MaxAlibabaRun by CWE-bench | 45.4% | Run by CWE-bench | Not reported | Open |
| 13 | Kimi K3.1 ProMoonshot AIRun by CWE-bench | 45.3% | Run by CWE-bench | Not reported | Open |
| 14 | GLM-5.3Z.AIRun by CWE-bench | 42.0% | Run by CWE-bench | Not reported | Open |
- Lab-reported
- Benchmark-run
- Independent
CWE-Bench
CWE-Bench evaluates automated vulnerability patching. BenchLM stores exact provider-reported pass@1 results as display-only cybersecurity evidence until a stable benchmark-owned result artifact is available for the exact model setup.
| Source rank | Source model or configuration | Score | Who ran it | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash CyberGoogle | 47.2% | Reported by Google | CollinearOriginal result | Not reported | Closed |
CWE-bench version history
Every CWE-bench version BenchLM has recorded, newest first. Version 2 changes the task set (250 tasks, 185 weakness types, a public set of 100), so its pass@1 scores are not comparable with v1; ranks are the fairer way to follow a model across versions.
| Version | Released | Tasks | What changed | Leader | Models |
|---|---|---|---|---|---|
| CWE-bench v2Current | Nov 2, 2026 | 250 tasks (100 public, 150 held out) | Adds a public set of 100 tasks to 150 held-out ones and raises coverage from 73 to 185 weakness types. Pass@1 is over all 250 tasks. | LeaderClaude Fable 5.2 75%Official board | 14 models |
| CWE-bench v1 | Sep 25, 2026 | 120 held-out tasks | 120 held-out tasks across 73 weakness types, grouped by the OWASP Top 10. The source no longer shows this table; scores are as last captured on October 28, 2026. | LeaderGemini 4 Carbon 71%Official board | 12 models |
Rankings across CWE-bench versions
Each cell is the model’s rank among catalog models on that version’s leaderboard, with its pass@1 below. A dash means the model has no row on that version.
| Model | CWE-bench v214 models | CWE-bench v112 models |
|---|---|---|
| Claude Fable 5.2 | #175% | Not on this board |
| Gemini 4 Hydrogen | #268.1% | Not on this board |
| Claude Opus 5.5 | #367.1% | #567% |
| Muse Wonder | #466.1% | Not on this board |
| Gemini 4 Argon | #565% | #268% |
| GPT-6 Astra | #664.2% | #268% |
| Hy4 | #764.1% | Not on this board |
| DeepSeek V4.1 Flash 1017 | #857.6% | #855% |
| Grok 4.7 | #954.7% | #268% |
| Hy4 preview | #1047.3% | #1053% |
| Mistral Large 4 | #1145.6% | Not on this board |
| Qwen3.8 Max | #1245.4% | Not on this board |
| Kimi K3.1 Pro | #1345.3% | Not on this board |
| GLM-5.3 | #1442% | Not on this board |
| Gemini 4 Carbon | Not on this board | #171% |
| Claude Fable 5.1 | Not on this board | #658% |
| Grok 4.6 | Not on this board | #757% |
| Muse Spark 1.3 | Not on this board | #855% |
| GPT-6 Sol | Not on this board | #1152% |
| Inkling | Not on this board | #1237% |
Previous releaseReleased · 120 held-out tasks · Official leaderboard · Not comparable with v2
CWE-bench v1
The deterministic programmatic pass@1 score requires blocking the exploit while preserving existing tests. Models use high reasoning, four rollouts per task, and a one-hour limit per rollout. Version 1 uses 120 tasks and stays separate from v0, pass@4, and judge-panel scores.
| Source rank | Source model or configuration | Score | Who ran it | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 CarbonGoogle | 71.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 2 | GPT-6 AstraOpenAI | 68.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 3 | Gemini 4 ArgonGoogle | 68.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 4 | Grok 4.7xAI | 68.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 5 | Claude Opus 5.5Anthropic | 67.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 6 | Claude Fable 5.1Anthropic | 58.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 7 | Grok 4.6xAI | 57.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 8 | DeepSeek V4.1 Flash 1017DeepSeek | 55.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Open |
| 9 | Muse Spark 1.3Meta | 55.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 10 | Hy4 previewTencent | 53.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Open |
| 11 | GPT-6 SolOpenAI | 52.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Closed |
| 12 | InklingThinking Machines Lab | 37.0% | Run by CWE-bench v1 | Collinear AI · Original result | Not reported | Open |
Among the reported CWE-bench rows, Claude Fable 5.2 is first at 75.0%. The third row is 7.9 points behind. The broader top-10 range is 27.7 points, so the table still separates the published systems.
14 models have been evaluated on CWE-bench. The benchmark falls in the Agentic category. CWE-bench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About CWE-bench
Year
2026
Tasks
Audit and patch C/C++, Go, Java, TypeScript/JavaScript, Python, and Rust repositories; 100 public and 150 held-out tasks
Format
Pass@1
Difficulty
Frontier agent evaluation
The deterministic programmatic pass@1 score requires blocking the exploit while preserving existing tests, over all 250 tasks. Models use high reasoning, four rollouts per task, and a one-hour limit per rollout. Version 2 (November 2, 2026) adds a public set of 100 tasks and raises coverage from 73 to 185 weakness types, so its scores are not comparable with v1, whose last scores stay on the same page. Pass@4 and judge-panel scores are separate metrics.
CWE-bench is one of the cyber benchmarks we track. The cybersecurity benchmark comparison sets it beside the others, one table per security job.
Freshness and provenance
Version
CWE-bench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does CWE-bench measure?
Defensive vulnerability patching on 250 audit-and-patch tasks spanning 185 weakness types, 100 public and 150 held out.
Which model scores highest on CWE-bench?
Claude Fable 5.2 by Anthropic currently leads with a score of 75.0% on CWE-bench.
How many models are evaluated on CWE-bench?
14 AI models have published results on CWE-bench in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.