Skip to main content
BenchLM

CWE-bench v1

We show this table for reference; we do not rank on it.

Data verified 37 confirmed releases in the last 30 daysFollow model changes

Defensive vulnerability patching on 120 held-out tasks spanning 73 weakness types.

Benchmark score on CWE-bench v1 — September 30, 2026

We compile the CWE-bench v1 rows from benchmark-owner or independent runs. GPT-6 Astra leads the table at 68.0%, followed by Gemini 4 Argon (68.0%) and Grok 4.7 (68.0%). We do not use these results to rank models overall.

1Proprietary

GPT-6 Astra

OpenAI

68.0%
Overall 88.69Context 1.05M
2Proprietary

Gemini 4 Argon

Google

68.0%
Overall 77.18
3Proprietary

Grok 4.7

xAI

68.0%
Context 500K

11 modelsAgenticCurrentDisplay onlyUpdated September 30, 2026

Benchmark score table (11 models)

Score
1
GPT-6 AstraOpenAI · Closed
68.0%
2
Gemini 4 ArgonGoogle · Closed
68.0%
3
Grok 4.7xAI · Closed
68.0%
4
Claude Opus 5.5Anthropic · Closed
67.0%
5
Claude Fable 5.1Anthropic · Closed
58.0%
6
Grok 4.6xAI · Closed
57.0%
7
Muse Spark 1.3Meta · Closed
55.0%
8
DeepSeek V4.1 FlashDeepSeek · Open weight
55.0%
9
Hy4 previewTencent · Open weight
53.0%
10
GPT-6 SolOpenAI · Closed
52.0%
11
InklingThinking Machines Lab · Open weight
37.0%

Among the reported CWE-bench v1 rows, GPT-6 Astra is first at 68.0%. The third row is 0.0 points behind. The broader top-10 range is 16.0 points, so the table still separates the published systems.

11 models have been evaluated on CWE-bench v1. The benchmark falls in the Agentic category. CWE-bench v1 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About CWE-bench v1

Year

2026

Tasks

Audit and patch held-out C/C++, Go, Java, TypeScript/JavaScript, Python, and Rust repositories

Format

Pass@1

Difficulty

Frontier agent evaluation

The deterministic programmatic pass@1 score requires blocking the exploit while preserving existing tests. Models use high reasoning, four rollouts per task, and a one-hour limit per rollout. Version 1 uses 120 tasks and stays separate from v0, pass@4, and judge-panel scores.

CWE-bench v1 leaderboard and methodology

Freshness and provenance

Version

CWE-bench v1 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does CWE-bench v1 measure?

Defensive vulnerability patching on 120 held-out tasks spanning 73 weakness types.

Which model scores highest on CWE-bench v1?

GPT-6 Astra by OpenAI currently leads with a score of 68.0% on CWE-bench v1.

How many models are evaluated on CWE-bench v1?

11 AI models have been evaluated on CWE-bench v1 on BenchLM.

Compare top models on CWE-bench v1

GPT-6 Astra vs Gemini 4 ArgonGemini 4 Argon vs Grok 4.7Grok 4.7 vs Claude Opus 5.5Claude Opus 5.5 vs Claude Fable 5.1
Last updated: September 30, 2026 · BenchLM version CWE-bench v1 2026

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.