Artificial Analysis DeepsecBench-AA (DeepsecBench-AA)
We show this table for reference; we do not rank on it.
Artificial Analysis' implementation of Vercel's DeepsecBench, testing agents on finding security vulnerabilities in real web application code. One of the three components of the AA Cyber Index.
Benchmark score on DeepsecBench-AA — October 29, 2026
We compile the DeepsecBench-AA rows from secondary reports. GPT-6 Sol leads the table at 46.3%, followed by GPT-6 Astra (36.9%) and Kimi K3.1 Pro (36.7%). We do not use these results to rank models overall.
GPT-6 Sol
OpenAI
GPT-6 Astra
OpenAI
Kimi K3.1 Pro
Moonshot AI
21 modelsAgenticCurrentDisplay onlyUpdated October 29, 2026
Benchmark score table (21 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 SolOpenAI | 46.3% | Not reported | Closed |
| 2 | GPT-6 AstraOpenAI | 36.9% | Not reported | Closed |
| 3 | Kimi K3.1 ProMoonshot AI | 36.7% | Not reported | Pending |
| 4 | DeepSeek V4.1 Flash 1017DeepSeek | 31.2% | Not reported | Open |
| 5 | Claude Opus 5.5Anthropic | 26.9% | Not reported | Closed |
| 6 | Grok 4.7xAI | 26.9% | Not reported | Closed |
| 7 | MiMo-V2.6-ProXiaomi | 26.2% | Not reported | Open |
| 8 | Claude Opus 5Anthropic | 25.9% | Not reported | Closed |
| 9 | GPT-5.6 TerraOpenAI | 25.1% | Not reported | Closed |
| 10 | GLM-5.3Z.AI | 24.3% | Not reported | Open |
| 11 | Claude Fable 5.1Anthropic | 23.5% | Not reported | Closed |
| 12 | GPT-6 LunaOpenAI | 23.5% | Not reported | Closed |
| 13 | GLM-5.3-FlashZ.AI | 19.9% | Not reported | Open |
| 14 | Mistral Large 4Mistral | 16.0% | Not reported | Open |
| 15 | Step 5 PreviewStepFun | 15.8% | Not reported | Open |
| 16 | Gemini 3.8 FlashGoogle | 14.5% | Not reported | Closed |
| 17 | Kimi K3Moonshot AI | 12.5% | Not reported | Open |
| 18 | Qwen3.8-27BAlibaba | 11.5% | Not reported | Open |
| 19 | Beam 1.0 501B-A23BReflection AI | 6.9% | Not reported | Open |
| 20 | MiniMax M3MiniMax | 1.1% | Not reported | Open |
| 21 | Nemotron 3 UltraNVIDIA | 1.0% | Not reported | Open |
Among the reported DeepsecBench-AA rows, GPT-6 Sol is first at 46.3%. The third row is 9.6 points behind. The broader top-10 range is 22.0 points, so the table still separates the published systems.
21 models have been evaluated on DeepsecBench-AA. The benchmark falls in the Agentic category. DeepsecBench-AA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About DeepsecBench-AA
Year
2026
Tasks
Vulnerability discovery in real codebases
Format
Pass@1
Difficulty
Security code review
Independently run by Artificial Analysis under its published harness. Display-only; not yet admitted as independent-run evidence.
Freshness and provenance
Version
DeepsecBench-AA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does DeepsecBench-AA measure?
Artificial Analysis' implementation of Vercel's DeepsecBench, testing agents on finding security vulnerabilities in real web application code. One of the three components of the AA Cyber Index.
Which model scores highest on DeepsecBench-AA?
GPT-6 Sol by OpenAI currently leads with a score of 46.3% on DeepsecBench-AA.
How many models are evaluated on DeepsecBench-AA?
21 AI models have published results on DeepsecBench-AA in the BenchLM catalog.
Compare top models on DeepsecBench-AA
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.