Artificial Analysis CyberGym-E2E-AA (CyberGym-E2E-AA)
We show this table for reference; we do not rank on it.
Artificial Analysis' implementation of Berkeley RDI's CyberGym-E2E: agents find a real memory-safety vulnerability in a C/C++ open-source project, write a proof-of-concept input that crashes it, and patch it without breaking the project's tests.
Benchmark score on CyberGym-E2E-AA — October 29, 2026
We compile the CyberGym-E2E-AA rows from secondary reports. Kimi K3.1 Pro leads the table at 94.7%, followed by Mistral Large 4 (81.7%) and MiMo-V2.6-Pro (78.6%). We do not use these results to rank models overall.
Kimi K3.1 Pro
Moonshot AI
Mistral Large 4
Mistral
MiMo-V2.6-Pro
Xiaomi
19 modelsAgenticCurrentDisplay onlyUpdated October 29, 2026
Benchmark score table (19 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AI | 94.7% | Not reported | Pending |
| 2 | Mistral Large 4Mistral | 81.7% | Not reported | Open |
| 3 | MiMo-V2.6-ProXiaomi | 78.6% | Not reported | Open |
| 4 | GPT-6 LunaOpenAI | 77.9% | Not reported | Closed |
| 5 | Step 5 PreviewStepFun | 74.8% | Not reported | Open |
| 6 | GLM-5.3-FlashZ.AI | 74.0% | Not reported | Open |
| 7 | Grok 4.7xAI | 74.0% | Not reported | Closed |
| 8 | Kimi K3Moonshot AI | 58.0% | Not reported | Open |
| 9 | Beam 1.0 501B-A23BReflection AI | 45.0% | Not reported | Open |
| 10 | GLM-5.3Z.AI | 29.0% | Not reported | Open |
| 11 | MiniMax M3MiniMax | 24.4% | Not reported | Open |
| 12 | DeepSeek V4.1 Flash 1017DeepSeek | 22.9% | Not reported | Open |
| 13 | Nemotron 3 UltraNVIDIA | 10.7% | Not reported | Open |
| 14 | Gemini 3.8 FlashGoogle | 8.4% | Not reported | Closed |
| 15 | Claude Fable 5.1Anthropic | 0.8% | Not reported | Closed |
| 16 | Claude Opus 5.5Anthropic | 0.8% | Not reported | Closed |
| 17 | GPT-6 AstraOpenAI | 0.0% | Not reported | Closed |
| 18 | GPT-6 SolOpenAI | 0.0% | Not reported | Closed |
| 19 | Qwen3.8-27BAlibaba | 0.0% | Not reported | Open |
Among the reported CyberGym-E2E-AA rows, Kimi K3.1 Pro is first at 94.7%. The third row is 16.1 points behind. The broader top-10 range is 65.7 points, so the table still separates the published systems.
19 models have been evaluated on CyberGym-E2E-AA. The benchmark falls in the Agentic category. CyberGym-E2E-AA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About CyberGym-E2E-AA
Year
2026
Tasks
131 end-to-end vulnerability reproduction and patching tasks
Format
Pass@1
Difficulty
Real-world cybersecurity
131 tasks, one per OSS-Fuzz project, 90 minutes each in the Stirrup harness. Separate from the provider-reported CyberGym rows. Independently run by Artificial Analysis under its published harness. Display-only; not yet admitted as independent-run evidence.
Freshness and provenance
Version
CyberGym-E2E-AA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does CyberGym-E2E-AA measure?
Artificial Analysis' implementation of Berkeley RDI's CyberGym-E2E: agents find a real memory-safety vulnerability in a C/C++ open-source project, write a proof-of-concept input that crashes it, and patch it without breaking the project's tests.
Which model scores highest on CyberGym-E2E-AA?
Kimi K3.1 Pro by Moonshot AI currently leads with a score of 94.7% on CyberGym-E2E-AA.
How many models are evaluated on CyberGym-E2E-AA?
19 AI models have published results on CyberGym-E2E-AA in the BenchLM catalog.
Compare top models on CyberGym-E2E-AA
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.