Software Engineering Benchmark Verified (SWE-bench Verified)
We show this table for reference; we do not rank on it.
Claude Opus 5 has the highest reported SWE-bench Verified score at 96%, ahead of Claude Mythos 5 (95.5%) and Claude Fable 5 (95%) among 94 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.
A curated, human-verified subset of SWE-bench that tests models on resolving real GitHub issues from popular open-source Python repositories like Django, Flask, and scikit-learn.
Benchmark score on SWE-bench Verified — October 10, 2026
We compile the SWE-bench Verified rows from provider self-reports and secondary reports. Claude Opus 5 leads the table at 96%, followed by Claude Mythos 5 (95.5%) and Claude Fable 5 (95%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
Claude Mythos 5
Anthropic
Claude Fable 5
Anthropic
94 modelsCodingRefreshingDisplay onlyUpdated October 10, 2026
Benchmark score table (94 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 5Anthropic | 96% | Not reported | Closed |
| 2 | Claude Mythos 5Anthropic | 95.5% | Not reported | Closed |
| 3 | Claude Fable 5Anthropic | 95% | Not reported | Closed |
| 4 | Ember-1Fireworks | 92.2% | Not reported | Closed |
| 5 | Claude Opus 4.8Anthropic | 88.6% | Not reported | Closed |
| 6 | Claude Opus 4.7 (Adaptive)Anthropic | 87.6% | Not reported | Closed |
| 7 | Ornith-1.5-397BOrnith AI | 86% | Not reported | Open |
| 8 | Claude Sonnet 5Anthropic | 85.2% | Not reported | Closed |
| 9 | GPT-5.3 CodexOpenAI | 85% | Not reported | Closed |
| 10 | Ornith-1.0-397BDeepReinforce AI | 82.4% | Not reported | Open |
| 11 | Claude Opus 4.5Anthropic | 80.9% | Not reported | Closed |
| 12 | BeamReflection AI | 80.9% | Not reported | Pending |
| 13 | Claude Opus 4.6Anthropic | 80.8% | Not reported | Closed |
| 14 | DeepSeek V4 Pro 0813DeepSeek | 80.6% | Not reported | Open |
| 15 | MiniMax M3MiniMax | 80.5% | Not reported | Open |
| 16 | Qwen3.7 MaxAlibaba | 80.4% | Not reported | Closed |
| 17 | Kimi K2.6Moonshot AI | 80.2% | Not reported | Open |
| 18 | Inkling-SmallThinking Machines Lab | 80.2% | Not reported | Open |
| 19 | GPT-5.2OpenAI | 80% | Not reported | Closed |
| 20 | Claude Sonnet 4.6Anthropic | 79.6% | Not reported | Closed |
| 21 | DeepSeek V4 Pro (High)DeepSeek | 79.4% | Not reported | Open |
| 22 | DeepSeek V4 Flash 0731DeepSeek | 79% | Not reported | Open |
| 23 | Ornith-1.5-35B-A3BOrnith AI | 79% | Not reported | Open |
| 24 | Qwen3.6 PlusAlibaba | 78.8% | Not reported | Closed |
| 25 | DeepSeek V4 Flash (High)DeepSeek | 78.6% | Not reported | Open |
| 26 | dots3-note PreviewDots Studio | 78.4% | Not reported | Open |
| 27 | BTL-4Bad Theory Labs | 78.4% | Not reported | Open |
| 28 | MiMo-V2-ProXiaomi | 78% | Not reported | Closed |
| 29 | GLM-5Z.AI | 77.8% | Not reported | Open |
| 30 | Qwen3.7 PlusAlibaba | 77.7% | Not reported | Closed |
| 31 | Apodex 1.1Apodex | 77.7% | Not reported | Closed |
| 32 | InklingThinking Machines Lab | 77.6% | Not reported | Open |
| 33 | Mistral Medium 3.5 128BMistral | 77.6% | Not reported | Open |
| 34 | Muse SparkMeta | 77.4% | Not reported | Closed |
| 35 | Qwen3.6-27BAlibaba | 77.2% | Not reported | Open |
| 36 | Claude Sonnet 4.5Anthropic | 77.2% | Not reported | Closed |
| 37 | Kimi K2.5 (Reasoning)Moonshot AI | 76.8% | Not reported | Closed |
| 38 | Kimi K2.5Moonshot AI | 76.8% | Not reported | Open |
| 39 | Grok 4.20xAI | 76.7% | Not reported | Closed |
| 40 | Qwen3.5 397BAlibaba | 76.2% | Not reported | Open |
| 41 | Muse Glimmer 30BMeta | 76% | Not reported | Open |
| 42 | Ornith-1.0-35BDeepReinforce AI | 75.6% | Not reported | Open |
| 43 | MiMo-V2-OmniXiaomi | 74.8% | Not reported | Closed |
| 44 | Laguna M.1Poolside | 74.6% | Not reported | Closed |
| 45 | Claude 4.1 OpusAnthropic | 74.5% | Not reported | Closed |
| 46 | Step 3.5 FlashStepFun | 74.4% | Not reported | Open |
| 47 | Hy3 PreviewTencent | 74.4% | Not reported | Open |
| 48 | GLM-4.7Z.AI | 73.8% | Not reported | Open |
| 49 | DeepSeek V4 FlashDeepSeek | 73.7% | Not reported | Open |
| 50 | DeepSeek V4 ProDeepSeek | 73.6% | Not reported | Open |
| 51 | MAI-Thinking-1Microsoft | 73.5% | Not reported | Closed |
| 52 | Qwen3.6-35B-A3BAlibaba | 73.4% | Not reported | Open |
| 53 | MiMo-V2-FlashXiaomi | 73.4% | Not reported | Open |
| 54 | Claude Haiku 4.5Anthropic | 73.3% | Not reported | Closed |
| 55 | Claude 4 SonnetAnthropic | 72.7% | Not reported | Closed |
| 56 | MAI-Code-1.1-FlashMicrosoft | 72.6% | Not reported | Closed |
| 57 | Qwen3.5-27BAlibaba | 72.4% | Not reported | Open |
| 58 | Qwen3.5-122B-A10BAlibaba | 72% | Not reported | Open |
| 59 | Nemotron 3 UltraNVIDIA | 71.9% | Not reported | Open |
| 60 | Laguna XS 2.1Poolside | 70.9% | Not reported | Open |
| 61 | Grok Code Fast 1xAI | 70.8% | Not reported | Closed |
| 62 | Ornith-1.5-9BOrnith AI | 70.6% | Not reported | Open |
| 63 | Solar Pro 4Upstage | 70.6% | Not reported | Closed |
| 64 | Solar Open 2Upstage | 70.4% | Not reported | Open |
| 65 | Laguna XS.2Poolside | 69.9% | Not reported | Open |
| 66 | Ornith-1.0-9BDeepReinforce AI | 69.4% | Not reported | Open |
| 67 | Qwen3.5-35B-A3BAlibaba | 69.2% | Not reported | Open |
| 68 | LongCat-Flash-Lite-SparseMeituan | 68.2% | Not reported | Open |
| 69 | K-EXAONE 2.0LG AI Research | 68.2% | Not reported | Open |
| 70 | DeepSeek V3.1DeepSeek | 66% | Not reported | Open |
| 71 | Kimi K2Moonshot AI | 65.8% | Not reported | Closed |
| 72 | Gemini 2.5 ProGoogle | 63.8% | Not reported | Closed |
| 73 | GPT-OSS 120BOpenAI | 62.4% | Not reported | Open |
| 74 | Ternary Bonsai 2 27BPrism ML | 60.8% | Not reported | Open |
| 75 | GPT-OSS 20BOpenAI | 60.7% | Not reported | Open |
| 76 | Nemotron 3 Super 100BNVIDIA | 60.5% | Not reported | Open |
| 77 | GLM-4.7-FlashZ.AI | 59.2% | Not reported | Open |
| 78 | Granite 4.2 30BIBM | 57% | Not reported | Open |
| 79 | MiniMax M1 80kMiniMax | 56% | Not reported | Closed |
| 80 | GPT-4.1OpenAI | 54.6% | Not reported | Closed |
| 81 | ZAYA1-74B-PreviewZyphra | 53.2% | Not reported | Open |
| 82 | Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIA | 52.8% | Not reported | Open |
| 83 | K-ExaoneLG AI Research | 49.4% | Not reported | Closed |
| 84 | o3-miniOpenAI | 49.3% | Not reported | Closed |
| 85 | LLaDA2.2-flashInclusionAI | 49.3% | Not reported | Open |
| 86 | DeepSeek-R1DeepSeek | 49.2% | Not reported | Open |
| 87 | Claude 3.5 SonnetAnthropic | 49% | Not reported | Closed |
| 88 | Granite 4.2 8BIBM | 47.7% | Not reported | Open |
| 89 | Mellum2.1-12B-A2.5B-ThinkingJetBrains | 47% | Not reported | Open |
| 90 | MiniCPM5-2BOpenBMB | 46.4% | Not reported | Open |
| 91 | Sarvam 105BSarvam | 45% | Not reported | Open |
| 92 | DeepSeek V3DeepSeek | 42% | Not reported | Open |
| 93 | Sarvam 30BSarvam | 34% | Not reported | Open |
| 94 | GPT-4.1 miniOpenAI | 23.6% | Not reported | Closed |
SWE-bench Verified*
BenchLM stores this chart-specific SWE-bench Verified row separately because Arcee notes all models were evaluated in mini-swe-agent-v2.
| Source rank | Source model or configuration | Score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.6Anthropic | 75.6% | Arcee AIOriginal result | Not reported | Closed |
| 2 | MiniMax M2.7MiniMax | 75.4% | Arcee AIOriginal result | Not reported | Open |
| 3 | GLM-5Z.AI | 72.8% | Arcee AIOriginal result | Not reported | Open |
| 4 | Kimi K2.5Moonshot AI | 70.8% | Arcee AIOriginal result | Not reported | Open |
| 5 | Trinity-Large-ThinkingArcee AI | 63.2% | Arcee AIOriginal result | Not reported | Open |
Among the reported SWE-bench Verified rows, Claude Opus 5 is first at 96%. The third row is 1.0 points behind. The broader top-10 range is 13.6 points, so the table still separates the published systems.
94 models have been evaluated on SWE-bench Verified. The benchmark falls in the Coding category. SWE-bench Verified is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings. We moved it to display only in BenchAlign v5.6 because its tests are contaminated and saturated.
About SWE-bench Verified
Year
2024
Tasks
500 verified issues
Format
Code patch generation
Difficulty
Professional software engineering
SWE-bench Verified is the most-cited benchmark for AI coding agents on real repository tasks. Each task requires understanding codebases, writing patches, and passing test suites.
When scores here cluster at the top, the DeepSWE leaderboard spreads coding agents apart again, on original tasks drawn from active open-source repositories.
Freshness and provenance
Version
SWE-bench Verified 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-bench Verified measure?
A curated, human-verified subset of SWE-bench that tests models on resolving real GitHub issues from popular open-source Python repositories like Django, Flask, and scikit-learn.
Which model scores highest on SWE-bench Verified?
Claude Opus 5 by Anthropic currently leads with a score of 96% on SWE-bench Verified.
How many models are evaluated on SWE-bench Verified?
94 AI models have published results on SWE-bench Verified in the BenchLM catalog.
Compare top models on SWE-bench Verified
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,500+ readers.
One email each week. Unsubscribe anytime.