Software Engineering Benchmark Verified (SWE-bench Verified)
We show this table for reference; we do not rank on it.
Claude Opus 5 has the highest reported SWE-bench Verified score at 96%, ahead of Claude Mythos 5 (95.5%) and Claude Fable 5 (95%) among 116 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.
SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.
Benchmark score on SWE-bench Verified — October 27, 2026
We compile the SWE-bench Verified rows from provider self-reports and secondary reports. Claude Opus 5 leads the table at 96%, followed by Claude Mythos 5 (95.5%) and Claude Fable 5 (95%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
Claude Mythos 5
Anthropic
Claude Fable 5
Anthropic
116 modelsCodingRefreshingDisplay onlyUpdated October 27, 2026
API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.
Benchmark score table (116 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5Anthropic | 96% | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 2 | Claude Mythos 5Anthropic | 95.5% | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 3 | Claude Fable 5Anthropic | 95% | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 4 | Ember-1Fireworks | 92.2% | Not reported | Closed | $3 / $15 | 1.04M | Proprietary |
| 5 | Claude Opus 4.8Anthropic | 88.6% | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 6 | Claude Opus 4.7 (Adaptive)Anthropic | 87.6% | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 7 | Ornith-1.5-397BOrnith AI | 86% | Not reported | Open | Not recorded | 262K | MIT |
| 8 | Claude Sonnet 5Anthropic | 85.2% | Not reported | Closed | $2 / $10 | 1M | Proprietary |
| 9 | GPT-5.3 CodexOpenAI | 85% | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 10 | Ornith-1.0-397BDeepReinforce AI | 82.4% | Not reported | Open | Not recorded | 256K | MIT |
| 11 | Beam 1.0 501B-A23BReflection AI | 81.6% | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 12 | Claude Opus 4.5Anthropic | 80.9% | Not reported | Closed | $5 / $25 | 200K | Proprietary |
| 13 | Claude Opus 4.6Anthropic | 80.8% | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 14 | Beam 1.0 501B-A23B NVFP4Reflection AI | 80.7% | Not reported | Open | Not recorded | 1M | Not recorded |
| 15 | DeepSeek V4 Pro 0813DeepSeek | 80.6% | Not reported | Open | $1.32 / $3.96 | 1M | MIT |
| 16 | MiniMax M3MiniMax | 80.5% | Not reported | Open | $0.3 / $1.2 | 1M | MiniMax Community License |
| 17 | Qwen3.7 MaxAlibaba | 80.4% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 18 | Kimi K2.6Moonshot AI | 80.2% | Not reported | Open | $0.95 / $4 | 256K | Modified MIT |
| 19 | Inkling-SmallThinking Machines Lab | 80.2% | Not reported | Open | $0.58 / $1.44 | 1M | Not recorded |
| 20 | Beam 1.0 501B-A23B FP8Reflection AI | 80.2% | Not reported | Open | Not recorded | 1M | Not recorded |
| 21 | GPT-5.2OpenAI | 80% | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 22 | Claude Sonnet 4.6Anthropic | 79.6% | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 23 | DeepSeek V4 Pro (High)DeepSeek | 79.4% | Not reported | Open | Not recorded | 1M | Not recorded |
| 24 | DeepSeek V4 Flash 0731DeepSeek | 79% | Not reported | Open | $0.14 / $0.28 | 1M | MIT |
| 25 | Ornith-1.5-35B-A3BOrnith AI | 79% | Not reported | Open | Not recorded | 262K | Not recorded |
| 26 | Qwen3.6 PlusAlibaba | 78.8% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 27 | DeepSeek V4 Flash (High)DeepSeek | 78.6% | Not reported | Open | Not recorded | 1M | Not recorded |
| 28 | dots3-note PreviewDots Studio | 78.4% | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 29 | BTL-4Bad Theory Labs | 78.4% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 30 | MiMo-V2-ProXiaomi | 78% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 31 | GLM-5Z.AI | 77.8% | Not reported | Open | $1 / $3.2 | 200K | MIT |
| 32 | Qwen3.7 PlusAlibaba | 77.7% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 33 | Apodex 1.1Apodex | 77.7% | Not reported | Closed | Not recorded | N/A | Proprietary |
| 34 | InklingThinking Machines Lab | 77.6% | Not reported | Open | $1.87 / $4.68 | 1M | Apache-2.0 |
| 35 | Mistral Medium 3.5 128BMistral | 77.6% | Not reported | Open | $1.5 / $7.5 | 256K | Modified MIT |
| 36 | Muse SparkMeta | 77.4% | Not reported | Closed | Not recorded | 262K | Proprietary |
| 37 | Qwen3.6-27BAlibaba | 77.2% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 38 | Claude Sonnet 4.5Anthropic | 77.2% | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 39 | Kimi K2.5 (Reasoning)Moonshot AI | 76.8% | Not reported | Closed | $0.6 / $3 | 128K | Proprietary |
| 40 | Kimi K2.5Moonshot AI | 76.8% | Not reported | Open | $0.6 / $3 | 256K | Modified MIT |
| 41 | Grok 4.20xAI | 76.7% | Not reported | Closed | $1.25 / $2.5 | 2M | Proprietary |
| 42 | Qwen3.5 397BAlibaba | 76.2% | Not reported | Open | $0.6 / $3.6 | 128K | Apache-2.0 |
| 43 | Muse Glimmer 30BMeta | 76% | Not reported | Open | Not recorded | 131K | Apache-2.0 |
| 44 | Ornith-1.0-35BDeepReinforce AI | 75.6% | Not reported | Open | Not recorded | 256K | Not recorded |
| 45 | MiMo-V2-OmniXiaomi | 74.8% | Not reported | Closed | Not recorded | 262K | Proprietary |
| 46 | Laguna M.1Poolside | 74.6% | Not reported | Closed | Not recorded | 256K | Proprietary |
| 47 | Claude 4.1 OpusAnthropic | 74.5% | Not reported | Closed | $15 / $75 | 200K | Proprietary |
| 48 | Step 3.5 FlashStepFun | 74.4% | Not reported | Open | $0.1 / $0.3 | 256K | Apache-2.0 |
| 49 | Hy3 PreviewTencent | 74.4% | Not reported | Open | Not recorded | 256K | Not recorded |
| 50 | FlintRecursive | 74.2% | Not reported | Open | Not recorded | 256K | Apache-2.0 |
| 51 | GLM-4.7Z.AI | 73.8% | Not reported | Open | Not recorded | 200K | Not recorded |
| 52 | DeepSeek V4 FlashDeepSeek | 73.7% | Not reported | Open | Not recorded | 1M | Not recorded |
| 53 | DeepSeek V4 ProDeepSeek | 73.6% | Not reported | Open | Not recorded | 1M | Not recorded |
| 54 | MAI-Thinking-1Microsoft | 73.5% | Not reported | Closed | Not recorded | 256K | Proprietary |
| 55 | Qwen3.6-35B-A3BAlibaba | 73.4% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 56 | MiMo-V2-FlashXiaomi | 73.4% | Not reported | Open | Not recorded | 256K | MIT |
| 57 | Claude Haiku 4.5Anthropic | 73.3% | Not reported | Closed | $1 / $5 | 200K | Proprietary |
| 58 | Jais-3-MoVA-36B-A4BIFM | 72.8% | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 59 | Claude 4 SonnetAnthropic | 72.7% | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 60 | MAI-Code-1.1-FlashMicrosoft | 72.6% | Not reported | Closed | Not recorded | 256K | Proprietary |
| 61 | Claude 4 OpusAnthropic | 72.5% | Not reported | Closed | Not recorded | 200K | Proprietary |
| 62 | Qwen3.5-27BAlibaba | 72.4% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 63 | Jais-3-32BIFM | 72.4% | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 64 | Qwen3.5-122B-A10BAlibaba | 72% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 65 | Nemotron 3 UltraNVIDIA | 71.9% | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 66 | Nemotron 3.5 Super VL 120B A12BNVIDIA | 71.1% | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 67 | Nemotron 3.5 Super VL 120B A12B NVFP4NVIDIA | 71.1% | Not reported | Open | Not recorded | 1M | Not recorded |
| 68 | Laguna XS 2.1Poolside | 70.9% | Not reported | Open | $0.1 / $0.2 | 262K | Not recorded |
| 69 | Grok Code Fast 1xAI | 70.8% | Not reported | Closed | $0.2 / $1.5 | 256K | Proprietary |
| 70 | Ornith-1.5-9BOrnith AI | 70.6% | Not reported | Open | Not recorded | 262K | Not recorded |
| 71 | Solar Pro 4Upstage | 70.6% | Not reported | Closed | $0.3 / $1.2 | 512K | Proprietary |
| 72 | Solar Open 2Upstage | 70.4% | Not reported | Open | Not recorded | 1M | Upstage Solar License |
| 73 | Jais-3-375B-A23BIFM | 70.2% | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 74 | Jais-3-7BIFM | 70.2% | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 75 | Laguna XS.2Poolside | 69.9% | Not reported | Open | Not recorded | 256K | Not recorded |
| 76 | Qwen3-Coder 480B-A35BAlibaba | 69.6% | Not reported | Open | Not recorded | 256K | Not recorded |
| 77 | Ornith-1.0-9BDeepReinforce AI | 69.4% | Not reported | Open | Not recorded | 256K | Not recorded |
| 78 | Qwen3.5-35B-A3BAlibaba | 69.2% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 79 | LongCat-Flash-Lite-SparseMeituan | 68.2% | Not reported | Open | Not recorded | 1M | MIT |
| 80 | K-EXAONE 2.0LG AI Research | 68.2% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 81 | DeepSeek V3.1DeepSeek | 66% | Not reported | Open | Not recorded | 128K | Not recorded |
| 82 | Kimi K2Moonshot AI | 65.8% | Not reported | Closed | $0.6 / $2.5 | 128K | Proprietary |
| 83 | Gemini 2.5 ProGoogle | 63.8% | Not reported | Closed | $1.25 / $10 | 1M | Proprietary |
| 84 | Spark-X2.6-4BSparkLLM | 62.8% | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 85 | GPT-OSS 120BOpenAI | 62.4% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 86 | Claude 3.7 SonnetAnthropic | 62.3% | Not reported | Closed | Not recorded | 200K | Proprietary |
| 87 | Ternary Bonsai 2 27BPrism ML | 60.8% | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 88 | GPT-OSS 20BOpenAI | 60.7% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 89 | Nemotron 3 Super 100BNVIDIA | 60.5% | Not reported | Open | Not recorded | 1M | Not recorded |
| 90 | GLM-4.7-FlashZ.AI | 59.2% | Not reported | Open | Not recorded | 200K | MIT |
| 91 | DeepSeek-R1-0528DeepSeek | 57.6% | Not reported | Open | Not recorded | 128K | Not recorded |
| 92 | Granite 4.2 30BIBM | 57% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 93 | MiniMax M1 80kMiniMax | 56% | Not reported | Closed | Not recorded | 80K | Proprietary |
| 94 | GPT-4.1OpenAI | 54.6% | Not reported | Closed | $2 / $8 | 1M | Proprietary |
| 95 | ZAYA1-74B-PreviewZyphra | 53.2% | Not reported | Open | Not recorded | 256K | Apache-2.0 |
| 96 | Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIA | 52.8% | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 97 | LFM2.5-24B-A2BLiquidAI | 51% | Not reported | Open | $0.1 / $1 | 131K | LFM Open License v1.0 |
| 98 | K-ExaoneLG AI Research | 49.4% | Not reported | Closed | Not recorded | 256K | Proprietary |
| 99 | o3-miniOpenAI | 49.3% | Not reported | Closed | $1.1 / $4.4 | 200K | Proprietary |
| 100 | LLaDA2.2-flashInclusionAI | 49.3% | Not reported | Open | Not recorded | 128K | Not recorded |
| 101 | DeepSeek-R1DeepSeek | 49.2% | Not reported | Open | $0.55 / $2.19 | 128K | MIT |
| 102 | Claude 3.5 SonnetAnthropic | 49% | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 103 | Granite 4.2 8BIBM | 47.7% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 104 | Mellum2.1-12B-A2.5B-ThinkingJetBrains | 47% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 105 | Devstral SmallMistral | 46.8% | Not reported | Open | Not recorded | 128K | Not recorded |
| 106 | MiniCPM5-2BOpenBMB | 46.4% | Not reported | Open | Not recorded | 131K | Apache-2.0 |
| 107 | Sarvam 105BSarvam | 45% | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 108 | LFM3-1.7BLiquidAI | 43.4% | Not reported | Open | Not recorded | 256K | LFM Open License v1.0 |
| 109 | DeepSeek V3DeepSeek | 42% | Not reported | Open | $0.27 / $1.1 | 128K | MIT |
| 110 | Claude 3.5 HaikuAnthropic | 40.6% | Not reported | Closed | Not recorded | 200K | Proprietary |
| 111 | Sarvam 30BSarvam | 34% | Not reported | Open | Not recorded | 64K | Apache-2.0 |
| 112 | Gemini 2.5 Flash-LiteGoogle | 31.6% | Not reported | Closed | $0.1 / $0.4 | 1M | Proprietary |
| 113 | GPT-4.1 miniOpenAI | 23.6% | Not reported | Closed | $0.4 / $1.6 | 1M | Proprietary |
| 114 | Gemini 2.0 FlashGoogle | 21.4% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 115 | Gemini 2.0 Flash-LiteGoogle | 12.5% | Not reported | Closed | Not recorded | 1M | Proprietary |
| 116 | Gemini 1.5 FlashGoogle | 9.6% | Not reported | Closed | Not recorded | 1M | Proprietary |
SWE-bench Verified*
BenchLM stores this chart-specific SWE-bench Verified row separately because Arcee notes all models were evaluated in mini-swe-agent-v2.
| Source rank | Source model or configuration | Score | Source setup | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6Anthropic | 75.6% | Arcee AIOriginal result | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 2 | MiniMax M2.7MiniMax | 75.4% | Arcee AIOriginal result | Not reported | Open | $0.3 / $1.2 | 200K | MiniMax License |
| 3 | GLM-5Z.AI | 72.8% | Arcee AIOriginal result | Not reported | Open | $1 / $3.2 | 200K | MIT |
| 4 | Kimi K2.5Moonshot AI | 70.8% | Arcee AIOriginal result | Not reported | Open | $0.6 / $3 | 256K | Modified MIT |
| 5 | Trinity-Large-ThinkingArcee AI | 63.2% | Arcee AIOriginal result | Not reported | Open | $0.25 / $0.9 | 512K | OpenMDW-1.1 |
How to read this leaderboard
SWE-bench Verified is a 500-task subset of the original SWE-bench repository-repair benchmark. The official leaderboard separates agent submissions from the Bash Only view, where models use the same mini-SWE-agent environment.
Variant check: Verified, Lite, Multilingual, and Pro use different task sets. Keep the benchmark version, scaffold, tool budget, and retry policy with the score. A higher percentage on another version is not a like-for-like improvement.
Use this table to inspect published repository-repair results. We moved SWE-bench Verified to display only in BenchAlign v5.6; its published percentages do not contribute to the current public ranking. Pair it with newer repository tasks and a test on your own codebase.
Operator receipt: 116 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 96%.
Honest limit: Provider-reported scores can use different agents and budgets. Catalog API prices and context limits describe the model, not the cost or context used in a benchmark run. Missing setup fields remain unreported.
Among the reported SWE-bench Verified rows, Claude Opus 5 is first at 96%. The third row is 1.0 points behind. The broader top-10 range is 13.6 points, so the table still separates the published systems.
116 models have been evaluated on SWE-bench Verified. The benchmark falls in the Coding category. SWE-bench Verified is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings. We moved it to display only in BenchAlign v5.6 because its tests are contaminated and saturated.
About SWE-bench Verified
Year
2024
Tasks
500 verified issues
Format
Code patch generation
Difficulty
Professional software engineering
SWE-bench Verified is the most-cited benchmark for AI coding agents on real repository tasks. Each task requires understanding codebases, writing patches, and passing test suites.
When scores here cluster at the top, the DeepSWE leaderboard spreads coding agents apart again, on original tasks drawn from active open-source repositories.
Freshness and provenance
Version
SWE-bench Verified 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-bench Verified measure?
SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.
Which model scores highest on SWE-bench Verified?
Claude Opus 5 by Anthropic currently leads with a score of 96% on SWE-bench Verified.
How many models are evaluated on SWE-bench Verified?
116 AI models have published results on SWE-bench Verified in the BenchLM catalog.
Compare top models on SWE-bench Verified
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.