Software Engineering Benchmark Verified (SWE-bench Verified)
We show this table for reference; we do not rank on it.
Claude Opus 5 has the highest reported SWE-bench Verified score at 96%, ahead of Claude Mythos 5 (95.5%) and Claude Fable 5 (95%) among 117 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.
SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.
Benchmark score on SWE-bench Verified — November 21, 2026
We compile the SWE-bench Verified rows from provider self-reports and secondary reports. Claude Opus 5 leads the table at 96%, followed by Claude Mythos 5 (95.5%) and Claude Fable 5 (95%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
Reported by AnthropicClaude Mythos 5
Anthropic
Reported by AnthropicClaude Fable 5
Anthropic
Reported by Anthropic117 modelsCodingRefreshingDisplay onlyUpdated November 21, 2026
API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.
Benchmark score table (117 models)
None of the 117 results on this page has a documented run by the benchmark or an independent evaluator: 111 reported by the labs that built the models, 1 reported in another publisher’s document and 5 carried from an upstream page, where who ran them is not recorded.
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5AnthropicReported by Anthropic | 96% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 2 | Claude Mythos 5AnthropicReported by Anthropic | 95.5% | Reported by Anthropic | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 3 | Claude Fable 5AnthropicReported by Anthropic | 95% | Reported by Anthropic | Not reported | Closed | $10 / $50 | 1M+ | Proprietary |
| 4 | Ember-1FireworksReported by Fireworks | 92.2% | Reported by Fireworks | Not reported | Closed | $3 / $15 | 1.04M | Proprietary |
| 5 | Claude Opus 4.8AnthropicReported by Anthropic | 88.6% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 6 | Claude Opus 4.7AnthropicReported by Anthropic | 87.6% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 7 | Ornith-1.5-397BOrnith AIReported by Ornith AI | 86% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | MIT |
| 8 | Claude Sonnet 5AnthropicReported by Anthropic | 85.2% | Reported by Anthropic | Not reported | Closed | $2 / $10 | 1M | Proprietary |
| 9 | GPT-5.3 CodexOpenAIReported by OpenAI | 85% | Reported by OpenAI | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 10 | Ornith-1.0-397BDeepReinforce AIReported by DeepReinforce AI | 82.4% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | MIT |
| 11 | Inkling 2 SmallThinking Machines LabReported by Thinking Machines Lab | 82.2% | Reported by Thinking Machines Lab | Not reported | Closed | $0.3 / $1 | 1M | Proprietary |
| 12 | Beam 1.0 501B-A23BReflection AIReported by Reflection AI | 81.6% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 13 | Claude Opus 4.5AnthropicReported by Anthropic | 80.9% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 200K | Proprietary |
| 14 | Claude Opus 4.6AnthropicReported by Anthropic | 80.8% | Reported by Anthropic | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 15 | Beam 1.0 501B-A23B NVFP4Reflection AIReported by Reflection AI | 80.7% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Not recorded |
| 16 | DeepSeek V4 ProDeepSeekReported by DeepSeek | 80.6% | Reported by DeepSeek | Not reported | Open | $1.32 / $3.96 | 1M | MIT |
| 17 | MiniMax M3MiniMaxReported by MiniMax | 80.5% | Reported by MiniMax | Not reported | Open | $0.3 / $1.2 | 1M | MiniMax Community License |
| 18 | Qwen3.7 MaxAlibabaReported by Alibaba | 80.4% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 19 | Kimi K2.6Moonshot AIReported by Moonshot AI | 80.2% | Reported by Moonshot AI | Not reported | Open | $0.95 / $4 | 256K | Modified MIT |
| 20 | Inkling-SmallThinking Machines LabReported by Thinking Machines Lab | 80.2% | Reported by Thinking Machines Lab | Not reported | Open | $0.58 / $1.44 | 1M | Not recorded |
| 21 | Beam 1.0 501B-A23B FP8Reflection AIReported by Reflection AI | 80.2% | Reported by Reflection AI | Not reported | Open | Not recorded | 1M | Not recorded |
| 22 | GPT-5.2OpenAIReported by OpenAI | 80% | Reported by OpenAI | Not reported | Closed | $1.75 / $14 | 400K | Proprietary |
| 23 | Claude Sonnet 4.6AnthropicReported by Anthropic | 79.6% | Reported by Anthropic | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 24 | DeepSeek V4 Pro (High)DeepSeekReported by DeepSeek | 79.4% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 25 | Inkling 2 MiniThinking Machines LabReported by Thinking Machines Lab | 79.2% | Reported by Thinking Machines Lab | Not reported | Open | $0.15 / $0.5 | 1M | Not recorded |
| 26 | DeepSeek V4 Flash 0731DeepSeekReported by DeepSeek | 79% | Reported by DeepSeek | Not reported | Open | $0.14 / $0.28 | 1M | MIT |
| 27 | Ornith-1.5-35B-A3BOrnith AIReported by Ornith AI | 79% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | Not recorded |
| 28 | Qwen3.6 PlusAlibabaReported by Alibaba | 78.8% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 29 | DeepSeek V4 Flash (High)DeepSeekReported by DeepSeek | 78.6% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 30 | dots3-note PreviewDots StudioReported by Dots Studio | 78.4% | Reported by Dots Studio | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 31 | BTL-4Bad Theory LabsReported by Bad Theory Labs | 78.4% | Reported by Bad Theory Labs | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 32 | MiMo-V2-ProXiaomiCarried from an upstream page | 78% | Carried from an upstream page | Not reported | Closed | Not recorded | 1M | Proprietary |
| 33 | GLM-5Z.AIReported by Z.AI | 77.8% | Reported by Z.AI | Not reported | Open | $1 / $3.2 | 200K | MIT |
| 34 | Qwen3.7 PlusAlibabaReported by Alibaba | 77.7% | Reported by Alibaba | Not reported | Closed | Not recorded | 1M | Proprietary |
| 35 | Apodex 1.1ApodexReported by Apodex | 77.7% | Reported by Apodex | Not reported | Closed | Not recorded | N/A | Proprietary |
| 36 | InklingThinking Machines LabReported by Thinking Machines Lab | 77.6% | Reported by Thinking Machines Lab | Not reported | Open | $1.87 / $4.68 | 1M | Apache-2.0 |
| 37 | Mistral Medium 3.5 128BMistralReported by Mistral | 77.6% | Reported by Mistral | Not reported | Open | $1.5 / $7.5 | 256K | Modified MIT |
| 38 | Muse SparkMetaReported by Meta | 77.4% | Reported by Meta | Not reported | Closed | Not recorded | 262K | Proprietary |
| 39 | Qwen3.6-27BAlibabaReported by Alibaba | 77.2% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 40 | Claude Sonnet 4.5AnthropicReported by Anthropic | 77.2% | Reported by Anthropic | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 41 | Kimi K2.5Moonshot AIReported by Moonshot AI | 76.8% | Reported by Moonshot AI | Not reported | Open | $0.6 / $3 | 256K | Modified MIT |
| 42 | Grok 4.20xAIReported by Meta AI (comparison table) | 76.7% | Reported by Meta AI (comparison table) | Not reported | Closed | $1.25 / $2.5 | 2M | Proprietary |
| 43 | Ripple-65B PreviewRicursive IntelligenceReported by Ricursive Intelligence | 76.5% | Reported by Ricursive Intelligence | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 44 | Qwen3.5 397BAlibabaReported by Alibaba | 76.2% | Reported by Alibaba | Not reported | Open | $0.6 / $3.6 | 128K | Apache-2.0 |
| 45 | Muse Glimmer 30BMetaReported by Meta | 76% | Reported by Meta | Not reported | Open | Not recorded | 131K | Apache-2.0 |
| 46 | Ornith-1.0-35BDeepReinforce AIReported by DeepReinforce AI | 75.6% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | Not recorded |
| 47 | MiMo-V2-OmniXiaomiCarried from an upstream page | 74.8% | Carried from an upstream page | Not reported | Closed | Not recorded | 262K | Proprietary |
| 48 | Laguna M.1PoolsideReported by Poolside | 74.6% | Reported by Poolside | Not reported | Closed | Not recorded | 256K | Proprietary |
| 49 | Claude 4.1 OpusAnthropicReported by Anthropic | 74.5% | Reported by Anthropic | Not reported | Closed | $15 / $75 | 200K | Proprietary |
| 50 | Step 3.5 FlashStepFunReported by StepFun | 74.4% | Reported by StepFun | Not reported | Open | $0.1 / $0.3 | 256K | Apache-2.0 |
| 51 | Hy3 PreviewTencentReported by Tencent | 74.4% | Reported by Tencent | Not reported | Open | Not recorded | 256K | Not recorded |
| 52 | FlintRecursiveReported by Recursive | 74.2% | Reported by Recursive | Not reported | Open | Not recorded | 256K | Apache-2.0 |
| 53 | GLM-4.7Z.AIReported by Z.AI | 73.8% | Reported by Z.AI | Not reported | Open | Not recorded | 200K | Not recorded |
| 54 | DeepSeek V4 FlashDeepSeekReported by DeepSeek | 73.7% | Reported by DeepSeek | Not reported | Open | Not recorded | 1M | Not recorded |
| 55 | MAI-Thinking-1MicrosoftReported by Microsoft | 73.5% | Reported by Microsoft | Not reported | Closed | Not recorded | 256K | Proprietary |
| 56 | Qwen3.6-35B-A3BAlibabaReported by Alibaba | 73.4% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 57 | MiMo-V2-FlashXiaomiReported by Xiaomi | 73.4% | Reported by Xiaomi | Not reported | Open | Not recorded | 256K | MIT |
| 58 | Claude Haiku 4.5AnthropicReported by Anthropic | 73.3% | Reported by Anthropic | Not reported | Closed | $1 / $5 | 200K | Proprietary |
| 59 | Jais-3-MoVA-36B-A4BIFMReported by IFM | 72.8% | Reported by IFM | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 60 | Claude 4 SonnetAnthropicReported by Anthropic | 72.7% | Reported by Anthropic | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 61 | MAI-Code-1.1-FlashMicrosoftReported by Microsoft | 72.6% | Reported by Microsoft | Not reported | Closed | $0.2 / $1.2 | 256K | Proprietary |
| 62 | Claude 4 OpusAnthropicReported by Anthropic | 72.5% | Reported by Anthropic | Not reported | Closed | Not recorded | 200K | Proprietary |
| 63 | Qwen3.5-27BAlibabaReported by Alibaba | 72.4% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 64 | Jais-3-32BIFMReported by IFM | 72.4% | Reported by IFM | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 65 | Qwen3.5-122B-A10BAlibabaReported by Alibaba | 72% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 66 | Nemotron 3 UltraNVIDIAReported by NVIDIA | 71.9% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 67 | Nemotron 3.5 Super VL 120B A12BNVIDIAReported by NVIDIA | 71.1% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 68 | Nemotron 3.5 Super VL 120B A12B NVFP4NVIDIAReported by NVIDIA | 71.1% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | Not recorded |
| 69 | Laguna XS 2.1PoolsideReported by Poolside | 70.9% | Reported by Poolside | Not reported | Open | $0.1 / $0.2 | 262K | Not recorded |
| 70 | Grok Code Fast 1xAICarried from an upstream page | 70.8% | Carried from an upstream page | Not reported | Closed | $0.2 / $1.5 | 256K | Proprietary |
| 71 | Ornith-1.5-9BOrnith AIReported by Ornith AI | 70.6% | Reported by Ornith AI | Not reported | Open | Not recorded | 262K | Not recorded |
| 72 | Solar Pro 4UpstageReported by Upstage | 70.6% | Reported by Upstage | Not reported | Closed | $0.3 / $1.2 | 512K | Proprietary |
| 73 | Solar Open 2UpstageReported by Upstage | 70.4% | Reported by Upstage | Not reported | Open | Not recorded | 1M | Upstage Solar License |
| 74 | Jais-3-375B-A23BIFMReported by IFM | 70.2% | Reported by IFM | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 75 | Jais-3-7BIFMReported by IFM | 70.2% | Reported by IFM | Not reported | Open | Not recorded | 512K | Apache-2.0 |
| 76 | Laguna XS.2PoolsideReported by Poolside | 69.9% | Reported by Poolside | Not reported | Open | Not recorded | 256K | Not recorded |
| 77 | Qwen3-Coder 480B-A35BAlibabaReported by Alibaba | 69.6% | Reported by Alibaba | Not reported | Open | Not recorded | 256K | Not recorded |
| 78 | Ornith-1.0-9BDeepReinforce AIReported by DeepReinforce AI | 69.4% | Reported by DeepReinforce AI | Not reported | Open | Not recorded | 256K | Not recorded |
| 79 | Qwen3.5-35B-A3BAlibabaReported by Alibaba | 69.2% | Reported by Alibaba | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 80 | LongCat-Flash-Lite-SparseMeituanReported by Meituan | 68.2% | Reported by Meituan | Not reported | Open | Not recorded | 1M | MIT |
| 81 | K-EXAONE 2.0LG AI ResearchReported by LG AI Research | 68.2% | Reported by LG AI Research | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 82 | DeepSeek V3.1DeepSeekReported by DeepSeek | 66% | Reported by DeepSeek | Not reported | Open | Not recorded | 128K | Not recorded |
| 83 | Kimi K2Moonshot AIReported by Moonshot AI | 65.8% | Reported by Moonshot AI | Not reported | Closed | $0.6 / $2.5 | 128K | Proprietary |
| 84 | Gemini 2.5 ProGoogleReported by Google | 63.8% | Reported by Google | Not reported | Closed | $1.25 / $10 | 1M | Proprietary |
| 85 | Spark-X2.6-4BSparkLLMReported by SparkLLM | 62.8% | Reported by SparkLLM | Not reported | Open | Not recorded | 1M | Apache-2.0 |
| 86 | GPT-OSS 120BOpenAIReported by OpenAI | 62.4% | Reported by OpenAI | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 87 | Claude 3.7 SonnetAnthropicReported by Anthropic | 62.3% | Reported by Anthropic | Not reported | Closed | Not recorded | 200K | Proprietary |
| 88 | Ternary Bonsai 2 27BPrism MLReported by Prism ML | 60.8% | Reported by Prism ML | Not reported | Open | Not recorded | 262K | Apache-2.0 |
| 89 | GPT-OSS 20BOpenAIReported by OpenAI | 60.7% | Reported by OpenAI | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 90 | Nemotron 3 Super 100BNVIDIAReported by NVIDIA | 60.5% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | Not recorded |
| 91 | GLM-4.7-FlashZ.AIReported by Z.AI | 59.2% | Reported by Z.AI | Not reported | Open | Not recorded | 200K | MIT |
| 92 | DeepSeek-R1-0528DeepSeekReported by DeepSeek | 57.6% | Reported by DeepSeek | Not reported | Open | Not recorded | 128K | Not recorded |
| 93 | Granite 4.2 30BIBMReported by IBM | 57% | Reported by IBM | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 94 | MiniMax M1 80kMiniMaxReported by MiniMax | 56% | Reported by MiniMax | Not reported | Closed | Not recorded | 80K | Proprietary |
| 95 | GPT-4.1OpenAIReported by OpenAI | 54.6% | Reported by OpenAI | Not reported | Closed | $2 / $8 | 1M | Proprietary |
| 96 | ZAYA1-74B-PreviewZyphraReported by Zyphra | 53.2% | Reported by Zyphra | Not reported | Open | Not recorded | 256K | Apache-2.0 |
| 97 | Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIAReported by NVIDIA | 52.8% | Reported by NVIDIA | Not reported | Open | Not recorded | 1M | OpenMDW-1.1 |
| 98 | LFM2.5-24B-A2BLiquidAIReported by LiquidAI | 51% | Reported by LiquidAI | Not reported | Open | $0.1 / $1 | 131K | LFM Open License v1.0 |
| 99 | K-ExaoneLG AI ResearchReported by LG AI Research | 49.4% | Reported by LG AI Research | Not reported | Closed | Not recorded | 256K | Proprietary |
| 100 | o3-miniOpenAIReported by OpenAI | 49.3% | Reported by OpenAI | Not reported | Closed | $1.1 / $4.4 | 200K | Proprietary |
| 101 | LLaDA2.2-flashInclusionAIReported by InclusionAI | 49.3% | Reported by InclusionAI | Not reported | Open | Not recorded | 128K | Not recorded |
| 102 | DeepSeek-R1DeepSeekReported by DeepSeek | 49.2% | Reported by DeepSeek | Not reported | Open | $0.55 / $2.19 | 128K | MIT |
| 103 | Claude 3.5 SonnetAnthropicCarried from an upstream page | 49% | Carried from an upstream page | Not reported | Closed | $3 / $15 | 200K | Proprietary |
| 104 | Granite 4.2 8BIBMReported by IBM | 47.7% | Reported by IBM | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 105 | Mellum2.1-12B-A2.5B-ThinkingJetBrainsReported by JetBrains | 47% | Reported by JetBrains | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 106 | Devstral SmallMistralReported by Mistral | 46.8% | Reported by Mistral | Not reported | Open | Not recorded | 128K | Not recorded |
| 107 | MiniCPM5-2BOpenBMBReported by OpenBMB | 46.4% | Reported by OpenBMB | Not reported | Open | Not recorded | 131K | Apache-2.0 |
| 108 | Sarvam 105BSarvamReported by Sarvam | 45% | Reported by Sarvam | Not reported | Open | Not recorded | 128K | Apache-2.0 |
| 109 | LFM3-1.7BLiquidAIReported by LiquidAI | 43.4% | Reported by LiquidAI | Not reported | Open | Not recorded | 256K | LFM Open License v1.0 |
| 110 | DeepSeek V3DeepSeekCarried from an upstream page | 42% | Carried from an upstream page | Not reported | Open | $0.27 / $1.1 | 128K | MIT |
| 111 | Claude 3.5 HaikuAnthropicReported by Anthropic | 40.6% | Reported by Anthropic | Not reported | Closed | Not recorded | 200K | Proprietary |
| 112 | Sarvam 30BSarvamReported by Sarvam | 34% | Reported by Sarvam | Not reported | Open | Not recorded | 64K | Apache-2.0 |
| 113 | Gemini 2.5 Flash-LiteGoogleReported by Google | 31.6% | Reported by Google | Not reported | Closed | $0.1 / $0.4 | 1M | Proprietary |
| 114 | GPT-4.1 miniOpenAIReported by OpenAI | 23.6% | Reported by OpenAI | Not reported | Closed | $0.4 / $1.6 | 1M | Proprietary |
| 115 | Gemini 2.0 FlashGoogleReported by Google | 21.4% | Reported by Google | Not reported | Closed | Not recorded | 1M | Proprietary |
| 116 | Gemini 2.0 Flash-LiteGoogleReported by Google | 12.5% | Reported by Google | Not reported | Closed | Not recorded | 1M | Proprietary |
| 117 | Gemini 1.5 FlashGoogleReported by Google | 9.6% | Reported by Google | Not reported | Closed | Not recorded | 1M | Proprietary |
- Lab-reported
- Benchmark-run
- Independent
SWE-bench Verified*
BenchLM stores this chart-specific SWE-bench Verified row separately because Arcee notes all models were evaluated in mini-swe-agent-v2.
| Source rank | Source model or configuration | Score | Who ran it | Source setup | Parameters (B) | Open / closed | API input / output ($/1M) | Context | License |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6Anthropic | 75.6% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Closed | $5 / $25 | 1M | Proprietary |
| 2 | MiniMax M2.7MiniMax | 75.4% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open | $0.3 / $1.2 | 200K | MiniMax License |
| 3 | GLM-5Z.AI | 72.8% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open | $1 / $3.2 | 200K | MIT |
| 4 | Kimi K2.5Moonshot AI | 70.8% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open | $0.6 / $3 | 256K | Modified MIT |
| 5 | Trinity-Large-ThinkingArcee AI | 63.2% | Another lab’s comparison table | Arcee AIOriginal result | Not reported | Open | $0.25 / $0.9 | 512K | OpenMDW-1.1 |
How to read this leaderboard
SWE-bench Verified is a 500-task subset of the original SWE-bench repository-repair benchmark. The official leaderboard separates agent submissions from the Bash Only view, where models use the same mini-SWE-agent environment.
Variant check: Verified, Lite, Multilingual, and Pro use different task sets. Keep the benchmark version, scaffold, tool budget, and retry policy with the score. A higher percentage on another version is not a like-for-like improvement.
Use this table to inspect published repository-repair results. We moved SWE-bench Verified to display only in BenchAlign v5.6; its published percentages do not contribute to the current public ranking. Pair it with newer repository tasks and a test on your own codebase.
Operator receipt: 117 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 96%.
Honest limit: Provider-reported scores can use different agents and budgets. Catalog API prices and context limits describe the model, not the cost or context used in a benchmark run. Missing setup fields remain unreported.
Among the reported SWE-bench Verified rows, Claude Opus 5 is first at 96%. The third row is 1.0 points behind. The broader top-10 range is 13.6 points, so the table still separates the published systems.
117 models have been evaluated on SWE-bench Verified. The benchmark falls in the Coding category. SWE-bench Verified is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings. We moved it to display only in BenchAlign v5.6 because its tests are contaminated and saturated.
About SWE-bench Verified
Year
2024
Tasks
500 verified issues
Format
Code patch generation
Difficulty
Professional software engineering
SWE-bench Verified is the most-cited benchmark for AI coding agents on real repository tasks. Each task requires understanding codebases, writing patches, and passing test suites.
When scores here cluster at the top, the DeepSWE leaderboard spreads coding agents apart again, on original tasks drawn from active open-source repositories.
Freshness and provenance
Version
SWE-bench Verified 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-bench Verified measure?
SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.
Which model scores highest on SWE-bench Verified?
Claude Opus 5 by Anthropic currently leads with a score of 96% on SWE-bench Verified.
How many models are evaluated on SWE-bench Verified?
117 AI models have published results on SWE-bench Verified in the BenchLM catalog.
Compare top models on SWE-bench Verified
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.