Skip to main content
BenchLM
Data

Software Engineering Benchmark Verified (SWE-bench Verified)

We show this table for reference; we do not rank on it.

Claude Opus 5 has the highest reported SWE-bench Verified score at 96%, ahead of Claude Mythos 5 (95.5%) and Claude Fable 5 (95%) among 117 models. We compile the table from provider self-reports and secondary reports and keep it for display only; it does not affect overall rankings.

Data verified

SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.

Benchmark score on SWE-bench Verified — November 21, 2026

We compile the SWE-bench Verified rows from provider self-reports and secondary reports. Claude Opus 5 leads the table at 96%, followed by Claude Mythos 5 (95.5%) and Claude Fable 5 (95%). We do not use these results to rank models overall.

117 modelsCodingRefreshingDisplay onlyUpdated November 21, 2026

API rates are catalog prices per million input/output tokens, not measured benchmark-run costs. Context is the published model limit. License names come from recorded publisher terms; unrecorded values remain unavailable.

Benchmark score table (117 models)

None of the 117 results on this page has a documented run by the benchmark or an independent evaluator: 111 reported by the labs that built the models, 1 reported in another publisher’s document and 5 carried from an upstream page, where who ran them is not recorded.

Benchmark score results for SWE-bench Verified
RankModel / configurationScoreParameters (B)Open / closedAPI input / output ($/1M)ContextLicense
1Claude Opus 5AnthropicReported by Anthropic
96%
Not reportedClosed$5 / $251MProprietary
2Claude Mythos 5AnthropicReported by Anthropic
95.5%
Not reportedClosed$10 / $501M+Proprietary
3Claude Fable 5AnthropicReported by Anthropic
95%
Not reportedClosed$10 / $501M+Proprietary
4Ember-1FireworksReported by Fireworks
92.2%
Not reportedClosed$3 / $151.04MProprietary
5Claude Opus 4.8AnthropicReported by Anthropic
88.6%
Not reportedClosed$5 / $251MProprietary
6Claude Opus 4.7AnthropicReported by Anthropic
87.6%
Not reportedClosed$5 / $251MProprietary
7Ornith-1.5-397BOrnith AIReported by Ornith AI
86%
Not reportedOpenNot recorded262KMIT
8Claude Sonnet 5AnthropicReported by Anthropic
85.2%
Not reportedClosed$2 / $101MProprietary
9GPT-5.3 CodexOpenAIReported by OpenAI
85%
Not reportedClosed$1.75 / $14400KProprietary
10Ornith-1.0-397BDeepReinforce AIReported by DeepReinforce AI
82.4%
Not reportedOpenNot recorded256KMIT
11Inkling 2 SmallThinking Machines LabReported by Thinking Machines Lab
82.2%
Not reportedClosed$0.3 / $11MProprietary
12Beam 1.0 501B-A23BReflection AIReported by Reflection AI
81.6%
Not reportedOpenNot recorded1MApache-2.0
13Claude Opus 4.5AnthropicReported by Anthropic
80.9%
Not reportedClosed$5 / $25200KProprietary
14Claude Opus 4.6AnthropicReported by Anthropic
80.8%
Not reportedClosed$5 / $251MProprietary
15Beam 1.0 501B-A23B NVFP4Reflection AIReported by Reflection AI
80.7%
Not reportedOpenNot recorded1MNot recorded
16DeepSeek V4 ProDeepSeekReported by DeepSeek
80.6%
Not reportedOpen$1.32 / $3.961MMIT
17MiniMax M3MiniMaxReported by MiniMax
80.5%
Not reportedOpen$0.3 / $1.21MMiniMax Community License
18Qwen3.7 MaxAlibabaReported by Alibaba
80.4%
Not reportedClosedNot recorded1MProprietary
19Kimi K2.6Moonshot AIReported by Moonshot AI
80.2%
Not reportedOpen$0.95 / $4256KModified MIT
20Inkling-SmallThinking Machines LabReported by Thinking Machines Lab
80.2%
Not reportedOpen$0.58 / $1.441MNot recorded
21Beam 1.0 501B-A23B FP8Reflection AIReported by Reflection AI
80.2%
Not reportedOpenNot recorded1MNot recorded
22GPT-5.2OpenAIReported by OpenAI
80%
Not reportedClosed$1.75 / $14400KProprietary
23Claude Sonnet 4.6AnthropicReported by Anthropic
79.6%
Not reportedClosed$3 / $15200KProprietary
24DeepSeek V4 Pro (High)DeepSeekReported by DeepSeek
79.4%
Not reportedOpenNot recorded1MNot recorded
25Inkling 2 MiniThinking Machines LabReported by Thinking Machines Lab
79.2%
Not reportedOpen$0.15 / $0.51MNot recorded
26DeepSeek V4 Flash 0731DeepSeekReported by DeepSeek
79%
Not reportedOpen$0.14 / $0.281MMIT
27Ornith-1.5-35B-A3BOrnith AIReported by Ornith AI
79%
Not reportedOpenNot recorded262KNot recorded
28Qwen3.6 PlusAlibabaReported by Alibaba
78.8%
Not reportedClosedNot recorded1MProprietary
29DeepSeek V4 Flash (High)DeepSeekReported by DeepSeek
78.6%
Not reportedOpenNot recorded1MNot recorded
30dots3-note PreviewDots StudioReported by Dots Studio
78.4%
Not reportedOpenNot recorded512KApache-2.0
31BTL-4Bad Theory LabsReported by Bad Theory Labs
78.4%
Not reportedOpenNot recorded262KApache-2.0
32MiMo-V2-ProXiaomiCarried from an upstream page
78%
Not reportedClosedNot recorded1MProprietary
33GLM-5Z.AIReported by Z.AI
77.8%
Not reportedOpen$1 / $3.2200KMIT
34Qwen3.7 PlusAlibabaReported by Alibaba
77.7%
Not reportedClosedNot recorded1MProprietary
35Apodex 1.1ApodexReported by Apodex
77.7%
Not reportedClosedNot recordedN/AProprietary
36InklingThinking Machines LabReported by Thinking Machines Lab
77.6%
Not reportedOpen$1.87 / $4.681MApache-2.0
37Mistral Medium 3.5 128BMistralReported by Mistral
77.6%
Not reportedOpen$1.5 / $7.5256KModified MIT
38Muse SparkMetaReported by Meta
77.4%
Not reportedClosedNot recorded262KProprietary
39Qwen3.6-27BAlibabaReported by Alibaba
77.2%
Not reportedOpenNot recorded262KApache-2.0
40Claude Sonnet 4.5AnthropicReported by Anthropic
77.2%
Not reportedClosed$3 / $15200KProprietary
41Kimi K2.5Moonshot AIReported by Moonshot AI
76.8%
Not reportedOpen$0.6 / $3256KModified MIT
42Grok 4.20xAIReported by Meta AI (comparison table)
76.7%
Not reportedClosed$1.25 / $2.52MProprietary
43Ripple-65B PreviewRicursive IntelligenceReported by Ricursive Intelligence
76.5%
Not reportedOpenNot recorded1MApache-2.0
44Qwen3.5 397BAlibabaReported by Alibaba
76.2%
Not reportedOpen$0.6 / $3.6128KApache-2.0
45Muse Glimmer 30BMetaReported by Meta
76%
Not reportedOpenNot recorded131KApache-2.0
46Ornith-1.0-35BDeepReinforce AIReported by DeepReinforce AI
75.6%
Not reportedOpenNot recorded256KNot recorded
47MiMo-V2-OmniXiaomiCarried from an upstream page
74.8%
Not reportedClosedNot recorded262KProprietary
48Laguna M.1PoolsideReported by Poolside
74.6%
Not reportedClosedNot recorded256KProprietary
49Claude 4.1 OpusAnthropicReported by Anthropic
74.5%
Not reportedClosed$15 / $75200KProprietary
50Step 3.5 FlashStepFunReported by StepFun
74.4%
Not reportedOpen$0.1 / $0.3256KApache-2.0
51Hy3 PreviewTencentReported by Tencent
74.4%
Not reportedOpenNot recorded256KNot recorded
52FlintRecursiveReported by Recursive
74.2%
Not reportedOpenNot recorded256KApache-2.0
53GLM-4.7Z.AIReported by Z.AI
73.8%
Not reportedOpenNot recorded200KNot recorded
54DeepSeek V4 FlashDeepSeekReported by DeepSeek
73.7%
Not reportedOpenNot recorded1MNot recorded
55MAI-Thinking-1MicrosoftReported by Microsoft
73.5%
Not reportedClosedNot recorded256KProprietary
56Qwen3.6-35B-A3BAlibabaReported by Alibaba
73.4%
Not reportedOpenNot recorded262KApache-2.0
57MiMo-V2-FlashXiaomiReported by Xiaomi
73.4%
Not reportedOpenNot recorded256KMIT
58Claude Haiku 4.5AnthropicReported by Anthropic
73.3%
Not reportedClosed$1 / $5200KProprietary
59Jais-3-MoVA-36B-A4BIFMReported by IFM
72.8%
Not reportedOpenNot recorded512KApache-2.0
60Claude 4 SonnetAnthropicReported by Anthropic
72.7%
Not reportedClosed$3 / $15200KProprietary
61MAI-Code-1.1-FlashMicrosoftReported by Microsoft
72.6%
Not reportedClosed$0.2 / $1.2256KProprietary
62Claude 4 OpusAnthropicReported by Anthropic
72.5%
Not reportedClosedNot recorded200KProprietary
63Qwen3.5-27BAlibabaReported by Alibaba
72.4%
Not reportedOpenNot recorded262KApache-2.0
64Jais-3-32BIFMReported by IFM
72.4%
Not reportedOpenNot recorded512KApache-2.0
65Qwen3.5-122B-A10BAlibabaReported by Alibaba
72%
Not reportedOpenNot recorded262KApache-2.0
66Nemotron 3 UltraNVIDIAReported by NVIDIA
71.9%
Not reportedOpenNot recorded1MOpenMDW-1.1
67Nemotron 3.5 Super VL 120B A12BNVIDIAReported by NVIDIA
71.1%
Not reportedOpenNot recorded1MOpenMDW-1.1
68Nemotron 3.5 Super VL 120B A12B NVFP4NVIDIAReported by NVIDIA
71.1%
Not reportedOpenNot recorded1MNot recorded
69Laguna XS 2.1PoolsideReported by Poolside
70.9%
Not reportedOpen$0.1 / $0.2262KNot recorded
70Grok Code Fast 1xAICarried from an upstream page
70.8%
Not reportedClosed$0.2 / $1.5256KProprietary
71Ornith-1.5-9BOrnith AIReported by Ornith AI
70.6%
Not reportedOpenNot recorded262KNot recorded
72Solar Pro 4UpstageReported by Upstage
70.6%
Not reportedClosed$0.3 / $1.2512KProprietary
73Solar Open 2UpstageReported by Upstage
70.4%
Not reportedOpenNot recorded1MUpstage Solar License
74Jais-3-375B-A23BIFMReported by IFM
70.2%
Not reportedOpenNot recorded512KApache-2.0
75Jais-3-7BIFMReported by IFM
70.2%
Not reportedOpenNot recorded512KApache-2.0
76Laguna XS.2PoolsideReported by Poolside
69.9%
Not reportedOpenNot recorded256KNot recorded
77Qwen3-Coder 480B-A35BAlibabaReported by Alibaba
69.6%
Not reportedOpenNot recorded256KNot recorded
78Ornith-1.0-9BDeepReinforce AIReported by DeepReinforce AI
69.4%
Not reportedOpenNot recorded256KNot recorded
79Qwen3.5-35B-A3BAlibabaReported by Alibaba
69.2%
Not reportedOpenNot recorded262KApache-2.0
80LongCat-Flash-Lite-SparseMeituanReported by Meituan
68.2%
Not reportedOpenNot recorded1MMIT
81K-EXAONE 2.0LG AI ResearchReported by LG AI Research
68.2%
Not reportedOpenNot recorded262KApache-2.0
82DeepSeek V3.1DeepSeekReported by DeepSeek
66%
Not reportedOpenNot recorded128KNot recorded
83Kimi K2Moonshot AIReported by Moonshot AI
65.8%
Not reportedClosed$0.6 / $2.5128KProprietary
84Gemini 2.5 ProGoogleReported by Google
63.8%
Not reportedClosed$1.25 / $101MProprietary
85Spark-X2.6-4BSparkLLMReported by SparkLLM
62.8%
Not reportedOpenNot recorded1MApache-2.0
86GPT-OSS 120BOpenAIReported by OpenAI
62.4%
Not reportedOpenNot recorded128KApache-2.0
87Claude 3.7 SonnetAnthropicReported by Anthropic
62.3%
Not reportedClosedNot recorded200KProprietary
88Ternary Bonsai 2 27BPrism MLReported by Prism ML
60.8%
Not reportedOpenNot recorded262KApache-2.0
89GPT-OSS 20BOpenAIReported by OpenAI
60.7%
Not reportedOpenNot recorded128KApache-2.0
90Nemotron 3 Super 100BNVIDIAReported by NVIDIA
60.5%
Not reportedOpenNot recorded1MNot recorded
91GLM-4.7-FlashZ.AIReported by Z.AI
59.2%
Not reportedOpenNot recorded200KMIT
92DeepSeek-R1-0528DeepSeekReported by DeepSeek
57.6%
Not reportedOpenNot recorded128KNot recorded
93Granite 4.2 30BIBMReported by IBM
57%
Not reportedOpenNot recorded128KApache-2.0
94MiniMax M1 80kMiniMaxReported by MiniMax
56%
Not reportedClosedNot recorded80KProprietary
95GPT-4.1OpenAIReported by OpenAI
54.6%
Not reportedClosed$2 / $81MProprietary
96ZAYA1-74B-PreviewZyphraReported by Zyphra
53.2%
Not reportedOpenNot recorded256KApache-2.0
97Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIAReported by NVIDIA
52.8%
Not reportedOpenNot recorded1MOpenMDW-1.1
98LFM2.5-24B-A2BLiquidAIReported by LiquidAI
51%
Not reportedOpen$0.1 / $1131KLFM Open License v1.0
99K-ExaoneLG AI ResearchReported by LG AI Research
49.4%
Not reportedClosedNot recorded256KProprietary
100o3-miniOpenAIReported by OpenAI
49.3%
Not reportedClosed$1.1 / $4.4200KProprietary
101LLaDA2.2-flashInclusionAIReported by InclusionAI
49.3%
Not reportedOpenNot recorded128KNot recorded
102DeepSeek-R1DeepSeekReported by DeepSeek
49.2%
Not reportedOpen$0.55 / $2.19128KMIT
103Claude 3.5 SonnetAnthropicCarried from an upstream page
49%
Not reportedClosed$3 / $15200KProprietary
104Granite 4.2 8BIBMReported by IBM
47.7%
Not reportedOpenNot recorded128KApache-2.0
105Mellum2.1-12B-A2.5B-ThinkingJetBrainsReported by JetBrains
47%
Not reportedOpenNot recorded128KApache-2.0
106Devstral SmallMistralReported by Mistral
46.8%
Not reportedOpenNot recorded128KNot recorded
107MiniCPM5-2BOpenBMBReported by OpenBMB
46.4%
Not reportedOpenNot recorded131KApache-2.0
108Sarvam 105BSarvamReported by Sarvam
45%
Not reportedOpenNot recorded128KApache-2.0
109LFM3-1.7BLiquidAIReported by LiquidAI
43.4%
Not reportedOpenNot recorded256KLFM Open License v1.0
110DeepSeek V3DeepSeekCarried from an upstream page
42%
Not reportedOpen$0.27 / $1.1128KMIT
111Claude 3.5 HaikuAnthropicReported by Anthropic
40.6%
Not reportedClosedNot recorded200KProprietary
112Sarvam 30BSarvamReported by Sarvam
34%
Not reportedOpenNot recorded64KApache-2.0
113Gemini 2.5 Flash-LiteGoogleReported by Google
31.6%
Not reportedClosed$0.1 / $0.41MProprietary
114GPT-4.1 miniOpenAIReported by OpenAI
23.6%
Not reportedClosed$0.4 / $1.61MProprietary
115Gemini 2.0 FlashGoogleReported by Google
21.4%
Not reportedClosedNot recorded1MProprietary
116Gemini 2.0 Flash-LiteGoogleReported by Google
12.5%
Not reportedClosedNot recorded1MProprietary
117Gemini 1.5 FlashGoogleReported by Google
9.6%
Not reportedClosedNot recorded1MProprietary
  • Lab-reported
  • Benchmark-run
  • Independent
What the labels mean

SWE-bench Verified*

BenchLM stores this chart-specific SWE-bench Verified row separately because Arcee notes all models were evaluated in mini-swe-agent-v2.

SWE-bench Verified*; source configurations remain separate from the canonical benchmark table.
Source rankSource model or configurationScoreWho ran itSource setupParameters (B)Open / closedAPI input / output ($/1M)ContextLicense
1Claude Opus 4.6Anthropic75.6%Another lab’s comparison tableArcee AIOriginal resultNot reportedClosed$5 / $251MProprietary
2MiniMax M2.7MiniMax75.4%Another lab’s comparison tableArcee AIOriginal resultNot reportedOpen$0.3 / $1.2200KMiniMax License
3GLM-5Z.AI72.8%Another lab’s comparison tableArcee AIOriginal resultNot reportedOpen$1 / $3.2200KMIT
4Kimi K2.5Moonshot AI70.8%Another lab’s comparison tableArcee AIOriginal resultNot reportedOpen$0.6 / $3256KModified MIT
5Trinity-Large-ThinkingArcee AI63.2%Another lab’s comparison tableArcee AIOriginal resultNot reportedOpen$0.25 / $0.9512KOpenMDW-1.1

How to read this leaderboard

SWE-bench Verified is a 500-task subset of the original SWE-bench repository-repair benchmark. The official leaderboard separates agent submissions from the Bash Only view, where models use the same mini-SWE-agent environment.

Variant check: Verified, Lite, Multilingual, and Pro use different task sets. Keep the benchmark version, scaffold, tool budget, and retry policy with the score. A higher percentage on another version is not a like-for-like improvement.

Use this table to inspect published repository-repair results. We moved SWE-bench Verified to display only in BenchAlign v5.6; its published percentages do not contribute to the current public ranking. Pair it with newer repository tasks and a test on your own codebase.

Operator receipt: 117 sourced rows are currently displayable on this page; the leading published row is Claude Opus 5 at 96%.

Honest limit: Provider-reported scores can use different agents and budgets. Catalog API prices and context limits describe the model, not the cost or context used in a benchmark run. Missing setup fields remain unreported.

Among the reported SWE-bench Verified rows, Claude Opus 5 is first at 96%. The third row is 1.0 points behind. The broader top-10 range is 13.6 points, so the table still separates the published systems.

117 models have been evaluated on SWE-bench Verified. The benchmark falls in the Coding category. SWE-bench Verified is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings. We moved it to display only in BenchAlign v5.6 because its tests are contaminated and saturated.

About SWE-bench Verified

Year

2024

Tasks

500 verified issues

Format

Code patch generation

Difficulty

Professional software engineering

SWE-bench Verified is the most-cited benchmark for AI coding agents on real repository tasks. Each task requires understanding codebases, writing patches, and passing test suites.

When scores here cluster at the top, the DeepSWE leaderboard spreads coding agents apart again, on original tasks drawn from active open-source repositories.

Freshness and provenance

Version

SWE-bench Verified 2024

Refresh cadence

Annual

Staleness state

Refreshing

Question availability

Public benchmark set

RefreshingDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does SWE-bench Verified measure?

SWE-bench Verified tests patches for 500 human-filtered GitHub issues. A score measures how many tasks an evaluated model-and-agent setup resolved; the model name alone does not identify that setup.

Which model scores highest on SWE-bench Verified?

Claude Opus 5 by Anthropic currently leads with a score of 96% on SWE-bench Verified.

How many models are evaluated on SWE-bench Verified?

117 AI models have published results on SWE-bench Verified in the BenchLM catalog.

Last updated: November 21, 2026 · BenchLM version SWE-bench Verified 2024

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 7,000+ subscribers.

One email each week. Unsubscribe anytime.