# Software Engineering Benchmark Verified (SWE-bench Verified)

> A curated, human-verified subset of SWE-bench that tests models on resolving real GitHub issues from popular open-source Python repositories like Django, Flask, and scikit-learn.

Canonical page: https://benchlm.ai/benchmarks/swe-bench-verified

- Category: [Coding](/coding)
- Last updated: September 10, 2026

## About SWE-bench Verified

- Year: 2024
- Tasks: 500 verified issues
- Format: Code patch generation
- Difficulty: Professional software engineering
- Paper: [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://arxiv.org/abs/2310.06770)

SWE-bench Verified is the most-cited benchmark for AI coding agents on real repository tasks. Each task requires understanding codebases, writing patches, and passing test suites.

SWE-bench Verified is currently weighted in BenchLM's scoring formula. The Coding category carries 20% of the overall score, and SWE-bench Verified contributes 10% of that category score.

## Leaderboard (74 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Claude Opus 5](/models/claude-opus-5) | Anthropic | 96% |
| 2 | [Claude Mythos 5](/models/claude-mythos-5) | Anthropic | 95.5% |
| 3 | [Claude Fable 5](/models/claude-fable) | Anthropic | 95% |
| 4 | [Claude Opus 4.8](/models/claude-opus-4-8) | Anthropic | 88.6% |
| 5 | [Claude Opus 4.7 (Adaptive)](/models/claude-opus-4-7-adaptive) | Anthropic | 87.6% |
| 6 | [Ornith-1.5-397B](/models/ornith-1-5-397b) | Ornith AI | 86% |
| 7 | [Claude Sonnet 5](/models/claude-sonnet-5) | Anthropic | 85.2% |
| 8 | [GPT-5.3 Codex](/models/gpt-5-3-codex) | OpenAI | 85% |
| 9 | [Ornith-1.0-397B](/models/ornith-1-0-397b) | DeepReinforce AI | 82.4% |
| 10 | [Claude Opus 4.5](/models/claude-opus-4-5) | Anthropic | 80.9% |
| 11 | [Claude Opus 4.6](/models/claude-opus-4-6) | Anthropic | 80.8% |
| 12 | [DeepSeek V4 Pro 0813](/models/deepseek-v4-pro-0813) | DeepSeek | 80.6% |
| 13 | [MiniMax M3](/models/minimax-m3) | MiniMax | 80.5% |
| 14 | [Qwen3.7 Max](/models/qwen3-7-max) | Alibaba | 80.4% |
| 15 | [Kimi K2.6](/models/kimi-2-6) | Moonshot AI | 80.2% |
| 16 | [Inkling-Small](/models/inkling-small) | Thinking Machines Lab | 80.2% |
| 17 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 80% |
| 18 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Anthropic | 79.6% |
| 19 | [Ornith-1.5-35B-A3B](/models/ornith-1-5-35b-a3b) | Ornith AI | 79% |
| 20 | [DeepSeek V4 Flash 0731](/models/deepseek-v4-flash-0731) | DeepSeek | 79% |
| 21 | [Qwen3.6 Plus](/models/qwen3-6-plus) | Alibaba | 78.8% |
| 22 | [dots3-note Preview](/models/dots3-note-preview) | Dots Studio | 78.4% |
| 23 | [BTL-4](/models/btl-4) | Bad Theory Labs | 78.4% |
| 24 | [MiMo-V2-Pro](/models/mimo-v2-pro) | Xiaomi | 78% |
| 25 | [GLM-5](/models/glm-5) | Z.AI | 77.8% |
| 26 | [Qwen3.7 Plus](/models/qwen3-7-plus) | Alibaba | 77.7% |
| 27 | [Apodex 1.1](/models/apodex-1-1) | Apodex | 77.7% |
| 28 | [Inkling](/models/inkling) | Thinking Machines Lab | 77.6% |
| 29 | [Mistral Medium 3.5 128B](/models/mistral-medium-3-5-128b) | Mistral | 77.6% |
| 30 | [Muse Spark](/models/muse-spark) | Meta | 77.4% |
| 31 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Anthropic | 77.2% |
| 32 | [Qwen3.6-27B](/models/qwen3-6-27b) | Alibaba | 77.2% |
| 33 | [Kimi K2.5 (Reasoning)](/models/kimi-k2-5-reasoning) | Moonshot AI | 76.8% |
| 34 | [Kimi K2.5](/models/kimi-k2-5) | Moonshot AI | 76.8% |
| 35 | [Grok 4.20](/models/grok-4-20-beta) | xAI | 76.7% |
| 36 | [Qwen3.5 397B](/models/qwen3-5-397b) | Alibaba | 76.2% |
| 37 | [Muse Glimmer 30B](/models/muse-glimmer-30b) | Meta | 76% |
| 38 | [Ornith-1.0-35B](/models/ornith-1-0-35b) | DeepReinforce AI | 75.6% |
| 39 | [MiMo-V2-Omni](/models/mimo-v2-omni) | Xiaomi | 74.8% |
| 40 | [Laguna M.1](/models/laguna-m-1) | Poolside | 74.6% |
| 41 | [Claude 4.1 Opus](/models/claude-4-1-opus) | Anthropic | 74.5% |
| 42 | [Hy3 Preview](/models/hy3-preview) | Tencent | 74.4% |
| 43 | [GLM-4.7](/models/glm-4-7) | Z.AI | 73.8% |
| 44 | [MAI-Thinking-1](/models/mai-thinking-1) | Microsoft | 73.5% |
| 45 | [MiMo-V2-Flash](/models/mimo-v2-flash) | Xiaomi | 73.4% |
| 46 | [Qwen3.6-35B-A3B](/models/qwen3-6-35b-a3b) | Alibaba | 73.4% |
| 47 | [Claude Haiku 4.5](/models/claude-haiku-4-5) | Anthropic | 73.3% |
| 48 | [Claude 4 Sonnet](/models/claude-4-sonnet) | Anthropic | 72.7% |
| 49 | [MAI-Code-1.1-Flash](/models/mai-code-1-1-flash) | Microsoft | 72.6% |
| 50 | [Qwen3.5-27B](/models/qwen3-5-27b) | Alibaba | 72.4% |
| 51 | [Qwen3.5-122B-A10B](/models/qwen3-5-122b-a10b) | Alibaba | 72% |
| 52 | [Nemotron 3 Ultra](/models/nemotron-3-ultra) | NVIDIA | 71.9% |
| 53 | [Laguna XS 2.1](/models/laguna-xs-2-1) | Poolside | 70.9% |
| 54 | [Grok Code Fast 1](/models/grok-code-fast-1) | xAI | 70.8% |
| 55 | [Ornith-1.5-9B](/models/ornith-1-5-9b) | Ornith AI | 70.6% |
| 56 | [Solar Pro 4](/models/solar-pro-4) | Upstage | 70.6% |
| 57 | [Solar Open 2](/models/solar-open-2) | Upstage | 70.4% |
| 58 | [Laguna XS.2](/models/laguna-xs-2) | Poolside | 69.9% |
| 59 | [Ornith-1.0-9B](/models/ornith-1-0-9b) | DeepReinforce AI | 69.4% |
| 60 | [Qwen3.5-35B-A3B](/models/qwen3-5-35b-a3b) | Alibaba | 69.2% |
| 61 | [LongCat-Flash-Lite-Sparse](/models/longcat-flash-lite-sparse) | Meituan | 68.2% |
| 62 | [K-EXAONE 2.0](/models/k-exaone-2-0) | LG AI Research | 68.2% |
| 63 | [Gemini 2.5 Pro](/models/gemini-2-5-pro) | Google | 63.8% |
| 64 | [Granite 4.2 30B](/models/granite-4-2-30b) | IBM | 57% |
| 65 | [GPT-4.1](/models/gpt-4-1) | OpenAI | 54.6% |
| 66 | [ZAYA1-74B-Preview](/models/zaya1-74b-preview) | Zyphra | 53.2% |
| 67 | [Nemotron 3.5 Lightning 30B A3B NVFP4](/models/nemotron-3-5-lightning-30b-a3b-nvfp4) | NVIDIA | 52.8% |
| 68 | [o3-mini](/models/o3-mini) | OpenAI | 49.3% |
| 69 | [LLaDA2.2-flash](/models/llada2-2-flash) | InclusionAI | 49.3% |
| 70 | [Claude 3.5 Sonnet](/models/claude-3-5-sonnet) | Anthropic | 49% |
| 71 | [Granite 4.2 8B](/models/granite-4-2-8b) | IBM | 47.7% |
| 72 | [MiniCPM5-2B](/models/minicpm5-2b) | OpenBMB | 46.4% |
| 73 | [DeepSeek V3](/models/deepseek-v3) | DeepSeek | 42% |
| 74 | [GPT-4.1 mini](/models/gpt-4-1-mini) | OpenAI | 23.6% |

## FAQ

### What does SWE-bench Verified measure?

A curated, human-verified subset of SWE-bench that tests models on resolving real GitHub issues from popular open-source Python repositories like Django, Flask, and scikit-learn.

### Which model scores highest on SWE-bench Verified?

Claude Opus 5 by Anthropic currently leads with a score of 96% on SWE-bench Verified.

### How many models are evaluated on SWE-bench Verified?

74 AI models have been evaluated on SWE-bench Verified on BenchLM.

### Does SWE-bench Verified affect BenchLM's overall score?

Yes. SWE-bench Verified is a weighted benchmark inside the Coding category, which carries 20% of BenchLM's overall score. SWE-bench Verified itself contributes 10% of that category score.

## Compare Top Models on SWE-bench Verified

- [Claude Opus 5 vs Claude Mythos 5](/compare/claude-mythos-5-vs-claude-opus-5)
- [Claude Mythos 5 vs Claude Fable 5](/compare/claude-fable-vs-claude-mythos-5)
- [Claude Fable 5 vs Claude Opus 4.8](/compare/claude-fable-vs-claude-opus-4-8)
- [Claude Opus 4.8 vs Claude Opus 4.7 (Adaptive)](/compare/claude-opus-4-7-adaptive-vs-claude-opus-4-8)

## Related Reading

- [SWE-bench Verified benchmark explainer](/blog/posts/swe-bench-explained)