SWE-bench, Vals AI run (SWE-bench (Vals))
We show this table for reference; we do not rank on it.
Vals AI’s independent run of the public SWE-bench issue set, reported by human time-to-fix bucket. Vals removed SWE-bench Verified from its index as saturated on 2026-05-04; this board is the standalone SWE-bench run.
Evaluation results published by Vals AI. Published leaderboard aggregates and evaluation settings only. Dataset contents are not included.
Vals SWE-bench score on SWE-bench (Vals) — September 1, 2026
We mirror the published vals swe-bench score view for SWE-bench (Vals). Claude Opus 5 leads the public snapshot at 97.0%, followed by DeepSeek V4 Pro 0813 (96.4%) and GPT-5.6 Sol (96.2%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
DeepSeek V4 Pro 0813
DeepSeek
max reasoning
GPT-5.6 Sol
OpenAI
max reasoning
88 modelsCodingCurrentDisplay onlyUpdated September 1, 2026
Vals SWE-bench score table (88 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Claude Opus 5Anthropic | 97.0% | Not reported | Closed |
| 2 | DeepSeek V4 Pro 0813DeepSeekmax reasoning | 96.4% | Not reported | Open |
| 3 | GPT-5.6 SolOpenAImax reasoning | 96.2% | Not reported | Closed |
| 4 | Grok 4.6xAIhigh reasoning | 95.6% | Not reported | Closed |
| 5 | GPT-5.6 TerraOpenAImax reasoning | 95.4% | Not reported | Closed |
| 6 | GLM-5.3Z.AImax reasoning | 95.4% | Not reported | Open |
| 7 | Claude Fable 5Anthropic | 95.0% | Not reported | Closed |
| 8 | Kimi K3Moonshot AI | 93.4% | Not reported | Open |
| 9 | GPT-5.6 LunaOpenAImax reasoning | 93.0% | Not reported | Closed |
| 10 | GLM-5.3-FlashZ.AImax reasoning | 92.0% | Not reported | Open |
| 11 | DeepSeek V4 Flash 0731DeepSeekhigh reasoning | 88.8% | Not reported | Open |
| 12 | Claude Opus 4.8Anthropic | 88.6% | Not reported | Closed |
| 13 | Grok 4.5xAIhigh reasoning | 86.6% | Not reported | Closed |
| 14 | Muse Spark 1.2Metaxhigh reasoning | 86.6% | Not reported | Closed |
| 15 | Qwen3.8-27BAlibabaxhigh reasoning | 86.0% | Not reported | Open |
| 16 | Claude Opus 4.8 Claude CodeAnthropicClaude CodeClaude Code | 85.8% | Not reported | Unknown |
| 17 | Qwen3.8 MaxAlibaba | 85.6% | Not reported | Open |
| 18 | GLM-5.2Z.AImax reasoning | 82.8% | Not reported | Open |
| 19 | GPT-5.5OpenAIxhigh reasoning | 82.6% | Not reported | Closed |
| 20 | Inkling-SmallThinking Machines Lab0.99 reasoning | 82.2% | Not reported | Open |
| 21 | Muse Spark 1.1Metaxhigh reasoning | 82.0% | Not reported | Closed |
| 22 | Claude Opus 4.7Anthropic | 82.0% | Not reported | Closed |
| 23 | Gemini 3.7 FlashGooglehigh reasoning | 80.8% | Not reported | Closed |
| 24 | Gemini 3.8 FlashGooglehigh reasoning | 80.0% | Not reported | Closed |
| 25 | Composer 2.5CursorCursor CLICursor CLI | 79.6% | Not reported | Closed |
| 26 | Gemini 3.6 FlashGooglehigh reasoning | 79.6% | Not reported | Closed |
| 27 | Claude Sonnet 5Anthropic | 79.6% | Not reported | Closed |
| 28 | Gemini 3.5 FlashGooglehigh reasoning | 78.8% | Not reported | Closed |
| 29 | Gemini 3.1 Pro PreviewGooglehigh reasoning | 78.8% | Not reported | Unknown |
| 30 | GPT-5.4OpenAIxhigh reasoning | 78.2% | Not reported | Closed |
| 31 | Claude Opus 4.6 (Adaptive)Anthropic | 78.2% | Not reported | Closed |
| 32 | Kimi K2.7 CodeMoonshot AI | 78.2% | Not reported | Open |
| 33 | GPT-5.3 CodexOpenAIxhigh reasoning | 78.0% | Not reported | Closed |
| 34 | InklingThinking Machines Lab0.99 reasoning | 77.6% | Not reported | Open |
| 35 | Claude Sonnet 4.6Anthropic | 77.4% | Not reported | Closed |
| 36 | DeepSeek V4 Pro 0813DeepSeekmax reasoning | 77.4% | Not reported | Open |
| 37 | GPT-5.5 CodexOpenAICodexCodex | 76.4% | Not reported | Unknown |
| 38 | Claude Opus 4.5 ThinkingAnthropic | 76.4% | Not reported | Closed |
| 39 | Gemini 3 Pro PreviewGooglehigh reasoning | 76.4% | Not reported | Unknown |
| 40 | GLM-5.1Z.AI | 76.4% | Not reported | Open |
| 41 | GPT-5.5 FactoryOpenAIFactoryFactory | 76.2% | Not reported | Unknown |
| 42 | Kimi K2.6Moonshot AI | 76.2% | Not reported | Open |
| 43 | GPT-5.2OpenAIxhigh reasoning | 75.8% | Not reported | Closed |
| 44 | Gemini 3 Flash PreviewGooglehigh reasoning | 75.0% | Not reported | Unknown |
| 45 | Gemini 3.5 Flash-LiteGooglehigh reasoning | 75.0% | Not reported | Closed |
| 46 | MiniMax M3MiniMax | 75.0% | Not reported | Open |
| 47 | MiniMax M2.1MiniMax | 74.8% | Not reported | Unknown |
| 48 | Muse SparkMeta | 74.4% | Not reported | Closed |
| 49 | MiniMax M2.5MiniMax | 74.2% | Not reported | Closed |
| 50 | MiMo-V2.5-ProXiaomi | 74.0% | Not reported | Closed |
| 51 | MiniMax M2.7MiniMax | 73.8% | Not reported | Open |
| 52 | Qwen3.6 PlusAlibaba | 73.4% | Not reported | Closed |
| 53 | GPT-5.4 miniOpenAIxhigh reasoning | 73.0% | Not reported | Closed |
| 54 | Qwen 3.6 Max (preview)Alibaba | 72.8% | Not reported | Closed |
| 55 | GPT-5.2-CodexOpenAIhigh reasoning | 72.4% | Not reported | Closed |
| 56 | Grok 4.20 0309 ReasoningxAI | 72.2% | Not reported | Unknown |
| 57 | Grok 4.3xAIhigh reasoning | 71.4% | Not reported | Closed |
| 58 | GLM 5 ThinkingZhipu AI | 71.4% | Not reported | Unknown |
| 59 | Qwen3.5 Plus ThinkingAlibaba | 71.2% | Not reported | Unknown |
| 60 | MiMo-V2.5Xiaomi | 71.0% | Not reported | Closed |
| 61 | Kimi K2.5 ThinkingMoonshot AI | 70.0% | Not reported | Unknown |
| 62 | Claude Sonnet 4.5 ThinkingAnthropic | 70.0% | Not reported | Closed |
| 63 | Qwen3.6-27BAlibaba | 70.0% | Not reported | Open |
| 64 | GPT-5.4 nanoOpenAIhigh reasoning | 69.8% | Not reported | Closed |
| 65 | GPT-5.1OpenAIhigh reasoning | 69.8% | Not reported | Closed |
| 66 | GLM-4.7Z.AI | 69.4% | Not reported | Open |
| 67 | GPT-5OpenAIhigh reasoning | 69.0% | Not reported | Unknown |
| 68 | Nemotron 3 Ultra 550b A55bNvidia | 69.0% | Not reported | Unknown |
| 69 | Qwen3.7 MaxAlibaba | 68.8% | Not reported | Closed |
| 70 | DeepSeek V3p2 ThinkingFireworks AIhigh reasoning | 67.6% | Not reported | Unknown |
| 71 | Claude Haiku 4.5 ThinkingAnthropic | 66.6% | Not reported | Closed |
| 72 | Mistral Medium 3.5Mistral AIhigh reasoning | 66.4% | Not reported | Unknown |
| 73 | Ling 3.0 Flash 2607Ant | 65.2% | Not reported | Unknown |
| 74 | Qwen3.5 FlashAlibaba | 64.4% | Not reported | Closed |
| 75 | Gemini 3.1 Flash Lite PreviewGooglehigh reasoning | 62.8% | Not reported | Unknown |
| 76 | Devstral 2512Mistral AI | 62.8% | Not reported | Unknown |
| 77 | GPT-5 miniOpenAIhigh reasoning | 60.8% | Not reported | Closed |
| 78 | Kimi K2 ThinkingMoonshot AI | 60.2% | Not reported | Unknown |
| 79 | Grok 4 0709xAI | 57.8% | Not reported | Unknown |
| 80 | Laguna M.1Poolside | 57.6% | Not reported | Closed |
| 81 | Laguna XS.2Poolside | 55.2% | Not reported | Open |
| 82 | Gemini 2.5 ProGoogle | 54.4% | Not reported | Closed |
| 83 | Nemotron Lightning 3p5 30b A3bFireworks AI | 52.8% | Not reported | Unknown |
| 84 | Grok 4 Fast (Reasoning)xAI | 45.4% | Not reported | Closed |
| 85 | Grok 4.1 Fast (Reasoning)xAI | 41.4% | Not reported | Closed |
| 86 | Mistral Large 2512Mistral AI | 41.4% | Not reported | Unknown |
| 87 | GPT-OSS 120BOpenAI | 33.6% | Not reported | Open |
| 88 | Command A 03 2025Cohere | 7.8% | Not reported | Unknown |
How Vals SWE-bench mirror is shown here
BenchLM mirrors the public Vals AI Vals SWE-bench mirror leaderboard captured from https://www.vals.ai/benchmarks/swebench and updated by Vals on September 1, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.
Vals SWE-bench mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.
Snapshot
The published SWE-bench (Vals) snapshot places Claude Opus 5 first at 97.0%. The third row is 0.8 points behind. The broader top-10 range is 5.0 points, so many of the published results sit in a relatively narrow band.
88 models have been evaluated on SWE-bench (Vals). The benchmark falls in the Coding category. SWE-bench (Vals) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings. We moved it to display only in BenchAlign v5.6 because frontier scores have saturated it.
About SWE-bench (Vals)
Year
2026
Tasks
Real repository issues by human time bucket
Format
Resolved rate
Difficulty
Frontier coding agents
BenchLM mirrors the Vals AI board on a dedicated key so a provider-run row on the canonical key is never overwritten. Vals publishes per-task accuracy with standard error, latency, and cost for every model it runs. Admitted as independent third-party evidence in methodology v5.5 (2026-09-04). Display-only since methodology v5.6 (2026-09-22), because frontier scores have saturated it.
Freshness and provenance
Version
SWE-bench (Vals) 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does SWE-bench (Vals) measure?
Vals AI’s independent run of the public SWE-bench issue set, reported by human time-to-fix bucket. Vals removed SWE-bench Verified from its index as saturated on 2026-05-04; this board is the standalone SWE-bench run.
Which model leads the published SWE-bench (Vals) snapshot?
Claude Opus 5 currently leads the published SWE-bench (Vals) snapshot with 97.0% vals swe-bench score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on SWE-bench (Vals)?
The September 1, 2026 snapshot contains 88 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.