SWE-bench, Vals AI run (SWE-bench (Vals))
Vals AI’s independent run of the public SWE-bench issue set, reported by human time-to-fix bucket. Vals removed SWE-bench Verified from its index as saturated on 2026-05-04; this board is the standalone SWE-bench run.
Data verified 23 confirmed releases in the last 30 daysSee the free Radar BriefHow BenchLM shows Vals SWE-bench mirror
BenchLM mirrors the public Vals AI Vals SWE-bench mirror leaderboard captured from https://www.vals.ai/benchmarks/swebench and updated by Vals on September 1, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.
Vals SWE-bench mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.
Snapshot
Vals SWE-bench score on SWE-bench (Vals) — September 1, 2026
We mirror the published vals swe-bench score view for SWE-bench (Vals). Claude Opus 5 leads the public snapshot at 97.0%, followed by DeepSeek V4 Pro 0813 (96.4%) and GPT-5.6 Sol (96.2%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
DeepSeek V4 Pro 0813
DeepSeek
max reasoning
GPT-5.6 Sol
OpenAI
max reasoning
88 modelsCodingCurrentDisplay onlyUpdated September 1, 2026
Vals SWE-bench score table (88 models)
ScoreThe published SWE-bench (Vals) snapshot places Claude Opus 5 first at 97.0%. The third row is 0.8 points behind. The broader top-10 range is 5.0 points, so many of the published results sit in a relatively narrow band.
88 models have been evaluated on SWE-bench (Vals). The benchmark falls in the Coding category. This category carries a 20% weight in BenchLM.ai's overall scoring system. SWE-bench (Vals) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About SWE-bench (Vals)
Year
2026
Tasks
Real repository issues by human time bucket
Format
Resolved rate
Difficulty
Frontier coding agents
BenchLM mirrors the Vals AI board on a dedicated key so a provider-run row on the canonical key is never overwritten. Vals publishes per-task accuracy with standard error, latency, and cost for every model it runs. Admitted as independent third-party evidence in methodology v5.5 (2026-09-04).
BenchLM freshness & provenance
Version
SWE-bench (Vals) 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does SWE-bench (Vals) measure?
Vals AI’s independent run of the public SWE-bench issue set, reported by human time-to-fix bucket. Vals removed SWE-bench Verified from its index as saturated on 2026-05-04; this board is the standalone SWE-bench run.
Which model leads the published SWE-bench (Vals) snapshot?
Claude Opus 5 currently leads the published SWE-bench (Vals) snapshot with 97.0% vals swe-bench score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on SWE-bench (Vals)?
The September 1, 2026 snapshot contains 88 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.