A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
BenchLM mirrors the public DeepSWE leaderboard from Datacurve. DeepSWE evaluates coding agents on original, long-horizon software engineering tasks from active open-source repositories, using isolated task environments and program-based verifiers.
DeepSWE is display only on BenchLM. The public rows include model, agent harness, and reasoning-effort settings, so BenchLM does not use these scores as weighted model-only ranking inputs.
BenchLM mirrors the published solve rate view for DeepSWE. gpt-5.5[xhigh] leads the public snapshot at 70% , followed by gpt-5.4[xhigh] (56%) and claude-opus-4.7[max] (54%). BenchLM does not use these results to rank models overall.
gpt-5.5[xhigh]
OpenAI
gpt-5.4[xhigh]
OpenAI
claude-opus-4.7[max]
Anthropic
The published DeepSWE snapshot is tightly clustered at the top: gpt-5.5[xhigh] sits at 70%, while the third row is only 16 points behind. The broader top-10 spread is 60 points, so the benchmark still separates strong models even when the leaders cluster.
12 models have been evaluated on DeepSWE. The benchmark falls in the External benchmark mirrors category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. DeepSWE is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
Year
2026
Tasks
113 software engineering tasks
Format
Solve rate
Difficulty
Long-horizon software engineering
DeepSWE includes original tasks with isolated environments and program-based verifiers. BenchLM mirrors the public DeepSWE leaderboard as display-only because each row combines a model, agent harness, and reasoning-effort setting rather than a pure model-only benchmark score.
Version
DeepSWE 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.
gpt-5.5[xhigh] currently leads the published DeepSWE snapshot with 70% solve rate. BenchLM shows this benchmark for display only and does not use it in overall rankings.
12 AI models are included in BenchLM's mirrored DeepSWE snapshot, based on the public leaderboard captured on May 2026.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.