FrontierSWE v2
We show this table for reference; we do not rank on it.
A 34-task expansion of FrontierSWE for ultra-long-horizon engineering and research work that remains far from saturation.
Previous release
The original 17-task FrontierSWE remains available as an archived table. It ranks model-and-harness pairs by dominance rather than mean task score, so movement between the two tables is not a like-for-like model comparison.
View the original FrontierSWEMean@5 score on FrontierSWE v2 — October 15, 2026
We mirror the published mean@5 score view for FrontierSWE v2. GPT-6 Astra leads the public snapshot at 65.5%, followed by Claude Opus 5.5 (62.3%) and Claude Fable 5.1 (56.3%). We do not use these results to rank models overall.
GPT-6 Astra
OpenAI
proximus
Claude Opus 5.5
Anthropic
proximus
Claude Fable 5.1
Anthropic
proximus
21 modelsCoding8% of Coding reference weightCurrentUpdated October 15, 2026
Mean@5 score table (21 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraOpenAIproximus | 65.5% | Not reported | Closed |
| 2 | Claude Opus 5.5Anthropicproximus | 62.3% | Not reported | Closed |
| 3 | Claude Fable 5.1Anthropicproximus | 56.3% | Not reported | Closed |
| 4 | Gemini 4 ArgonGoogleproximus | 55.0% | Not reported | Closed |
| 5 | Claude Opus 5Anthropicproximus | 52.0% | Not reported | Closed |
| 6 | Claude Fable 5Anthropicproximus | 47.0% | Not reported | Closed |
| 7 | Claude Haiku 5.5Anthropicproximus | 43.8% | Not reported | Closed |
| 8 | GPT-5.6 SolOpenAIproximus | 32.2% | Not reported | Closed |
| 9 | GLM-5.3Z.AIproximus | 30.2% | Not reported | Open |
| 10 | Grok 4.7xAIproximus | 29.5% | Not reported | Closed |
| 11 | Kimi K3Moonshot AIproximus | 25.9% | Not reported | Pending |
| 12 | Muse Spark 1.3Metaproximus | 25.8% | Not reported | Closed |
| 13 | Grok 4.6xAIproximus | 25.3% | Not reported | Closed |
| 14 | Gemini 3.7 FlashGoogleproximus | 20.3% | Not reported | Closed |
| 15 | Gemini 3.8 FlashGoogleproximus | 19.6% | Not reported | Closed |
| 16 | GLM-5.3-FlashZ.AIproximus | 18.1% | Not reported | Open |
| 17 | Qwen3.8-Max-0902Alibabaproximus | 17.8% | Not reported | Unknown |
| 18 | Qwen3.8 MaxAlibabaproximus | 15.8% | Not reported | Open |
| 19 | DeepSeek V4 Flash Vision ExpDeepSeekproximus | 14.8% | Not reported | Unknown |
| 20 | Muse Spark 1.2Metaproximus | 12.0% | Not reported | Closed |
| 21 | InklingThinking Machines Labproximus | 4.1% | Not reported | Open |
How we show FrontierSWE v2
We mirror Proximal's live Mean@5 table across all 34 v2 tasks. Every model runs in Proximal's own Proximus harness at maximum reasoning effort, with 5 trials per task and a 20-hour budget. Scores are the mean task score on a 0-100 scale, not the dominance metric used by the original release.
Rows on this page are benchmark-owned. Models that only appear in provider material, such as the Claude Opus 5 and Claude Fable 5 results in Anthropic's Fable 5.1 system card, keep provider-exact rows on their model pages and do not appear here.
Snapshot
The published FrontierSWE v2 snapshot places GPT-6 Astra first at 65.5%. The third row is 9.2 points behind. The broader top-10 range is 36.0 points, so the table still separates the published systems.
21 models have been evaluated on FrontierSWE v2. The benchmark falls in the Coding category. BenchLM shows the published table for reference; the per-model scores it stores feed the ranking. BenchAlign v5.8 gives FrontierSWE v2 8% of the Coding reference weight, so it moves the Coding leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.
About FrontierSWE v2
Year
2026
Tasks
34 ultra-long-horizon engineering and research tasks
Format
Five-trial mean task score (Mean@5), 0-100
Difficulty
Ultra-long-horizon frontier software engineering
FrontierSWE v2 keeps 13 of the original 17 tasks, retires four as saturated or unreliably scorable, and adds 21 new tasks across visual reasoning, scientific computing, and AI research. Proximal runs every model at maximum reasoning effort in its own Proximus harness with a 20-hour budget and five trials per task, and publishes mean, best, and worst scores on a 0-100 scale. BenchLM mirrors the public Mean@5 table as benchmark-owned rows; models that only appear in provider material stay provider-exact on the same scale.
Freshness and provenance
Version
FrontierSWE v2 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does FrontierSWE v2 measure?
A 34-task expansion of FrontierSWE for ultra-long-horizon engineering and research work that remains far from saturation.
Which model leads the published FrontierSWE v2 snapshot?
GPT-6 Astra currently leads the published FrontierSWE v2 snapshot with 65.5% mean@5 score. BenchLM shows the published table for reference; the per-model scores it stores feed the ranking.
How many models are evaluated on FrontierSWE v2?
The October 15, 2026 snapshot contains 21 AI models.
How is FrontierSWE v2 different from the original FrontierSWE?
The original release had 17 tasks. Version 2 keeps 13 of them, retires four that Proximal found saturated or impossible to score deterministically (PCQM4Mv2 gap prediction, Pyright type-checking optimization, Revideo rendering optimization, and the dependent type checker), and adds 21 new tasks across visual reasoning, scientific computing, and AI research, for 34 in total. Every model now runs in Proximal's Proximus harness, and each task reports a normalized 0-100 score instead of a dominance rank.
Can FrontierSWE v2 scores be compared with the original FrontierSWE?
No. The two releases are not comparable: the task set changed, the harness changed from each model's native agent to Proximus, the headline metric changed from dominance to mean task score, and performance tasks are now measured by a weighted instruction count on a pinned CPU microarchitecture rather than wall-clock time in shared sandboxes. BenchLM keeps the original 17-task table on its own page for that reason.
What is the Proximus harness?
Proximus is the minimal coding-agent harness Proximal built for ultra-long-horizon tasks. It compacts context by summarizing the trajectory, supports images and plots, and lets a model record a clean workspace as a candidate submission while reporting how much of the 20-hour budget remains, so models keep working instead of submitting early. Proximal reports that Claude Opus 5 and especially GPT-5.6 Sol score higher under Proximus than under their native harnesses.
How does FrontierSWE v2 guard against gaming the verifier?
Each workspace ships a documented self-check tool whose output tracks the verifier score, so models can measure progress honestly. Verification then applies one-to-one structural test mutation to defeat hardcoded answers, and it runs in a fresh verifier container booted from a pinned image after the agent's container is stopped. The agent also runs as a non-root user with root-owned protected directories.
Where do the Claude Opus 5 and Claude Fable 5 results come from?
Proximal's public v2 table does not list those two models. Their scores come from Anthropic's Claude Fable 5.1 and Claude Mythos 5.1 system card, which reports Proximal's own five-trial runs. BenchLM shows them on the model pages as provider-exact rows on the same 0-100 scale but keeps them off this benchmark-owned leaderboard.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.