Toloka Arena
We show this table for reference; we do not rank on it.
An independent agentic-intelligence evaluation from Toloka using private simulated workflows and a pass^5 metric.
pass^5 score on Toloka Arena — Captured 2026-11-12
We mirror the published pass^5 score view for Toloka Arena. Fable 5.2 leads the public snapshot at 77.0%, followed by Opus 5.5 (75.0%) and Gemini 4 Argon (73.0%). We do not use these results to rank models overall.
Fable 5.2
Anthropic
Opus 5.5
Anthropic
Gemini 4 Argon
39 modelsAgenticCurrentDisplay onlyUpdated Captured 2026-11-12
pass^5 score table (39 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Fable 5.2Anthropic | 77.0% | Not reported | Unknown |
| 2 | Opus 5.5Anthropic | 75.0% | Not reported | Unknown |
| 3 | Gemini 4 ArgonGoogle | 73.0% | Not reported | Closed |
| 4 | Fable 5.1Anthropic | 71.0% | Not reported | Unknown |
| 5 | GPT-6 AstraOpenAI | 67.0% | Not reported | Closed |
| 6 | GPT-6.1 SolOpenAI | 66.0% | Not reported | Closed |
| 7 | Muse WonderMeta | 65.7% | Not reported | Closed |
| 8 | Sonnet 5.5Anthropic | 64.0% | Not reported | Unknown |
| 9 | Opus 5Anthropic | 62.0% | Not reported | Unknown |
| 10 | GPT-5.6 SolOpenAI | 61.0% | Not reported | Closed |
| 11 | Gemini 3.8 FlashGoogle | 59.0% | Not reported | Closed |
| 12 | Gemini 3.7 FlashGoogle | 58.0% | Not reported | Closed |
| 13 | GPT-6 SolOpenAI | 58.0% | Not reported | Closed |
| 14 | Grok 4.6xAI | 56.0% | Not reported | Closed |
| 15 | GPT-6.1 TerraOpenAI | 55.0% | Not reported | Closed |
| 16 | MiMo V2.6 ProXiaomi | 51.0% | Not reported | Unknown |
| 17 | Kimi K3.1 ProMoonshot AI | 49.0% | Not reported | Open |
| 18 | Qwen 3.8 Max 0902Alibaba | 44.0% | Not reported | Unknown |
| 19 | Reflection BeamReflection AI | 40.0% | Not reported | Unknown |
| 20 | Qwen 3.8 27BAlibaba | 36.0% | Not reported | Unknown |
| 21 | Tencent Hy4Tencent | 32.0% | Not reported | Unknown |
| 22 | Grok 4.7xAI | 31.0% | Not reported | Closed |
| 23 | GPT-6 LunaOpenAI | 30.0% | Not reported | Closed |
| 24 | Mistral Large 4Mistral | 29.0% | Not reported | Open |
| 25 | DeepSeek V4.1 FlashDeepSeek | 26.0% | Not reported | Unknown |
| 26 | Minimax M3.1 FlashMiniMax | 22.0% | Not reported | Unknown |
| 27 | Haiku 5.5Anthropic | 21.0% | Not reported | Unknown |
| 28 | Sonnet 5Anthropic | 19.0% | Not reported | Unknown |
| 29 | Muse Spark OpenMeta | 18.0% | Not reported | Open |
| 30 | Step 5 PreviewStepFun | 9.0% | Not reported | Open |
| 31 | Gemini 3.1 ProGoogle | 8.0% | Not reported | Closed |
| 32 | GLM 5.3Z.AI | 7.0% | Not reported | Unknown |
| 33 | Kimi K3Moonshot AI | 7.0% | Not reported | Open |
| 34 | Nemotron 3.5 Super VLNVIDIA | 6.0% | Not reported | Unknown |
| 35 | Flint v1.0Recursive | 1.0% | Not reported | Unknown |
| 36 | Cohere Command A+Cohere | 0.0% | Not reported | Unknown |
| 37 | Granite 4.2 8BIBM | 0.0% | Not reported | Open |
| 38 | LFM2.5 24B A2BLiquid AI | 0.0% | Not reported | Unknown |
| 39 | Mellum2.1 12B A2.5B ThinkingJetBrains | 0.0% | Not reported | Unknown |
How Toloka Arena is shown here
We mirror Toloka Arena's published results for 39 models, with the published pass^5 scores and confidence intervals. The tasks simulate enterprise workflows with tools, databases, policies, and multiple turns.
Toloka says it refined the criteria and refreshed the task set for this table. Its default configuration uses medium reasoning, 16,384 maximum tokens, and temperature 0.6 unless stated otherwise. The source does not publish a task count. We keep these agent results outside weighted rankings.
Snapshot
The published Toloka Arena snapshot places Fable 5.2 first at 77.0%. The third row is 4.0 points behind. The broader top-10 range is 16.0 points, so the table still separates the published systems.
39 models have been evaluated on Toloka Arena. The benchmark falls in the Agentic category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Toloka Arena is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Toloka Arena
Year
2026
Tasks
Private simulated enterprise workflows
Format
pass^5 arena score
Difficulty
Agentic workflow reliability
Toloka Arena evaluates agents on private simulated workflows with tools, databases, policies, and multi-turn tasks. This table mirrors the published pass^5 results and confidence intervals from the source dataset v3. Toloka says it refined the criteria and refreshed the task set. Its default configuration uses medium reasoning, 16,384 maximum tokens, and temperature 0.6 unless stated otherwise. These agent results remain outside weighted rankings.
Freshness and provenance
Version
Toloka Arena 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Toloka Arena measure?
An independent agentic-intelligence evaluation from Toloka using private simulated workflows and a pass^5 metric.
Which model leads the published Toloka Arena snapshot?
Fable 5.2 currently leads the published Toloka Arena snapshot with 77.0% pass^5 score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Toloka Arena?
The Captured 2026-11-12 snapshot contains 39 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.