Toolathlon Verified Pass@3
We show this table for reference; we do not rank on it.
Fraction of Toolathlon Verified tasks solved in at least one of three trials.
Benchmark score on Toolathlon Verified Pass@3 — November 13, 2026
We compile the Toolathlon Verified Pass@3 rows from provider self-reports. Claude Opus 5 leads the table at 87.0%, followed by Claude Sonnet 5.5 (85.2%) and Claude Opus 5.5 (82.4%). We do not use these results to rank models overall.
Claude Opus 5
Anthropic
Reported by AnthropicClaude Sonnet 5.5
Anthropic
Reported by AnthropicClaude Opus 5.5
Anthropic
Reported by Anthropic4 modelsAgenticCurrentDisplay onlyUpdated November 13, 2026
Benchmark score table (4 models)
All 4 results on this page were reported by the labs that built the models.
| Rank | Model / configuration | Score | Who ran it | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Opus 5AnthropicReported by Anthropic | 87.0% | Reported by Anthropic | Not reported | Closed |
| 2 | Claude Sonnet 5.5AnthropicReported by Anthropic | 85.2% | Reported by Anthropic | Not reported | Closed |
| 3 | Claude Opus 5.5AnthropicReported by Anthropic | 82.4% | Reported by Anthropic | Not reported | Closed |
| 4 | Claude Fable 5.1AnthropicReported by Anthropic | 81.5% | Reported by Anthropic | Not reported | Closed |
- Lab-reported
- Benchmark-run
- Independent
Among the reported Toolathlon Verified Pass@3 rows, Claude Opus 5 is first at 87.0%. The third row is 4.6 points behind. The broader top-10 range is 5.5 points, so many of the published results sit in a relatively narrow band.
4 models have been evaluated on Toolathlon Verified Pass@3. The benchmark falls in the Agentic category. Toolathlon Verified Pass@3 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About Toolathlon Verified Pass@3
Year
2026
Tasks
108 verified real-world tool-use tasks
Format
Pass@3
Difficulty
Long-horizon application tool use
Table 8.13.6.A reports this alongside Pass@1, Pass³, and average turns from Anthropic's pinned internal harness.
Freshness and provenance
Version
Toolathlon Verified Pass@3 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Toolathlon Verified Pass@3 measure?
Fraction of Toolathlon Verified tasks solved in at least one of three trials.
Which model scores highest on Toolathlon Verified Pass@3?
Claude Opus 5 by Anthropic currently leads with a score of 87.0% on Toolathlon Verified Pass@3.
How many models are evaluated on Toolathlon Verified Pass@3?
4 AI models have published results on Toolathlon Verified Pass@3 in the BenchLM catalog.
Compare top models on Toolathlon Verified Pass@3
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.