Benchmark profile
Toolathlon Verified Pass@3
Fraction of Toolathlon Verified tasks solved in at least one of three trials.
Data verifiedBenchmark score on Toolathlon Verified Pass@3 — July 24, 2026
BenchLM mirrors the published score view for Toolathlon Verified Pass@3. Claude Opus 5 leads the public snapshot at 87.0%. BenchLM does not use these results to rank models overall.
Benchmark score table (1 model)
ScoreAbout Toolathlon Verified Pass@3
Year
2026
Tasks
108 verified real-world tool-use tasks
Format
Pass@3
Difficulty
Long-horizon application tool use
Table 8.13.6.A reports this alongside Pass@1, Pass³, and average turns from Anthropic's pinned internal harness.
BenchLM freshness & provenance
Version
Toolathlon Verified Pass@3 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does Toolathlon Verified Pass@3 measure?
Fraction of Toolathlon Verified tasks solved in at least one of three trials.
Which model scores highest on Toolathlon Verified Pass@3?
Claude Opus 5 by Anthropic currently leads with a score of 87.0% on Toolathlon Verified Pass@3.
How many models are evaluated on Toolathlon Verified Pass@3?
1 AI models have been evaluated on Toolathlon Verified Pass@3 on BenchLM.