Skip to main content
BenchLM

Toolathlon Verified Pass@3

We show this table for reference; we do not rank on it.

Data verified 34 confirmed releases in the last 30 daysFollow model changes

Fraction of Toolathlon Verified tasks solved in at least one of three trials.

Benchmark score on Toolathlon Verified Pass@3 — September 27, 2026

We compile the Toolathlon Verified Pass@3 rows from provider self-reports. Claude Opus 5 leads the table at 87.0%, followed by Claude Opus 5.5 (82.4%) and Claude Fable 5.1 (81.5%). We do not use these results to rank models overall.

1Proprietary

Claude Opus 5

Anthropic

87.0%
Overall 79.81
2Proprietary

Claude Opus 5.5

Anthropic

82.4%
Overall 86.98Context 1M
3Proprietary

Claude Fable 5.1

Anthropic

81.5%
Overall 83Context 1M

3 modelsAgenticCurrentDisplay onlyUpdated September 27, 2026

Benchmark score table (3 models)

Score
1
Claude Opus 5Anthropic · Closed
87.0%
2
Claude Opus 5.5Anthropic · Closed
82.4%
3
Claude Fable 5.1Anthropic · Closed
81.5%

Among the reported Toolathlon Verified Pass@3 rows, Claude Opus 5 is first at 87.0%. The third row is 5.5 points behind. The broader top-10 range is 5.5 points, so many of the published results sit in a relatively narrow band.

3 models have been evaluated on Toolathlon Verified Pass@3. The benchmark falls in the Agentic category. Toolathlon Verified Pass@3 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About Toolathlon Verified Pass@3

Year

2026

Tasks

108 verified real-world tool-use tasks

Format

Pass@3

Difficulty

Long-horizon application tool use

Table 8.13.6.A reports this alongside Pass@1, Pass³, and average turns from Anthropic's pinned internal harness.

Claude Opus 5 System Card

Freshness and provenance

Version

Toolathlon Verified Pass@3 2026

Refresh cadence

Quarterly

Staleness state

Current

Question availability

Public benchmark set

CurrentDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does Toolathlon Verified Pass@3 measure?

Fraction of Toolathlon Verified tasks solved in at least one of three trials.

Which model scores highest on Toolathlon Verified Pass@3?

Claude Opus 5 by Anthropic currently leads with a score of 87.0% on Toolathlon Verified Pass@3.

How many models are evaluated on Toolathlon Verified Pass@3?

3 AI models have been evaluated on Toolathlon Verified Pass@3 on BenchLM.

Compare top models on Toolathlon Verified Pass@3

Claude Opus 5 vs Claude Opus 5.5Claude Opus 5.5 vs Claude Fable 5.1
Last updated: September 27, 2026 · BenchLM version Toolathlon Verified Pass@3 2026

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.