Tool-Agent-User Benchmark (TAU-bench)
Original TAU-bench evaluates customer-service agents that talk to a simulated user and update a database through tools. Its pass^k metric measures success in all k repeated trials, so average success alone does not establish reliability. This page publishes no model scores until the source, domain, and metric are identified. Original TAU-bench paper.
We show this table for reference; we do not rank on it.
Original TAU-bench tests an agent's tool use and policy compliance in simulated airline and retail conversations. Its pass^k metric asks whether the agent succeeds in every one of k repeated trials.
How to read original TAU-bench results
| Measurement | How to read it |
|---|---|
| Task domains | Airline and retail customer service, reported separately. |
| pass^1 | Average task success over one trial; the paper uses it as the main comparison metric. |
| pass^k | The chance that all k repeated trials succeed, averaged across tasks. It measures consistency, unlike pass@k, which requires at least one success. |
| Comparison boundary | Match the original task release, domain, agent and user models, tools, prompting, trial count, and metric. τ²-bench and τ³-bench use separate result pages. |
Method source: Original TAU-bench paper, evaluation method.
How to read this benchmark
Editorial review by Glevd · 2026-07-15
Read this route as the owner of original 2024 TAU-bench methodology. Compare a result only when the domain, archived task release, agent and user models, prompting strategy, tools, number of trials, and pass^k metric are identified. Do not merge it with τ² or τ³ results.
Operator receipt: 0 sourced rows are currently displayable on this page.
Honest limit: BenchLM stores 37 raw numbers under the original key, but none has an exact source attachment or the domain and pass^k labels needed for a valid comparison. The route therefore publishes no score table. The archived repository also warns that its airline and retail tasks are outdated.
About TAU-bench
Year
2024
Tasks
Airline and retail task sets in the archived 2024 release
Format
Domain-specific pass^1 through pass^4 task success
Difficulty
Policy-constrained, multi-turn customer service
The original release reports airline and retail separately and measures reliability with pass^1 through pass^4 across repeated trials. Its repository now warns that those task files are outdated and directs evaluators to the maintained τ³-bench repository.
Freshness and provenance
Version
TAU-bench 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
Why is there no original TAU-bench score table?
The 37 stored raw numbers do not have exact source attachments or the domain and pass^k labels required for a valid comparison. BenchLM withholds them until those receipts are attached instead of presenting unsupported values as a leaderboard.
Is original TAU-bench still the current release?
No. The original repository warns that its airline and retail task files are outdated and points evaluators to the maintained τ³-bench repository for corrected tasks and newer evaluation modes.
What does original TAU-bench measure?
It measures whether an agent can complete airline or retail customer-service tasks through multi-turn conversation and domain tools while following policy. The paper reports domain-specific pass^k reliability across repeated trials.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.