Live source snapshot
AudioAgentBench
Evaluates real-time voice agents on policy compliance, tool use, grounding, ambiguity, and state tracking.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 6 service workflows; synthetic and human audio
- Primary metric
- Checks passed
- Owner
- Arcada Labs
- Available evidence
- Machine-readable public run records
What it measures
Task completion
Can it follow policy, use tools, and finish a workflow?
Conversation dynamics
Can it handle turns, interruptions, ambiguity, and state?
Available results
Aggregated from benchmark-owner run records
| Rank | Model | Checks passed | Evidence |
|---|---|---|---|
| 1 | gpt-live-1 | 90.3% | 12,142 / 13,440 |
| 2 | grok-voice-3.0 | 89.6% | 8,589 / 9,591 |
| 3 | gpt-realtime-2.1@high | 80.4% | 1,082 / 1,345 |
| 4 | Grok Voice Think Fast 2.0 | 80.2% | 10,617 / 13,232 |
| 5 | Grok Voice Think Fast 1.0 | 79.9% | 5,370 / 6,722 |
| 6 | Grok Realtime | 78.3% | 10,526 / 13,440 |
| 7 | Ultravox v0.7 | 78.1% | 9,367 / 11,991 |
| 8 | GPT Realtime | 77.5% | 10,415 / 13,440 |
| 9 | Gemini 2.5 Flash Native Audio | 74.7% | 10,045 / 13,440 |
| 10 | GPT Realtime 2 | 73.1% | 4,909 / 6,720 |
| 11 | Gemini 3.1 Flash Live | 71.3% | 9,583 / 13,440 |
| 12 | Amazon Nova 2 Sonic | 53.7% | 5,999 / 11,163 |
| 13 | GLM Realtime Flash | 29.6% | 2,722 / 9,195 |
| 14 | GLM Realtime Air | 18.3% | 510 / 2,790 |
Interpretation limit
Scores are judge-derived checks aggregated across recent public runs. Human and synthesized audio remain separate views.