Live source snapshot
Audio Realism Benchmark
Blind listening comparisons measure how human text-to-speech output sounds across phone-agent, conversational, and explainer speech.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 500 held-out American-English prompts; female and male voices; human recordings included in the comparison pool
- Primary metric
- Bradley–Terry Elo-style rating, win rate, and uncertainty
- Owner
- Design Arena
- Available evidence
- Machine-readable public leaderboard
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
How the protocol works
- 01
Vetted native speakers hear two clips generated from the same prompt and select the one that sounds more human.
- 02
The benchmark uses 500 private, held-out prompts split across phone-agent, conversational, and explainer speech.
- 03
One female and one male American-English voice are selected for each model, and comparisons use the same gender.
- 04
Scores are fitted with a Bradley–Terry model and presented as an Elo-style rating; real human recordings remain in the comparison pool.
Available results
| Rank | Model | Elo-style rating | BT uncertainty | Win rate | Battles | Avg. generation |
|---|---|---|---|---|---|---|
| 1 | Bland Speech v3 Bland AI | 1365 | ±23.9 | 82.5% | 372 | 1.68s |
| 2 | MAI-Voice-2 Microsoft | 1224 | ±19.8 | 68.9% | 376 | 1.50s |
| 3 | Grok TTS xAI | 1156 | ±18.8 | 60.9% | 378 | 2.27s |
| 4 | Gemini 2.5 Pro TTS Preview Google | 1109 | ±18.2 | 54.9% | 390 | 6.95s |
| 5 | Cartesia Sonic 3.5 Cartesia | 1096 | ±18.2 | 51.6% | 395 | 1.55s |
| 6 | MiniMax Speech-02 HD MiniMax | 1093 | ±18.6 | 52.4% | 380 | 3.36s |
| 7 | Gemini 3.1 Flash TTS Preview Google | 1037 | ±18.4 | 41.5% | 388 | 4.99s |
| 8 | GPT-4o mini TTS OpenAI | 1011 | ±18.9 | 42.8% | 362 | 2.01s |
| 9 | Gemini 2.5 Flash TTS Preview Google | 985 | ±18.9 | 37.9% | 385 | 5.16s |
| 10 | Lightning v3.1 Pro Smallest AI | 952 | ±19.4 | 34.0% | 380 | 1.94s |
| 11 | ElevenLabs Eleven v3 ElevenLabs | 938 | ±20.1 | 29.1% | 364 | 3.35s |
| 12 | Inworld TTS-1.5 Max Inworld | 763 | ±24.9 | 14.2% | 373 | 4.97s |
Interpretation limit
The leaderboard measures perceived human-likeness for the selected American-English voices and prompt mix. It does not measure factual accuracy, task completion, full-duplex interaction, or multilingual quality.