Live source snapshot
Audio Realism Benchmark
Blind listening comparisons measure how human text-to-speech output sounds across phone-agent, conversational, and explainer speech.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 500 held-out American-English prompts; female and male voices; human recordings included in the comparison pool
- Primary metric
- Bradley–Terry Elo-style rating, win rate, and uncertainty
- Owner
- Design Arena
- Available evidence
- Machine-readable public leaderboard
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
How the protocol works
- 01
Vetted native speakers hear two clips generated from the same prompt and select the one that sounds more human.
- 02
The benchmark uses 500 private, held-out prompts split across phone-agent, conversational, and explainer speech.
- 03
One female and one male American-English voice are selected for each model, and comparisons use the same gender.
- 04
Scores are fitted with a Bradley–Terry model and presented as an Elo-style rating; real human recordings remain in the comparison pool.
Available results
| Rank | Model | Elo-style rating | BT uncertainty | Win rate | Battles | Avg. generation |
|---|---|---|---|---|---|---|
| 1 | Freya TTS Adam V1 Freya | 1442 | ±23.8 | 79.3% | 381 | 2.41s |
| 2 | Kalpa TTS Beta v0.1 Kalpa Labs | 1380 | ±18.8 | 76.3% | 586 | 2.91s |
| 3 | Bland Speech v3 Bland AI | 1379 | ±18.4 | 77.8% | 626 | 1.40s |
| 4 | Cartesia Sonic 3.6 Cartesia | 1341 | ±27.4 | 73.6% | 250 | 1.82s |
| 5 | Deepgram Flux TTS deepgram | 1310 | ±29.7 | 70.2% | 208 | 7.21s |
| 6 | ElevenLabs Eleven v3 Conversational ElevenLabs | 1238 | ±17.7 | 64.6% | 508 | 2.61s |
| 7 | Murf Falcon 2 Murf AI | 1232 | ±27.6 | 63.3% | 218 | 1.62s |
| 8 | Cartesia Sonic 3.5 Cartesia | 1218 | ±13.9 | 63.1% | 842 | 1.61s |
| 9 | MAI-Voice-2 Microsoft | 1208 | ±15.8 | 61.1% | 640 | 1.40s |
| 10 | HeyGen Voice heygen | 1185 | ±21.7 | 62.2% | 304 | 14.96s |
| 11 | Grok TTS xAI | 1179 | ±15.2 | 58.3% | 667 | 1.79s |
| 12 | MAI-Voice-2-Flash microsoft | 1137 | ±31.0 | 51.2% | 166 | 1.65s |
| 13 | MiniMax Speech 2.8 HD MiniMax | 1135 | ±26.6 | 51.1% | 217 | 7.08s |
| 14 | MiniMax Speech-02 HD MiniMax | 1121 | ±15.0 | 50.9% | 674 | 2.95s |
| 15 | Gemini 2.5 Pro TTS Preview Google | 1110 | ±14.7 | 49.6% | 688 | 5.96s |
| 16 | GPT Realtime 2 OpenAI | 1103 | ±19.9 | 47.2% | 362 | 2.46s |
| 17 | Google Gemini 3.8 Flash TTS google | 1080 | ±18.1 | 48.2% | 423 | 5.01s |
| 18 | GPT Realtime 2.1 OpenAI | 1064 | ±24.5 | 46.2% | 236 | 3.21s |
| 19 | Eleven v4 elevenlabs | 1031 | ±21.4 | 42.1% | 304 | 2.53s |
| 20 | GPT-4o mini TTS OpenAI | 1029 | ±15.3 | 40.3% | 650 | 1.99s |
| 21 | Gemini 3.1 Flash TTS Preview Google | 1025 | ±15.1 | 37.2% | 683 | 4.20s |
| 22 | Voxtral TTS 3 mistral | 1019 | ±21.4 | 41.3% | 300 | 2.26s |
| 23 | Eleven v4 Turbo elevenlabs | 998 | ±20.7 | 38.3% | 332 | 1.90s |
| 24 | ElevenLabs Eleven v3 ElevenLabs | 970 | ±15.7 | 30.5% | 673 | 2.83s |
| 25 | Gemini 2.5 Flash TTS Preview Google | 970 | ±15.4 | 32.5% | 690 | 4.32s |
| 26 | Fish Audio Drama 3 fish-audio | 958 | ±21.1 | 34.2% | 330 | 2.01s |
| 27 | Lightning v3.1 Pro Smallest AI | 955 | ±15.8 | 31.0% | 675 | 1.73s |
| 28 | Fish Audio S2.1 Pro Fish Audio | 953 | ±20.4 | 33.2% | 373 | 2.77s |
| 29 | Magpie TTS Multilingual nvidia | 875 | ±17.5 | 24.3% | 588 | 15.20s |
| 30 | Inworld Realtime TTS-2 Inworld | 860 | ±28.6 | 24.7% | 219 | 2.56s |
| 31 | Qwen-Audio-3.0 TTS Flash alibaba | 854 | ±33.3 | 23.0% | 174 | 2.76s |
| 32 | Inworld TTS-1.5 Max Inworld | 789 | ±19.3 | 15.1% | 667 | 4.11s |
Interpretation limit
The leaderboard measures perceived human-likeness for the selected American-English voices and prompt mix. It does not measure factual accuracy, task completion, full-duplex interaction, or multilingual quality.