Skip to main content
BenchLM
Data
Voice systems

Voice benchmarks, separated by what they measure

No single score covers every voice-agent workload. These tables keep reasoning, workflow success, conversation dynamics, experience, and latency in their own protocols.

Source snapshots refreshed
Five measurement lanesVOICE / S2S

Reasoning, task completion, conversation dynamics, experience, and latency stay separate.

08

Spoken reasoning

Does the system understand and answer spoken requests?

11

Task completion

Can it follow policy, use tools, and finish a workflow?

17

Conversation dynamics

Can it handle turns, interruptions, ambiguity, and state?

35

Voice experience

Is the exchange natural, robust, and responsive?

23

Latency

How long do responses, tool calls, and full tasks take?

Current source-backed results

The tabs are separate leaderboards, not ingredients in a BenchLM ranking. Audio Realism uses blind listening preferences; VoiceBench uses its own overall score; AudioAgentBench shows checks passed; Full-Duplex-Bench reports strict pass@1 and seconds.

Protocol details
Audio-realism arena results: Elo-style rating, uncertainty, win rate, battles, and generation time per model.
RankModelElo-style ratingBT uncertaintyWin rateBattlesAvg. generation
1Freya TTS Adam V1
Freya
1442±23.879.3%3812.41s
2Kalpa TTS Beta v0.1
Kalpa Labs
1380±18.876.3%5862.91s
3Bland Speech v3
Bland AI
1379±18.477.8%6261.40s
4Cartesia Sonic 3.6
Cartesia
1341±27.473.6%2501.82s
5Deepgram Flux TTS
deepgram
1310±29.770.2%2087.21s
6ElevenLabs Eleven v3 Conversational
ElevenLabs
1238±17.764.6%5082.61s
7Murf Falcon 2
Murf AI
1232±27.663.3%2181.62s
8Cartesia Sonic 3.5
Cartesia
1218±13.963.1%8421.61s
9MAI-Voice-2
Microsoft
1208±15.861.1%6401.40s
10HeyGen Voice
heygen
1185±21.762.2%30414.96s
11Grok TTS
xAI
1179±15.258.3%6671.79s
12MAI-Voice-2-Flash
microsoft
1137±31.051.2%1661.65s
13MiniMax Speech 2.8 HD
MiniMax
1135±26.651.1%2177.08s
14MiniMax Speech-02 HD
MiniMax
1121±15.050.9%6742.95s
15Gemini 2.5 Pro TTS Preview
Google
1110±14.749.6%6885.96s
16GPT Realtime 2
OpenAI
1103±19.947.2%3622.46s
17Google Gemini 3.8 Flash TTS
google
1080±18.148.2%4235.01s
18GPT Realtime 2.1
OpenAI
1064±24.546.2%2363.21s
19Eleven v4
elevenlabs
1031±21.442.1%3042.53s
20GPT-4o mini TTS
OpenAI
1029±15.340.3%6501.99s
21Gemini 3.1 Flash TTS Preview
Google
1025±15.137.2%6834.20s
22Voxtral TTS 3
mistral
1019±21.441.3%3002.26s
23Eleven v4 Turbo
elevenlabs
998±20.738.3%3321.90s
24ElevenLabs Eleven v3
ElevenLabs
970±15.730.5%6732.83s
25Gemini 2.5 Flash TTS Preview
Google
970±15.432.5%6904.32s
26Fish Audio Drama 3
fish-audio
958±21.134.2%3302.01s
27Lightning v3.1 Pro
Smallest AI
955±15.831.0%6751.73s
28Fish Audio S2.1 Pro
Fish Audio
953±20.433.2%3732.77s
29Magpie TTS Multilingual
nvidia
875±17.524.3%58815.20s
30Inworld Realtime TTS-2
Inworld
860±28.624.7%2192.56s
31Qwen-Audio-3.0 TTS Flash
alibaba
854±33.323.0%1742.76s
32Inworld TTS-1.5 Max
Inworld
789±19.315.1%6674.11s

Snapshot sources: benchmark-owner leaderboard and run records. Full-Duplex-Bench v3 values are transcribed from the official paper table.

Benchmark coverage

Voice benchmarks with the capability lanes each one covers and the evidence status.
BenchmarkSpoken reasoningTask completionConversation dynamicsVoice experienceLatencyEvidence
Dessa launch recordings

Block shares live recordings of people talking with Dessa, a full-duplex model trained only on human speech. Block publishes no benchmark scores; the model is waitlist-only and coming soon.

Provider results

Launch page with 6 featured and 15 additional recordings, October 2026

Eleven Realtime provider evaluation

ElevenLabs reports separate speech-reasoning, conversation, human-preference, task-success, and response-latency measurements for Eleven Realtime v1 Mini and Max.

Provider results

Provider-published selected figures; no shared overall score

Voxtral 3 launch evaluation

Mistral reports 4.0% FLEURS word error rate and 18.4% diarization cpWER for Voxtral Small Transcribe 3, the lowest cpWER of the seven systems in its chart.

Provider results

Exact values in the chart data of the October 20, 2026 launch post

Ming-3.0 speech evaluation

inclusionAI reports 0.800% Chinese and 1.584% English word error rate on the Seed-TTS testset for Ming-3.0-tts-0.5B, and 3.45 Chinese CER with 3.69 English WER for Ming-3.0-audio-flash.

Provider results

Exact values in the Hugging Face model cards; the audio and ASR cards publish them as table images

What the current results can answer

Which TTS model sounds most human?

Start with Audio Realism. Its blind pairwise listening protocol covers selected American-English voices across phone-agent, conversational, and explainer prompts.

Which model answers spoken questions best?

Start with VoiceBench. Check its architecture and individual task columns before relying on the owner-defined overall score.

Which agent completes service workflows?

Use AudioAgentBench for recent public run aggregates, then compare human and synthesized audio rather than merging them.

Which full-duplex system balances success and speed?

Use Full-Duplex-Bench v3. Strict pass@1 and latency answer different questions and should be inspected together.