A map of the voice benchmark landscape

Block shares live recordings of people talking with Dessa, a full-duplex model trained only on human speech. Block publishes no benchmark scores; the model is waitlist-only and coming soon.

ElevenLabs reports separate speech-reasoning, conversation, human-preference, task-success, and response-latency measurements for Eleven Realtime v1 Mini and Max.

Mistral reports 4.0% FLEURS word error rate and 18.4% diarization cpWER for Voxtral Small Transcribe 3, the lowest cpWER of the seven systems in its chart.

inclusionAI reports 0.800% Chinese and 1.584% English word error rate on the Seed-TTS testset for Ming-3.0-tts-0.5B, and 3.45 Chinese CER with 3.69 English WER for Ming-3.0-audio-flash.

Liquid AI reports 6.23% English ASR word error rate, 3.83 UTMOSv2 English speech quality and 73.13 on VoiceBench for LFM3-Audio-2B, and 70.09 on Daily-Omni with 46.32 on OmniBench for LFM3-Omni-2B.

NVIDIA reports 4.71% mean WER on the nine-set English Open ASR Leaderboard, 10.98% on the long-form board, 4.62% on the seven-language multilingual board and 15.80% mean cpWER on full-session multi-speaker recordings for Nemotron 3.5 Transcribe, plus 73.40% on MMAU and 66.67% on MMAR with reasoning on.

IFM reports 22.85% word error rate and 8.37% character error rate, macro-averaged over six Arabic speech test sets. Lower error rates are better.

HeyGen reports a 76.1/100 voice-clone rating from 43 people judging their own digital twins. A separate platform study gave full-clone 41.7% of 6,153 three-way preference votes.

xAI reports 74% EVA-Bench accuracy for Grok Voice 3.0. The model launched in the Voice Agent API on October 14, 2026 at $0.08 per minute.

Microsoft lists two multilingual text-to-speech models at different latency and character-price points.

Microsoft launched streaming transcription in 60 languages at an introductory $0.54 per hour through the end of 2026.

Artificial Analysis reports 2.5% word error rate for final and first-partial transcripts from MAI-Transcribe-2-Streaming.

Phonon-2 has a reported 5.21% mean word error rate across seven English test sets. Fermion's launch report compares eight speech models and measures throughput across hardware and Mac runtimes.

Google’s September 23 launch post reports Hume voice-design and quality placements for Gemini 3.8 Flash TTS and Flash-Lite TTS.

xAI’s September 18, 2026 launch post reports a rank on an independent streaming speech-to-text leaderboard, four internal word-error-rate sets drawn from production traffic, and one exact multilingual short-phrase pair against Grok Voice Transcribe 1.0.

Qwen’s September 18, 2026 launch post reports a 32-row audio and audio-visual table for Qwen3.8-Omni-Flash against Qwen3.5-Omni-Plus, Gemini 3.8 Flash, Seed 2.0 Lite, and Muse Spark 1.2, plus a separate static-versus-agent comparison on three video sets.

Wispr’s September 17, 2026 launch post reports word error rates for Canto against five competing transcription systems on two private dictation sets and three public English datasets.

Google’s September 15 model evaluation reports separate results for Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking on spoken-agent task suites.

OpenAI’s model page documents the modalities, limits, and pricing of gpt-audio-1.5; it publishes no benchmark scores for the model.

OpenAI’s model page documents endpoints, features, and per-minute pricing for gpt-transcribe; it publishes no word-error-rate table on the page.

OpenAI’s model page documents the realtime-only endpoint, latency controls, and per-minute pricing for gpt-live-transcribe; it publishes no accuracy table on the page.

Google’s August 26, 2026 launch post reports word error rates on an independent transcription leaderboard and FLEURS for pre-recorded and streaming transcription, plus a latency gain over Chirp 3.

Microsoft AI’s September 3, 2026 launch post reports FLEURS word error rate across 60 languages, an independent leaderboard position, and relative speed against competing transcription models.

Microsoft’s June 2, 2026 Foundry post reports the FLEURS word error rate improvement for MAI-Transcribe-1.5 over MAI-Transcribe-1.

Microsoft AI’s July 23, 2026 post positions MAI-Voice-2-Flash as a faster, cheaper sibling of MAI-Voice-2 without publishing benchmark scores.

Cartesia’s August 27, 2026 launch post reports independent controlled-voice Elo ratings, blind preference win rates against Eleven v3, response latency, and locale coverage for Sonic 3.6.

Inworld’s launch post reports an independent speech-arena ranking, time-to-first-audio, and language coverage for Realtime TTS-2.

Inworld’s model documentation reports time-to-first-byte and coverage for the Flash variant of Realtime TTS-2.

ByteDance Seed’s August 5, 2026 announcement describes an end-to-end audio-visual full-duplex model and reports a halving of conversational pacing issues versus cascaded pipelines.

Tencent’s Hunyuan site describes Hy ASR 3.0 preview as a Hy3-based recogniser with context-aware correction and broad dialect coverage; launch coverage cites word error rates near 3%.

Deepgram’s October 2, 2025 launch announcement reports end-of-turn detection latency and Nova-3-level accuracy for Flux.

Deepgram’s April 29, 2026 press release reports end-of-turn latency and language coverage for Flux Multilingual.

Kyutai’s TTS page and repository document Pocket TTS’s size, CPU real-time factor, and first-audio latency.

Zyphra’s ZONOS2 GGUF model card reports word error rate, speaker similarity, and UTMOS for the reference F16 build.

IBM’s model card and Granite 4.2 blog document the model’s size, training data, and throughput; Open ASR leaderboard results are published as images.

Cohere’s catalog and Hugging Face repository document the Arabic transcription model’s lineage and availability; no evaluation table is published.

Microsoft’s model card documents streaming speaker-attributed transcription, hotword support, and ten-language coverage; evaluation results are published as an image.

Qwen’s model card reports word error rates for Qwen3-ASR 1.7B and 0.6B across AMI, Earnings22, GigaSpeech, LibriSpeech, SPGISpeech, and VoxPopuli.

Tencent’s model card documents AuK’s task coverage across speech generation, editing, enhancement, and separation; its benchmark chart is published as an image.

OpenAI’s API launch post reports spoken task success, full-duplex interactivity, tool calling, response quality, and turn-taking latency for GPT-Live-1 against GPT-Realtime-2.1 and GPT-Realtime-2.

Meta’s launch evaluation reports streaming transcription accuracy, speaker diarization, and real-time audio-perception capabilities for Muse Voice Transcribe.

Blind listening comparisons measure how human text-to-speech output sounds across phone-agent, conversational, and explainer speech.

A spoken question-answering suite spanning open-ended, knowledge, reasoning, instruction-following, and safety tasks.

Measures expert-level understanding and reasoning across speech, environmental sound, and music.

Tests harder audio reasoning across speech, sound, music, spatial audio, multiple clips, voice chat, and instruction following.

Evaluates real-time voice agents on policy compliance, tool use, grounding, ambiguity, and state tracking.

Tests full-duplex voice agents on multi-tool tasks with human speech, disfluencies, interruptions, and latency measurements.

Extends τ-bench into spoken customer-service tasks to measure the capability gap between text and voice agents.

Separates enterprise task accuracy from conversational experience across realistic service scenarios.

Measures the silence callers experience between finishing a turn and hearing a voice-agent platform begin its reply.

A multilingual voice-agent suite covering tools, workflows, multi-turn interaction, and safety.

Measures how voice systems handle ambiguous, disfluent, and underspecified spoken dialogue.