Skip to main content
BenchLM
Data
Official provider results

Scribe v3 launch evaluation

ElevenLabs reports 12.38% mean word error rate across 102 FLEURS languages for Scribe v3, down from 16.94% for Scribe v2, and the lowest diarization error of the batch models it tested. Scribe v3 Live cuts semantic WER by 43% against Scribe v2 Realtime and posts 3.1% realtime diarization error on English recordings.

Source snapshots refreshed
Five measurement lanesVOICE / S2S

Reasoning, task completion, conversation dynamics, experience, and latency stay separate.

Back to directory

Benchmark profile

Scope
An internal copy of FLEURS (short read speech, 102 languages), an unreleased ElevenLabs set of real meetings, agent calls and voice notes in 16 languages, and, for Scribe v3 Live only, Earnings-22 and VoxPopuli
Primary metric
Word error rate, semantic word error rate and diarization error rate (%, lower is better)
Owner
ElevenLabs
Available evidence
Exact values printed on the launch post’s charts and in the model documentation. ElevenLabs says it will open-source the 16-language set; it was not public on November 11, 2026.

What it measures

Voice experience
Is the exchange natural, robust, and responsive?

Available results

Batch transcription on FLEURS

ElevenLabs-reported, lower is better; internal FLEURS tests run October 23–29, 2026. Scribe v2 scored 16.94% WER (docs; 17.0% on the chart) and 5.4% semantic WER on the same 102 languages; Gemini 3.5 Transcribe 18.5% and 7.5%. On the languages each provider supports, ElevenLabs reports Scribe v3 against MAI-Transcribe-2 (60 languages) 6.5% vs 7.5% WER, GPT-Transcribe (58) 5.5% vs 9.7%, Deepgram Nova-3 (54) 5.8% vs 15.2%, Muse Voice Transcribe (25) 6.6% vs 16.6%, and AssemblyAI Universal-3.5 Pro (18) 4.0% vs 7.2%.

ElevenLabs-reported, lower is better; internal FLEURS tests run October 23–29, 2026. Scribe v2 scored 16.94% WER (docs; 17.0% on the chart) and 5.4% semantic WER on the same 102 languages; Gemini 3.5 Transcribe 18.5% and 7.5%. On the languages each provider supports, ElevenLabs reports Scribe v3 against MAI-Transcribe-2 (60 languages) 6.5% vs 7.5% WER, GPT-Transcribe (58) 5.5% vs 9.7%, Deepgram Nova-3 (54) 5.8% vs 15.2%, Muse Voice Transcribe (25) 6.6% vs 16.6%, and AssemblyAI Universal-3.5 Pro (18) 4.0% vs 7.2%.
ModelWER, 102 languagesSemantic WER, 102 languagesChange in WER vs Scribe v2
Scribe v312.38%4.4%−27%

Batch speaker diarization

ElevenLabs-reported diarization error rate (lower is better) on its unreleased set of real meetings, agent calls and voice notes in 16 languages, run October 26 and November 3, 2026. All recordings: Scribe v2 6.4%, MAI-Transcribe-2 8.5%, Muse Voice Transcribe 8.8%, AssemblyAI Universal-3.5 Pro 10.4%, Gemini 3.5 Transcribe 10.6%, Deepgram Nova-3 14.2%. Three or more speakers: Scribe v2 9.6%, Muse Voice Transcribe 10.6%, AssemblyAI Universal-3.5 Pro 13.7%, Gemini 3.5 Transcribe 14.3%, MAI-Transcribe-2 15.0%, Deepgram Nova-3 18.4%.

ElevenLabs-reported diarization error rate (lower is better) on its unreleased set of real meetings, agent calls and voice notes in 16 languages, run October 26 and November 3, 2026. All recordings: Scribe v2 6.4%, MAI-Transcribe-2 8.5%, Muse Voice Transcribe 8.8%, AssemblyAI Universal-3.5 Pro 10.4%, Gemini 3.5 Transcribe 10.6%, Deepgram Nova-3 14.2%. Three or more speakers: Scribe v2 9.6%, Muse Voice Transcribe 10.6%, AssemblyAI Universal-3.5 Pro 13.7%, Gemini 3.5 Transcribe 14.3%, MAI-Transcribe-2 15.0%, Deepgram Nova-3 18.4%.
ModelDER, all recordingsDER, 3+ speakers
Scribe v35.6%8.0%

Realtime transcription and diarization

ElevenLabs-reported, lower is better. Diarization is on the English recordings of the same unreleased set, run November 4, 2026, with each transcript aligned to the reference before scoring: Grok Voice Transcribe 2.0 5.9%, AssemblyAI Universal-3.6 Pro Realtime 6.1%, Deepgram Nova-3 7.3%. Scribe v2 Realtime scored 5.99% on Earnings-22 and 2.13% on VoxPopuli. FLEURS semantic WER on November 3, 2026, Scribe v3 Live vs Scribe v2 Realtime: Hindi 4.0% vs 6.3%, Japanese 3.6% vs 5.1%, Greek 2.3% vs 9.9%, Danish 3.7% vs 8.0%.

ElevenLabs-reported, lower is better. Diarization is on the English recordings of the same unreleased set, run November 4, 2026, with each transcript aligned to the reference before scoring: Grok Voice Transcribe 2.0 5.9%, AssemblyAI Universal-3.6 Pro Realtime 6.1%, Deepgram Nova-3 7.3%. Scribe v2 Realtime scored 5.99% on Earnings-22 and 2.13% on VoxPopuli. FLEURS semantic WER on November 3, 2026, Scribe v3 Live vs Scribe v2 Realtime: Hindi 4.0% vs 6.3%, Japanese 3.6% vs 5.1%, Greek 2.3% vs 9.9%, Danish 3.7% vs 8.0%.
ModelRealtime DER, English (aligned)FLEURS semantic WER vs Scribe v2 RealtimeFLEURS WER vs Scribe v2 RealtimeEarnings-22 WERVoxPopuli WER
Scribe v3 Live3.1%−43%−31%3.91%1.71%

Interpretation limit

ElevenLabs ran every result itself, on an internal version of FLEURS and on its own unreleased dataset, so nothing here is independently reproducible yet. Each FLEURS language is scored on its own and then averaged, which ElevenLabs notes gives higher numbers than pooling clips, so these WERs are not comparable with pooled results elsewhere. The comparisons against other providers use only the languages both models support. The realtime diarization figure aligns each transcript to the reference first and is not comparable with the batch diarization figures. The launch post reports Scribe v3 Live on 102 FLEURS languages; the documentation reports 73. Semantic WER counts an error only when the meaning changes. These transcription results stay separate from BenchLM’s weighted text-model ranking.