# Scribe v3 launch evaluation: Voice Benchmark Profile

> ElevenLabs reports 12.38% mean word error rate across 102 FLEURS languages for Scribe v3, down from 16.94% for Scribe v2, and the lowest diarization error of the batch models it tested. Scribe v3 Live cuts semantic WER by 43% against Scribe v2 Realtime and posts 3.1% realtime diarization error on English recordings.

- Scope: An internal copy of FLEURS (short read speech, 102 languages), an unreleased ElevenLabs set of real meetings, agent calls and voice notes in 16 languages, and, for Scribe v3 Live only, Earnings-22 and VoxPopuli
- Measurement lanes: Voice experience
- Primary metric: Word error rate, semantic word error rate and diarization error rate (%, lower is better)
- Available evidence: Exact values printed on the launch post’s charts and in the model documentation. ElevenLabs says it will open-source the 16-language set; it was not public on November 11, 2026.
- Owner: ElevenLabs
- Source snapshots refreshed: 2026-11-10

- Provider report retrieved: 2026-11-11

## Batch transcription on FLEURS

ElevenLabs-reported, lower is better; internal FLEURS tests run October 23–29, 2026. Scribe v2 scored 16.94% WER (docs; 17.0% on the chart) and 5.4% semantic WER on the same 102 languages; Gemini 3.5 Transcribe 18.5% and 7.5%. On the languages each provider supports, ElevenLabs reports Scribe v3 against MAI-Transcribe-2 (60 languages) 6.5% vs 7.5% WER, GPT-Transcribe (58) 5.5% vs 9.7%, Deepgram Nova-3 (54) 5.8% vs 15.2%, Muse Voice Transcribe (25) 6.6% vs 16.6%, and AssemblyAI Universal-3.5 Pro (18) 4.0% vs 7.2%.

| Model | WER, 102 languages | Semantic WER, 102 languages | Change in WER vs Scribe v2 |
| --- | --- | --- | --- |
| [Scribe v3](/models/scribe-v3) | 12.38% | 4.4% | −27% |

## Batch speaker diarization

ElevenLabs-reported diarization error rate (lower is better) on its unreleased set of real meetings, agent calls and voice notes in 16 languages, run October 26 and November 3, 2026. All recordings: Scribe v2 6.4%, MAI-Transcribe-2 8.5%, Muse Voice Transcribe 8.8%, AssemblyAI Universal-3.5 Pro 10.4%, Gemini 3.5 Transcribe 10.6%, Deepgram Nova-3 14.2%. Three or more speakers: Scribe v2 9.6%, Muse Voice Transcribe 10.6%, AssemblyAI Universal-3.5 Pro 13.7%, Gemini 3.5 Transcribe 14.3%, MAI-Transcribe-2 15.0%, Deepgram Nova-3 18.4%.

| Model | DER, all recordings | DER, 3+ speakers |
| --- | --- | --- |
| [Scribe v3](/models/scribe-v3) | 5.6% | 8.0% |

## Realtime transcription and diarization

ElevenLabs-reported, lower is better. Diarization is on the English recordings of the same unreleased set, run November 4, 2026, with each transcript aligned to the reference before scoring: Grok Voice Transcribe 2.0 5.9%, AssemblyAI Universal-3.6 Pro Realtime 6.1%, Deepgram Nova-3 7.3%. Scribe v2 Realtime scored 5.99% on Earnings-22 and 2.13% on VoxPopuli. FLEURS semantic WER on November 3, 2026, Scribe v3 Live vs Scribe v2 Realtime: Hindi 4.0% vs 6.3%, Japanese 3.6% vs 5.1%, Greek 2.3% vs 9.9%, Danish 3.7% vs 8.0%.

| Model | Realtime DER, English (aligned) | FLEURS semantic WER vs Scribe v2 Realtime | FLEURS WER vs Scribe v2 Realtime | Earnings-22 WER | VoxPopuli WER |
| --- | --- | --- | --- | --- | --- |
| [Scribe v3 Live](/models/scribe-v3-live) | 3.1% | −43% | −31% | 3.91% | 1.71% |

## Interpretation limit

ElevenLabs ran every result itself, on an internal version of FLEURS and on its own unreleased dataset, so nothing here is independently reproducible yet. Each FLEURS language is scored on its own and then averaged, which ElevenLabs notes gives higher numbers than pooling clips, so these WERs are not comparable with pooled results elsewhere. The comparisons against other providers use only the languages both models support. The realtime diarization figure aligns each transcript to the reference first and is not comparable with the batch diarization figures. The launch post reports Scribe v3 Live on 102 FLEURS languages; the documentation reports 73. Semantic WER counts an error only when the meaning changes. These transcription results stay separate from BenchLM’s weighted text-model ranking.

## Primary sources

- [Owner page](https://elevenlabs.io/blog/scribe-v3)
- [Launch post](https://elevenlabs.io/blog/scribe-v3)
- [Model documentation](https://elevenlabs.io/docs/overview/models)
- [API pricing](https://elevenlabs.io/pricing/api)

Canonical page: https://benchlm.ai/voice-benchmarks/scribe-v3-launch-evaluation