Live source snapshot
DAI-ASR-I18N multilingual speech recognition leaderboard
David AI’s DAI-ASR-I18N measures how accurately speech recognition systems transcribe and separate speakers in real, unscripted two-person conversations across 21 languages. In version 1.4 (November 10, 2026), MAI-Transcribe-2 has the best average rank for transcription at a 9.50% mean error rate, and Nemotron 3.5 Transcribe leads speaker diarization.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 21 languages, about 7 hours of conversation each (about 2 public and 5 private), 147 hours in all; 20 transcription systems and 17 diarization systems in version 1.4
- Primary metric
- Word error rate, or character error rate for Japanese, Korean, Thai and Mandarin (%, lower is better); diarization error rate for speaker separation (%, lower is better)
- Owner
- David AI
- Available evidence
- Public leaderboard with every version, MIT-licensed evaluation code and a public test set under an evaluation-only data use agreement
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
How the protocol works
- 01
Pairs of native speakers hold natural, unscripted conversations, each recorded on its own microphone, and annotators transcribe them verbatim with word timestamps.
- 02
For transcription, each speaker’s channel is transcribed separately; references and outputs pass through language-adapted normalizers before word (or character) error rate is computed.
- 03
For diarization, systems get the combined audio and are scored by diarization error rate against the reference speaker turns.
- 04
Systems are ordered by average rank across the 21 languages; a language a system does not support counts as one place below the last system, so narrow coverage costs rank.
Available results
Transcription
Version 1.4, separate speaker channels. Mean is the unweighted mean of the per-language word or character error rates a system covers (%, lower is better); public and private are the same mean on each test set alone. Average rank is David AI’s ordering.
| System | Access | Avg. rank | Mean error | Public set | Private set | Languages |
|---|---|---|---|---|---|---|
| MAI-Transcribe-2 | Proprietary | 1.33 | 9.50 | 9.74 | 9.41 | 21 / 21 |
| Scribe v3 | Proprietary | 4.38 | 14.02 | 14.14 | 13.98 | 21 / 21 |
| Scribe v2 | Proprietary | 4.67 | 14.26 | 14.53 | 14.15 | 21 / 21 |
| Nemotron 3.5 Transcribe | Open weights | 5.05 | 14.63 | 15.10 | 14.45 | 21 / 21 |
| Grok Voice Transcribe 2.0 | Proprietary | 6.62 | 9.36 | 9.60 | 9.27 | 17 / 21 |
| Gemini 3.5 Transcribe | Proprietary | 7.10 | 13.76 | 14.03 | 13.66 | 20 / 21 |
| Muse Voice Transcribe 1.0 | Proprietary | 7.90 | 15.42 | 15.93 | 15.22 | 20 / 21 |
| GPT-Transcribe | Proprietary | 8.24 | 16.16 | 17.26 | 15.73 | 21 / 21 |
| Voxtral Small Transcribe 3.0 | Proprietary | 8.81 | 26.08 | 22.95 | 27.43 | 21 / 21 |
| Reson8 Rumble-2 | Proprietary | 9.43 | 17.50 | 18.45 | 17.12 | 20 / 21 |
| Voxtral Mini Transcribe 3.0 | Proprietary | 9.43 | 39.63 | 48.55 | 35.86 | 21 / 21 |
| Qwen3-ASR-1.7B | Open weights | 12.19 | 15.67 | 15.99 | 15.56 | 17 / 21 |
| gpt-4o-transcribe-diarize | Proprietary | 13.29 | 34.30 | 34.74 | 34.14 | 21 / 21 |
| Whisper large-v3 | Open weights | 14.29 | 36.06 | 36.95 | 35.72 | 21 / 21 |
| Nemotron 3.5 ASR | Open weights | 16.33 | 23.05 | 22.87 | 23.11 | 14 / 21 |
| Voxtral Mini Transcribe 2.0 | Proprietary | 17.24 | 18.94 | 19.83 | 18.59 | 12 / 21 |
| Nova 2 Sonic | Proprietary | 18.33 | 14.87 | 15.64 | 14.57 | 7 / 21 |
| Parakeet TDT 0.6B v3 | Open weights | 19.29 | 24.47 | 27.15 | 23.43 | 7 / 21 |
| Cohere Transcribe (03-2026) | Open weights | 19.33 | 38.83 | 43.62 | 37.12 | 11 / 21 |
| Granite Speech 5.0 2B | Open weights | 20.48 | 41.46 | 56.63 | 35.52 | 6 / 21 |
Transcription by language
Version 1.4, both test sets combined. Word error rate except Japanese, Korean, Thai and Mandarin, which use character error rate (%, lower is better). A dash means the system does not support the language.
| System | AR WER | BN WER | DE WER | EN WER | ES WER | FR WER | HI WER | ID WER | IT WER | JA CER | KO CER | MR WER | PT WER | RU WER | TA WER | TE WER | TH CER | TL WER | TR WER | VI WER | ZH CER |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MAI-Transcribe-2 | 9.59 | 15.99 | 6.39 | 4.20 | 4.92 | 7.25 | 8.76 | 8.91 | 7.09 | 5.64 | 4.10 | 11.63 | 6.07 | 8.09 | 25.62 | 28.83 | 5.33 | 12.52 | 6.95 | 7.02 | 4.60 |
| Scribe v3 | 22.65 | 18.82 | 7.68 | 3.22 | 5.08 | 12.28 | 15.60 | 15.47 | 9.92 | 6.53 | 5.11 | 21.55 | 8.26 | 16.59 | 35.73 | 44.53 | 7.04 | 14.72 | 10.76 | 8.37 | 4.59 |
| Scribe v2 | 19.23 | 19.80 | 7.44 | 2.75 | 5.22 | 12.61 | 17.85 | 14.98 | 9.85 | 7.01 | 6.19 | 22.13 | 8.27 | 15.94 | 40.93 | 44.27 | 7.23 | 16.74 | 7.27 | 9.34 | 4.36 |
| Nemotron 3.5 Transcribe | 15.90 | 22.40 | 8.39 | 4.12 | 5.41 | 12.54 | 11.53 | 18.19 | 9.27 | 7.54 | 5.06 | 19.20 | 8.67 | 12.83 | 39.73 | 35.78 | 7.69 | 30.05 | 13.36 | 13.89 | 5.65 |
| Grok Voice Transcribe 2.0 | 16.21 | — | 7.99 | 3.91 | 5.66 | 11.20 | 11.15 | 11.32 | 10.45 | 6.48 | 4.43 | — | 8.07 | 10.34 | — | — | 7.03 | 15.73 | 14.35 | 9.05 | 5.72 |
| Gemini 3.5 Transcribe | 23.40 | 18.34 | 10.95 | 4.81 | 7.16 | 13.19 | 15.71 | 14.33 | 11.16 | 7.55 | 5.77 | 18.20 | 8.78 | 14.28 | — | 37.97 | 10.06 | 17.90 | 14.82 | 11.11 | 9.70 |
| Muse Voice Transcribe 1.0 | 20.55 | 17.06 | 14.26 | 5.87 | 10.43 | 18.09 | 11.63 | 15.32 | 12.94 | 7.59 | 5.81 | 19.61 | 12.39 | — | 32.70 | 46.66 | 8.86 | 17.25 | 14.49 | 11.51 | 5.38 |
| GPT-Transcribe | 22.20 | 24.23 | 13.90 | 5.79 | 10.06 | 15.82 | 13.22 | 15.79 | 12.78 | 8.10 | 6.98 | 20.97 | 11.22 | 15.65 | 33.25 | 39.41 | 10.40 | 20.79 | 16.97 | 14.14 | 7.79 |
| Voxtral Small Transcribe 3.0 | 26.35 | 66.79 | 9.88 | 3.70 | 7.86 | 14.34 | 89.09 | 17.84 | 10.94 | 9.78 | 7.08 | 35.55 | 11.51 | 14.08 | 29.97 | 35.31 | 10.89 | 75.93 | 48.60 | 14.17 | 7.93 |
| Reson8 Rumble-2 | 28.46 | 26.91 | 13.46 | 4.60 | 9.20 | 15.90 | 13.39 | 15.21 | 12.57 | 10.69 | 6.64 | 41.21 | 13.32 | 14.76 | 48.95 | — | 9.92 | 22.52 | 17.68 | 17.11 | 7.45 |
| Voxtral Mini Transcribe 3.0 | 78.10 | 292.12 | 10.80 | 4.73 | 39.07 | 14.31 | 84.03 | 16.46 | 11.44 | 10.17 | 6.96 | 44.77 | 11.11 | 14.02 | 88.14 | 42.37 | 9.54 | 17.73 | 16.48 | 13.75 | 6.10 |
| Qwen3-ASR-1.7B | 37.67 | — | 12.73 | 5.01 | 9.58 | 16.02 | 14.51 | 17.31 | 13.20 | 8.21 | 6.62 | — | 14.65 | 16.31 | — | — | 10.41 | 43.38 | 21.09 | 15.01 | 4.70 |
| gpt-4o-transcribe-diarize | 36.23 | 102.73 | 18.48 | 7.36 | 14.72 | 21.23 | 31.36 | 22.08 | 17.02 | 19.70 | 12.90 | 81.22 | 17.50 | 19.72 | 73.45 | 112.42 | 27.04 | 29.68 | 22.22 | 20.96 | 12.30 |
| Whisper large-v3 | 35.21 | 87.42 | 20.11 | 10.31 | 19.02 | 24.45 | 42.62 | 22.76 | 22.60 | 21.63 | 17.75 | 88.12 | 17.46 | 21.50 | 83.56 | 94.29 | 24.89 | 29.71 | 23.21 | 33.74 | 16.92 |
| Nemotron 3.5 ASR | 30.95 | — | 15.91 | 8.49 | 17.11 | 18.04 | 17.67 | — | 17.59 | 18.96 | 14.50 | — | 16.35 | 22.86 | — | — | — | — | 66.14 | 46.02 | 12.12 |
| Voxtral Mini Transcribe 2.0 | 35.67 | — | 19.81 | 7.78 | 14.36 | 23.32 | 25.91 | — | 16.96 | 20.08 | 13.74 | — | 16.61 | 19.12 | — | — | — | — | — | — | 13.98 |
| Nova 2 Sonic | — | — | 14.32 | 5.98 | 11.16 | 17.76 | 23.12 | — | 14.64 | — | — | — | 17.13 | — | — | — | — | — | — | — | — |
| Parakeet TDT 0.6B v3 | — | — | 17.19 | 5.37 | 32.38 | 53.98 | — | — | 19.61 | — | — | — | 23.68 | 19.10 | — | — | — | — | — | — | — |
| Cohere Transcribe (03-2026) | 105.84 | — | 21.74 | 12.40 | 20.04 | 28.69 | — | — | 28.84 | 25.01 | 49.74 | — | 27.86 | — | — | — | — | — | — | 54.92 | 52.08 |
| Granite Speech 5.0 2B | — | — | 54.22 | 11.32 | 34.94 | 44.28 | — | — | — | 42.20 | — | — | 61.79 | — | — | — | — | — | — | — | — |
Speaker diarization
Version 1.4, combined audio. Overall is the diarization error rate pooled over all clips (%, lower is better); public and private are the same pooled rate on each test set alone.
| System | Access | Avg. rank | Overall DER | Public set | Private set | Languages |
|---|---|---|---|---|---|---|
| Nemotron 3.5 Transcribe | Open weights | 1.81 | 18.26 | 18.55 | 18.15 | 21 / 21 |
| Nemotron 3 Diarization | Open weights | 2.38 | 18.38 | 18.83 | 18.20 | 21 / 21 |
| Scribe v2 | Proprietary | 5.24 | 23.37 | 23.30 | 23.39 | 21 / 21 |
| Scribe v3 | Proprietary | 5.38 | 23.39 | 23.41 | 23.38 | 21 / 21 |
| MAI-Transcribe-2 | Proprietary | 5.57 | 22.85 | 23.12 | 22.74 | 21 / 21 |
| DiariZen WavLM Large s80 MD v2 | Open weights | 6.24 | 23.60 | 23.94 | 23.47 | 21 / 21 |
| Sortformer 4spk-v2.1 | Open weights | 7.24 | 23.95 | 24.88 | 23.58 | 21 / 21 |
| pyannote Precision-3 | Proprietary | 7.33 | 24.27 | 24.88 | 24.04 | 21 / 21 |
| Grok Voice Transcribe 2.0 | Proprietary | 8.52 | 24.69 | 24.56 | 24.75 | 17 / 21 |
| pyannote Community-1 | Open weights | 8.67 | 25.44 | 25.75 | 25.32 | 21 / 21 |
| Gemini 3.5 Transcribe | Proprietary | 10.95 | 28.76 | 29.42 | 28.50 | 20 / 21 |
| Voxtral Mini Transcribe 3.0 | Proprietary | 11.62 | 30.73 | 31.55 | 30.41 | 21 / 21 |
| Voxtral Small Transcribe 3.0 | Proprietary | 12.14 | 30.91 | 31.56 | 30.65 | 21 / 21 |
| Muse Voice Transcribe 1.0 | Proprietary | 14.71 | 35.91 | 36.78 | 35.57 | 20 / 21 |
| Reson8 Rumble-2 | Proprietary | 15.10 | 39.79 | 40.69 | 39.44 | 20 / 21 |
| Voxtral Mini Transcribe 2.0 | Proprietary | 15.62 | 33.14 | 33.28 | 33.08 | 12 / 21 |
| gpt-4o-transcribe-diarize | Proprietary | 15.62 | 37.55 | 38.53 | 37.17 | 21 / 21 |
Version history
Every version David AI has published, with the date in its changelog and the system ranked first by average rank on each board.
| Version | Date | Transcription systems | Transcription leader (avg. rank) | Diarization systems | Diarization leader (avg. rank) |
|---|---|---|---|---|---|
| v1.4 | November 10, 2026 | 20 | MAI-Transcribe-2 (1.33) | 17 | Nemotron 3.5 Transcribe (1.81) |
| v1.3 | October 28, 2026 | 15 | MAI-Transcribe-2 (1.24) | 15 | Nemotron 3.5 Transcribe (1.67) |
| v1.2 | October 19, 2026 | 12 | MAI-Transcribe-2 (1.19) | 13 | Nemotron 3 Diarization (1.48) |
| v1.1.2 | October 19, 2026 | 10 | MAI-Transcribe-2 (1.14) | 11 | Nemotron 3 Diarization (1.48) |
| v1.1 | October 2, 2026 | 10 | MAI-Transcribe-2 (1.14) | 11 | Nemotron 3 Diarization (1.43) |
Interpretation limit
David AI runs the board and chose the systems and inference settings, following each provider’s documented configuration. The mean error rate averages only the languages a system covers, so a system with few languages can show a low mean while ranking low; compare average rank and coverage together. Character and word error rates are averaged together in the mean. The private set (about 5 hours per language) is held back to limit contamination, and the combined figures pool both sets. Conversations are two speakers on separate microphones, which can make diarization easier than single-microphone or larger-group audio. Snapshot of version 1.4, retrieved November 11, 2026; earlier versions are listed below. Scores are credited to David AI; its site terms describe personal, non-commercial use. These results do not enter text-model or voice-agent rankings.