Official provider results
Jais-ASR Arabic speech evaluation
IFM reports 22.85% word error rate and 8.37% character error rate, macro-averaged over six Arabic speech test sets. Lower error rates are better.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- Six public Arabic ASR test sets; separate Mixat code-switched test
- Primary metric
- Word error rate and character error rate (%, lower is better)
- Owner
- IFM
- Available evidence
- Exact provider tables in the model card and October 2026 launch blog
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
Available results
Arabic transcription accuracy
IFM-reported WER and CER (%). Lower is better. The published average is a macro-average across the six test sets.
| Test set | WER (%) | CER (%) |
|---|---|---|
| Casablanca (8 dialects) | 46.56 | 18.02 |
| Common Voice 18 (MSA) | 8.56 | 2.70 |
| MASC clean | 16.12 | 3.76 |
| MASC noisy | 25.44 | 7.51 |
| MGB-2 | 9.59 | 3.92 |
| SADA | 30.85 | 14.33 |
| Macro-average (six sets) | 22.85 | 8.37 |
Emirati code-switched speech (Mixat)
Separate 1,585-clip Mixat test set; Jais-ASR in code-switch mode.
| Model | WER (%) | CER (%) |
|---|---|---|
| Jais-ASR | 17.8 | 6.0 |
Interpretation limit
IFM evaluated Jais-ASR using the public leaderboard script. These are provider-reported measurements, not independently verified leaderboard submissions. The six-set Arabic macro-average, Mixat, and internal Arabic Speech Benchmark have different samples and aggregation rules and are not pooled. Speech error rates stay separate from weighted text-model scores. October 15 is the source retrieval date, not an established evaluation or first-release day.