Live source snapshot
Artificial Analysis speech-to-text leaderboard (non-streaming)
Artificial Analysis tests batch speech-to-text models through their APIs on three English sets and ranks them by the AA-WER Index. On November 12, 2026, Fun-Realtime-ASR-preview has the lowest index of 64 models at 1.73%, and SOV-Transcribe-1, Salesforce leads the 16 open-weight models at 2.28%.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 64 models (16 with open weights) and 65 model-provider rows; about 8 hours of English audio across AA-AgentTalk (AA's held-out voice-agent set, 469 clips), VoxPopuli (European Parliament speech) and Earnings22 (earnings calls), cleaned by Artificial Analysis
- Primary metric
- AA-WER Index: word error rate weighted 50% AA-AgentTalk, 25% VoxPopuli, 25% Earnings22 (%, lower is better); speed factor (seconds of audio per second, higher is faster) and price per 1,000 minutes
- Owner
- Artificial Analysis
- Available evidence
- Public leaderboard with per-set error rates, speed and price for each model-provider row, and a published methodology
What it measures
Voice experience
Is the exchange natural, robust, and responsive?
How the protocol works
- 01
Artificial Analysis calls each provider’s API directly, so results reflect what a customer of that endpoint gets.
- 02
Where a model accepts a prompt, it gets the same one: “Transcribe the audio verbatim, outputting only spoken words in sequence.”
- 03
Each set’s word error rate is a time-weighted average over its clips after text normalization; the AA-WER Index weights AA-AgentTalk 50%, VoxPopuli 25% and Earnings22 25%.
- 04
A model tested on several providers is ranked by its best row; the provider table lists every row with its median speed factor over the last seven days and its price per 1,000 minutes.
Available results
Models by AA-WER Index
Each model’s best model-provider row, lowest index first (%, lower is better). Rank and open-weights rank are Artificial Analysis’s ordering by the AA-WER Index; tied values share a place.
| Model | Creator | Weights | Rank | Open-weights rank | AA-WER Index | AA-AgentTalk | VoxPopuli | Earnings22 |
|---|---|---|---|---|---|---|---|---|
| Fun-Realtime-ASR-preview | Alibaba | Proprietary | #1 | — | 1.73 | 1.97 | 1.17 | 1.80 |
| Scribe v3, ElevenLabs | ElevenLabs | Proprietary | #2 | — | 1.95 | 1.47 | 1.52 | 3.34 |
| MAI-Transcribe-2 | Microsoft AI | Proprietary | #3 | — | 2.04 | 1.83 | 1.50 | 2.99 |
| Voxtral Small Transcribe 3 | Mistral | Proprietary | #4 | — | 2.09 | 1.65 | 1.51 | 3.55 |
| Scribe v2, ElevenLabs | ElevenLabs | Proprietary | #5 | — | 2.18 | 1.51 | 1.74 | 3.95 |
| Voxtral Mini Transcribe 3 | Mistral | Proprietary | #6 | — | 2.22 | 2.02 | 1.45 | 3.40 |
| SOV-Transcribe-1, Salesforce | Salesforce | Open | #7 | #1 | 2.28 | 1.79 | 1.50 | 4.06 |
| Grok Voice Transcribe 2.0 | SpaceXAI | Proprietary | #8 | — | 2.29 | 2.34 | 1.28 | 3.22 |
| MAI-Transcribe-1.5 | Microsoft AI | Proprietary | #9 | — | 2.38 | 1.96 | 1.62 | 3.96 |
| StepAudio 3 ASR | StepFun | Proprietary | #10 | — | 2.50 | 2.02 | 1.65 | 4.33 |
| Gemini 3.5 Transcribe | Proprietary | #11 | — | 2.60 | 2.01 | 1.67 | 4.71 | |
| Zoom Scribe V2 Pro | Zoom | Proprietary | #12 | — | 2.75 | 1.81 | 3.15 | 4.23 |
| Voxtral Small, Mistral | Mistral | Open | #13 | #2 | 2.77 | 2.27 | 1.75 | 4.78 |
| Nemotron 3.5 Transcribe | NVIDIA | Open | #14 | #3 | 2.81 | 1.78 | 1.99 | 5.69 |
| Gemini 3.1 Pro Preview (High) | Proprietary | #15 | — | 2.82 | 1.51 | 1.82 | 6.45 | |
| Gemini 2.5 Pro, Google | Proprietary | #16 | — | 2.86 | 2.25 | 2.68 | 4.25 | |
| Gemini 3 Flash (High), Google | Proprietary | #17 | — | 2.93 | 2.08 | 2.35 | 5.23 | |
| Soniox v6 Async | Soniox | Proprietary | #18 | — | 2.96 | 2.77 | 2.23 | 4.09 |
| Universal-3.5 Pro, AssemblyAI | AssemblyAI | Proprietary | #19 | — | 2.99 | 1.94 | 2.13 | 5.95 |
| Universal-3 Pro, AssemblyAI | AssemblyAI | Proprietary | #20 | — | 3.12 | 2.12 | 2.02 | 6.20 |
| Solaria-3, Gladia | Gladia | Proprietary | #21 | — | 3.23 | 2.18 | 2.91 | 5.63 |
| GPT Transcribe, OpenAI | OpenAI | Proprietary | #22 | — | 3.31 | 2.22 | 2.72 | 6.10 |
| Smallest AI Pulse | Smallest.ai | Proprietary | #23 | — | 3.33 | 2.97 | 2.55 | 4.82 |
| Resonant-1 | Reson8 | Proprietary | #24 | — | 3.39 | 2.09 | 2.61 | 6.78 |
| Smallest AI Pulse Pro | Smallest.ai | Proprietary | #25 | — | 3.41 | 2.26 | 3.22 | 5.92 |
| Gemini 3.1 Flash-Lite Preview (Minimal) | Proprietary | #26 | — | 3.43 | 2.53 | 2.16 | 6.50 | |
| Inkling (256K), Thinking Machines | Thinking Machines | Open | #27 | #4 | 3.47 | 3.09 | 2.26 | 5.44 |
| Qwen3.5 Omni Plus | Alibaba | Proprietary | #28 | — | 3.55 | 1.61 | 4.83 | 6.13 |
| Gemini 3.1 Pro Preview (Low) | Proprietary | #29 | — | 3.58 | 1.72 | 1.96 | 8.91 | |
| Voxtral Mini Transcribe 2, Mistral | Mistral | Proprietary | #30 | — | 3.59 | 2.96 | 2.16 | 6.28 |
| Soniox v5 Async | Soniox | Proprietary | #31 | — | 3.81 | 3.11 | 2.31 | 6.70 |
| Universal, AssemblyAI | AssemblyAI | Proprietary | #32 | — | 3.82 | 3.33 | 2.19 | 6.42 |
| Voxtral Mini Transcribe, Mistral | Mistral | Open | #33 | #5 | 3.84 | 3.12 | 2.39 | 6.73 |
| Inworld STT 1 | Inworld | Proprietary | #34 | — | 3.93 | 3.12 | 3.02 | 6.47 |
| GPT-4o Transcribe, OpenAI | OpenAI | Proprietary | #35 | — | 3.96 | 2.68 | 2.76 | 7.74 |
| Speechmatics Enhanced | Speechmatics | Proprietary | #36 | — | 4.05 | 3.38 | 2.86 | 6.59 |
| Whisper Large v2, OpenAI | OpenAI | Open | #37 | #6 | 4.06 | 3.19 | 2.81 | 7.04 |
| Solaria-1, Gladia | Gladia | Proprietary | #38 | — | 4.07 | 3.49 | 2.26 | 7.05 |
| Whisper Large v3, OpenAI | OpenAI | Open | #38 | #7 | 4.07 | 3.44 | 2.41 | 7.00 |
| Qwen3 ASR Flash, Alibaba | Alibaba | Open | #40 | #8 | 4.09 | 3.08 | 4.03 | 6.17 |
| Amazon Transcribe | Amazon | Proprietary | #41 | — | 4.12 | 3.04 | 4.35 | 6.04 |
| Modulate STT Batch English VFast | Modulate | Proprietary | #42 | — | 4.23 | 2.82 | 3.88 | 7.42 |
| Canary Qwen 2.5B, NVIDIA | NVIDIA | Open | #43 | #9 | 4.27 | 3.21 | 4.34 | 6.31 |
| Chirp 3, Google | Proprietary | #44 | — | 4.32 | 2.49 | 3.66 | 8.66 | |
| GPT-4o Mini Transcribe, OpenAI | OpenAI | Proprietary | #45 | — | 4.47 | 3.69 | 2.40 | 8.10 |
| Parakeet TDT 0.6B V3, NVIDIA | NVIDIA | Open | #46 | #10 | 4.55 | 4.29 | 3.31 | 6.30 |
| Cohere Transcribe | Cohere | Open | #47 | #11 | 4.57 | 3.09 | 5.36 | 6.73 |
| Whisper Large v3 Turbo, OpenAI | OpenAI | Open | #48 | #12 | 4.62 | 3.52 | 3.97 | 7.49 |
| StepAudio 2.5 ASR, StepFun | StepFun | Proprietary | #49 | — | 4.67 | 4.03 | 2.50 | 8.12 |
| Nova 2 Pro, Amazon | Amazon | Proprietary | #50 | — | 4.86 | 4.01 | 5.26 | 6.17 |
| Melia | Speechmatics | Proprietary | #51 | — | 4.87 | 4.19 | 3.92 | 7.19 |
| Speechmatics Standard | Speechmatics | Proprietary | #52 | — | 5.12 | 4.50 | 3.43 | 8.03 |
| Gemini 2.5 Flash, Google | Proprietary | #53 | — | 5.14 | 4.44 | 2.73 | 8.96 | |
| Gemini 2.5 Lite, Google | Proprietary | #54 | — | 5.17 | 3.55 | 2.92 | 10.64 | |
| Nova-3, Deepgram | Deepgram | Proprietary | #55 | — | 5.18 | 3.64 | 2.95 | 10.50 |
| Parakeet RNNT 1.1B, NVIDIA | NVIDIA | Open | #56 | #13 | 5.42 | 4.37 | 3.96 | 9.00 |
| Chirp 2, Google | Proprietary | #57 | — | 5.76 | 3.81 | 8.05 | 7.38 | |
| Rev AI | Rev AI | Proprietary | #58 | — | 5.92 | 5.85 | 3.57 | 8.40 |
| Raon Speech | Krafton | Open | #59 | #14 | 5.96 | 5.88 | 4.66 | 7.40 |
| Parakeet TDT 0.6B V2, NVIDIA | NVIDIA | Open | #60 | #15 | 6.43 | 5.84 | 4.60 | 9.42 |
| Gradium Speech-to-Text | Gradium | Proprietary | #61 | — | 6.79 | 7.30 | 3.25 | 9.32 |
| Gemma 4 12B, Google | Open | #62 | #16 | 8.77 | 8.03 | 5.34 | 13.69 | |
| Qwen3.5 Omni Flash | Alibaba | Proprietary | #63 | — | 13.52 | 22.08 | 4.12 | 5.80 |
| Cloud Speech-To-Text (Chirp), Google | Proprietary | #64 | — | 31.21 | 6.72 | 99.48 | 11.92 |
Every model-provider row
Each provider endpoint Artificial Analysis lists in its table, lowest index first. Speed factor is the median over the last seven days (higher is faster); price is AA’s estimate per 1,000 minutes of audio. Some models’ best rows run on endpoints this table does not list.
| Model | Provider | AA-WER Index | AA-AgentTalk | VoxPopuli | Earnings22 | Speed factor | Price per 1,000 min |
|---|---|---|---|---|---|---|---|
| Fun-Realtime-ASR-preview, Alibaba | Alibaba Cloud | 1.73 | 1.97 | 1.17 | 1.80 | 20.7× | $2.82 |
| Scribe v3, ElevenLabs | ElevenLabs | 1.95 | 1.47 | 1.52 | 3.34 | 74.3× | $3.33 |
| MAI-Transcribe-2 | Microsoft AI | 2.04 | 1.83 | 1.50 | 2.99 | 308.4× | $1.67 |
| Voxtral Small Transcribe 3 | Mistral | 2.09 | 1.65 | 1.51 | 3.55 | 62.9× | $4.00 |
| Scribe v2 | ElevenLabs | 2.18 | 1.51 | 1.74 | 3.95 | 67.6× | $3.67 |
| Voxtral Mini Transcribe 3 | Mistral | 2.22 | 2.02 | 1.45 | 3.40 | 76.3× | $3.50 |
| SOV-Transcribe-1, Salesforce | Salesforce | 2.28 | 1.79 | 1.50 | 4.06 | 60.1× | $0.00 |
| Grok Voice Transcribe 2.0 | SpaceXAI | 2.29 | 2.34 | 1.28 | 3.22 | 195.2× | $1.67 |
| MAI-Transcribe-1.5 | Microsoft AI | 2.38 | 1.96 | 1.62 | 3.96 | 77.3× | $6.00 |
| StepAudio 3 ASR | StepFun | 2.50 | 2.02 | 1.65 | 4.33 | 64.0× | $6.67 |
| Gemini 3.5 Transcribe | 2.60 | 2.01 | 1.67 | 4.71 | 94.2× | $5.00 | |
| Zoom Scribe V2 Pro | Zoom | 2.75 | 1.81 | 3.15 | 4.23 | 126.1× | $5.83 |
| Voxtral Small | Mistral | 2.77 | 2.27 | 1.75 | 4.78 | 61.5× | $4.00 |
| Gemini 3.1 Pro Preview (High) | 2.82 | 1.51 | 1.82 | 6.45 | 7.6× | $18.15 | |
| Gemini 2.5 Pro | 2.86 | 2.25 | 2.68 | 4.25 | 12.2× | $8.69 | |
| Gemini 3 Flash (High) | 2.93 | 2.08 | 2.35 | 5.23 | 24.2× | $4.17 | |
| Soniox v6 Async | Soniox | 2.96 | 2.77 | 2.23 | 4.09 | 81.1× | $1.66 |
| Universal-3.5 Pro | AssemblyAI | 2.99 | 1.94 | 2.13 | 5.95 | 100.5× | $3.50 |
| Universal-3 Pro | AssemblyAI | 3.12 | 2.12 | 2.02 | 6.20 | 103.5× | $3.50 |
| Solaria-3 | Gladia | 3.23 | 2.18 | 2.91 | 5.63 | 90.2× | $2.98 |
| GPT Transcribe | OpenAI | 3.31 | 2.22 | 2.72 | 6.10 | 42.7× | $4.50 |
| Smallest AI Pulse | Smallest.ai | 3.33 | 2.97 | 2.55 | 4.82 | 232.6× | $3.00 |
| Resonant-1 | Reson8 | 3.39 | 2.09 | 2.61 | 6.78 | 355.9× | $22.33 |
| Smallest AI Pulse Pro | Smallest.ai | 3.41 | 2.26 | 3.22 | 5.92 | 293.8× | $4.00 |
| Nemotron 3.5 Transcribe | DeepInfra | 3.42 | 2.43 | 2.39 | 6.42 | 63.6× | $2.00 |
| Gemini 3.1 Flash-Lite Preview (Minimal) | 3.43 | 2.53 | 2.16 | 6.50 | 88.2× | $1.10 | |
| Qwen3.5 Omni Plus | Alibaba Cloud | 3.55 | 1.61 | 4.83 | 6.13 | 153.1× | $0.00 |
| Gemini 3.1 Pro Preview (Low) | 3.58 | 1.72 | 1.96 | 8.91 | 7.5× | $7.72 | |
| Voxtral Mini Transcribe 2 | Mistral | 3.59 | 2.96 | 2.16 | 6.28 | 86.0× | $3.00 |
| Soniox v5 Async | Soniox | 3.81 | 3.11 | 2.31 | 6.70 | 48.3× | $1.66 |
| Universal | AssemblyAI | 3.82 | 3.33 | 2.19 | 6.42 | 106.6× | $2.50 |
| Voxtral Mini, Deepinfra | DeepInfra | 3.84 | 3.12 | 2.39 | 6.73 | 50.7× | $1.00 |
| Inworld STT 1 | Inworld | 3.93 | 3.12 | 3.02 | 6.47 | 73.8× | $2.50 |
| GPT-4o Transcribe | OpenAI | 3.96 | 2.68 | 2.76 | 7.74 | 37.5× | $6.00 |
| Enhanced | Speechmatics | 4.05 | 3.38 | 2.86 | 6.59 | 59.8× | $12.50 |
| Large v2 | OpenAI | 4.06 | 3.19 | 2.81 | 7.04 | 27.0× | $6.00 |
| Solaria-1 | Gladia | 4.07 | 3.49 | 2.26 | 7.05 | 91.0× | $5.02 |
| Whisper (L, v3), fal.ai | fal.ai | 4.07 | 3.44 | 2.41 | 7.00 | 99.4× | $1.15 |
| Amazon Transcribe | Amazon Bedrock | 4.12 | 3.04 | 4.35 | 6.04 | 39.6× | $6.00 |
| Modulate STT Batch English VFast | Modulate | 4.23 | 2.82 | 3.88 | 7.42 | 45.3× | $0.42 |
| Canary Qwen 2.5B, Replicate | Replicate | 4.27 | 3.21 | 4.34 | 6.31 | 7.9× | $0.74 |
| Chirp 3 | 4.32 | 2.49 | 3.66 | 8.66 | 28.4× | $16.00 | |
| GPT-4o Mini Transcribe | OpenAI | 4.47 | 3.69 | 2.40 | 8.10 | 47.3× | $3.00 |
| Whisper Large v3, together.ai | Together AI | 4.53 | 3.34 | 3.58 | 7.85 | 247.9× | $1.50 |
| Parakeet TDT 0.6B V3, Togetherai | Together AI | 4.55 | 4.29 | 3.31 | 6.30 | 664.5× | $1.50 |
| Whisper (L, v3, Turbo), Groq | Groq | 4.62 | 3.52 | 3.97 | 7.49 | 334.9× | $0.67 |
| StepAudio 2.5 ASR | StepFun | 4.67 | 4.03 | 2.50 | 8.12 | 84.9× | $0.37 |
| Wizper (L, v3), fal.ai | fal.ai | 4.69 | 3.05 | 5.27 | 7.38 | 180.1× | $0.50 |
| Nova-3, Telnyx | Telnyx | 4.82 | 3.26 | 2.47 | 10.28 | 325.7× | $5.00 |
| Nova 2 Pro | Amazon Bedrock | 4.86 | 4.01 | 5.26 | 6.17 | 23.8× | $3.10 |
| Melia | Speechmatics | 4.87 | 4.19 | 3.92 | 7.19 | 170.5× | $4.00 |
| Nova-2, Telnyx | Telnyx | 5.09 | 3.52 | 2.44 | 10.89 | 324.6× | $5.00 |
| Standard | Speechmatics | 5.12 | 4.50 | 3.43 | 8.03 | 66.2× | $7.50 |
| Gemini 2.5 Flash | 5.14 | 4.44 | 2.73 | 8.96 | 72.0× | $2.51 | |
| Gemini 2.5 Flash Lite | 5.17 | 3.55 | 2.92 | 10.64 | 82.4× | $0.67 | |
| Nova-3 | Deepgram | 5.18 | 3.64 | 2.95 | 10.50 | 510.8× | $4.30 |
| Parakeet RNNT 1.1B, Replicate | Replicate | 5.42 | 4.37 | 3.96 | 9.00 | 6.3× | $1.91 |
| Whisper Large v3 Turbo, Telnyx | Telnyx | 5.53 | 3.45 | 4.58 | 10.64 | 36.7× | $15.00 |
| Incredibly Fast, Replicate | Replicate | 5.65 | 4.44 | 3.50 | 10.20 | 92.3× | $1.49 |
| Rev AI | Rev AI | 5.92 | 5.85 | 3.57 | 8.40 | 11.6× | $3.33 |
| Parakeet TDT 0.6B V2 | NVIDIA | 6.43 | 5.84 | 4.60 | 9.42 | 90.6× | $0.00 |
| Gradium Speech-to-Text | Gradium | 6.79 | 7.30 | 3.25 | 9.32 | 1.9× | $13.00 |
| Whisper (L, v3), Replicate | Replicate | 10.06 | 3.79 | 3.48 | 29.18 | 3.2× | $4.23 |
| Qwen3.5 Omni Flash | Alibaba Cloud | 13.52 | 22.08 | 4.12 | 5.80 | 77.8× | $0.00 |
| Chirp | 31.21 | 6.72 | 99.48 | 11.92 | 14.4× | $16.00 |
Interpretation limit
Artificial Analysis runs the board and chooses the models, providers and settings. The sets are English only, so the index says nothing about other languages; AA-AgentTalk is proprietary and held out, so its clips cannot be inspected. Speed includes network latency and changes day to day; prices are AA’s estimate for 1,000 minutes and a $0.00 price marks a model with no hosted price, not a free one. The same weights on different providers can score differently. Snapshot retrieved November 12, 2026. Scores are credited to Artificial Analysis and kept out of BenchLM Data under its terms of use. These results do not enter text-model or voice-agent rankings.