Live source snapshot
VoiceBench
A spoken question-answering suite spanning open-ended, knowledge, reasoning, instruction-following, and safety tasks.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 11 dataset subsets; human and synthesized speech
- Primary metric
- Benchmark-defined overall score
- Owner
- VoiceBench authors
- Available evidence
- Machine-readable public leaderboard
What it measures
Spoken reasoning
Does the system understand and answer spoken requests?
Available results
43 models
| Model | Architecture | ||||||
|---|---|---|---|---|---|---|---|
BR-Voice-Reasoner #1 · open | Vision + audio | 90.46 | 4.87 | 4.71 | 4.66 | 85.70 | 83.20 |
LFG-3 #2 · open | Audio-LLM | 89.88 | 4.73 | 4.40 | 4.45 | 85.52 | 88.54 |
NVIDIA Nemotron 3 Nano Omni 30B A3B #3 · open | Vision + audio | 89.39 | 4.75 | 4.57 | 4.58 | 82.30 | 88.66 |
Ultravox-GLM-4P7 #4 · open | Audio-LLM | 88.86 | 4.87 | 4.30 | 4.55 | 83.82 | 76.26 |
Qwen3-Omni-30B-A3B-Thinking #5 · open | Vision + audio | 88.80 | 4.82 | 4.53 | 4.53 | 83.00 | 80.60 |
Ultravox-GLM-4P7 (thinking) #6 · open | Audio-LLM | 88.79 | 4.69 | 4.05 | 4.21 | 87.15 | 80.99 |
Whisper-v3-large + GPT-4o #7 · closed | Cascaded | 87.80 | 4.80 | 4.47 | 4.62 | 81.69 | 76.51 |
Ultravox-GLM-4P6 #8 · open | Audio-LLM | 87.05 | 4.93 | 4.42 | 4.57 | 80.40 | 75.59 |
LFG-2 #9 · open | Audio-LLM | 86.98 | 4.67 | 4.26 | 4.23 | 85.20 | 84.54 |
GPT-4o-Audio #10 · closed | omni_hd | 86.75 | 4.78 | 4.49 | 4.58 | 80.25 | 76.02 |
Qwen3-Omni-30B-A3B-Instruct #11 · open | omni_hd | 85.50 | 4.74 | 4.54 | 4.58 | 68.10 | 77.80 |
GPT-4o-mini-Audio #12 · closed | omni_hd | 82.84 | 4.75 | 4.24 | 4.40 | 72.90 | 72.90 |
Ultravox-v0.6-LLaMA-3.3-70B #13 · open | Audio-LLM | 81.81 | 4.69 | 4.26 | 4.38 | 69.20 | 61.50 |
LFG-1 #14 · open | Audio-LLM | 79.63 | 4.60 | 3.71 | 4.01 | 75.60 | 74.85 |
Parakeet-TDT-0.6b-V2 + Qwen3-8B #15 · open | Cascaded | 79.23 | 4.68 | 4.46 | 4.35 | 59.10 | 78.99 |
Whisper-v3-large + LLaMA-3.1-8B #16 · open | Cascaded | 77.48 | 4.53 | 4.04 | 4.16 | 62.43 | 69.53 |
Kimi-Audio #17 · open | omni_hd | 76.91 | 4.46 | 3.97 | 4.20 | 62.17 | 61.10 |
Whisper-v3-turbo + LLaMA-3.1-8B #18 · open | Cascaded | 76.09 | 4.55 | 4.02 | 4.12 | 62.04 | 71.12 |
Ultravox-v0.5-LLaMA-3.1-8B #19 · open | Audio-LLM | 74.86 | 4.59 | 4.11 | 4.28 | 54.16 | 66.51 |
Ultravox-v0.4.1-LLaMA-3.1-8B #20 · open | Audio-LLM | 72.09 | 4.55 | 3.90 | 4.12 | 47.17 | 66.88 |
Baichuan-Omni-1.5 #21 · open | omni_hd | 71.32 | 4.50 | 4.05 | 4.06 | 57.25 | 54.54 |
MiniCPM-o #22 · open | omni_hd | 71.23 | 4.42 | 4.15 | 3.94 | 54.78 | 49.25 |
Whisper-v3-turbo + LLaMA-3.2-3B #23 · open | Cascaded | 71.02 | 4.45 | 3.82 | 4.04 | 51.37 | 69.71 |
Baichuan-Audio #24 · open | omni_hd | 69.27 | 4.41 | 4.08 | 3.92 | 53.19 | 50.31 |
MERaLiON #25 · open | Audio-LLM | 65.04 | 4.50 | 3.77 | 4.12 | 34.95 | 62.93 |
VITA-1.5 #26 · open | omni_hd | 64.53 | 4.21 | 3.66 | 3.48 | 52.15 | 38.14 |
Phi-4-multimodal #27 · open | Vision + audio | 64.32 | 3.81 | 3.82 | 3.56 | 42.19 | 45.35 |
Ola #28 · open | omni_hd | 59.42 | 4.12 | 2.97 | 3.19 | 45.97 | 39.57 |
Lyra-Base #29 · open | omni_hd | 59.00 | 3.85 | 3.50 | 3.42 | 49.74 | 36.28 |
Ultravox-v0.5-LLaMA-3.2-1B #31 · open | Audio-LLM | 57.46 | 4.04 | 3.57 | 3.47 | 30.03 | 45.56 |
DiVA #32 · open | Audio-LLM | 57.39 | 3.67 | 3.54 | 3.74 | 25.76 | 39.15 |
GLM-4-Voice #33 · open | omni_hd | 56.48 | 3.97 | 3.42 | 3.18 | 39.75 | 25.92 |
Qwen2-Audio #34 · open | Audio-LLM | 55.80 | 3.74 | 3.43 | 3.01 | 35.72 | 26.33 |
Step-Audio #36 · open | omni_hd | 50.84 | 4.13 | 3.09 | 2.93 | 28.33 | 27.96 |
Megrez-3B-Omni #37 · open | omni_hd | 46.76 | 3.50 | 2.95 | 2.34 | 27.03 | 25.71 |
Ichigo #38 · open | omni_hd | 45.57 | 3.79 | 3.17 | 2.83 | 25.63 | 21.59 |
Lyra-Mini #39 · open | omni_hd | 45.26 | 2.99 | 2.69 | 2.58 | 31.42 | 20.91 |
Mair-hub-0.5B-Omni #40 · open | omni_hd | 44.59 | 3.06 | 2.87 | 2.48 | 25.60 | 14.85 |
LLaMA-Omni #41 · open | omni_hd | 41.12 | 3.70 | 3.46 | 2.92 | 25.93 | 14.87 |
VITA-1.0 #42 · open | omni_hd | 36.43 | 3.38 | 2.15 | 1.87 | 25.70 | 22.82 |
SLAM-Omni #43 · open | omni_hd | 35.30 | 1.90 | 1.79 | 1.60 | 26.06 | 13.38 |
Mini-Omni2 #44 · open | omni_hd | 33.49 | 2.32 | 2.18 | 1.79 | 24.27 | 11.56 |
Mini-Omni #45 · open | omni_hd | 30.42 | 1.95 | 2.02 | 1.61 | 24.69 | 13.58 |
Interpretation limit
Open-ended answers use an automatic model judge. Its overall score combines heterogeneous task scales using the benchmark owner’s method.