Live source snapshot

VoiceBench

A spoken question-answering suite spanning open-ended, knowledge, reasoning, instruction-following, and safety tasks.

Source snapshots refreshed 2026-08-04

Five measurement lanesVOICE / S2S

Reasoning, task completion, conversation dynamics, experience, and latency stay separate.

Back to directory

Benchmark profile

Scope: 11 dataset subsets; human and synthesized speech
Primary metric: Benchmark-defined overall score
Owner: VoiceBench authors
Available evidence: Machine-readable public leaderboard

What it measures

Spoken reasoning

Does the system understand and answer spoken requests?

Available results

Protocol details

38 models

Model	Architecture
NVIDIA Nemotron 3 Nano Omni 30B A3B #1 · open	Vision + audio	89.39	4.75	4.57	4.58	82.30	88.66
Ultravox-GLM-4P7 #2 · open	Audio-LLM	88.86	4.87	4.30	4.55	83.82	76.26
Ultravox-GLM-4P7 (thinking) #3 · open	Audio-LLM	88.79	4.69	4.05	4.21	87.15	80.99
Whisper-v3-large + GPT-4o #4 · closed	Cascaded	87.80	4.80	4.47	4.62	81.69	76.51
Ultravox-GLM-4P6 #5 · open	Audio-LLM	87.05	4.93	4.42	4.57	80.40	75.59
GPT-4o-Audio #6 · closed	omni_hd	86.75	4.78	4.49	4.58	80.25	76.02
GPT-4o-mini-Audio #7 · closed	omni_hd	82.84	4.75	4.24	4.40	72.90	72.90
Ultravox-v0.6-LLaMA-3.3-70B #8 · open	Audio-LLM	81.81	4.69	4.26	4.38	69.20	61.50
LFG-1 #9 · open	Audio-LLM	79.63	4.60	3.71	4.01	75.60	74.85
Parakeet-TDT-0.6b-V2 + Qwen3-8B #10 · open	Cascaded	79.23	4.68	4.46	4.35	59.10	78.99
Whisper-v3-large + LLaMA-3.1-8B #11 · open	Cascaded	77.48	4.53	4.04	4.16	62.43	69.53
Kimi-Audio #12 · open	omni_hd	76.91	4.46	3.97	4.20	62.17	61.10
Whisper-v3-turbo + LLaMA-3.1-8B #13 · open	Cascaded	76.09	4.55	4.02	4.12	62.04	71.12
Ultravox-v0.5-LLaMA-3.1-8B #14 · open	Audio-LLM	74.86	4.59	4.11	4.28	54.16	66.51
Ultravox-v0.4.1-LLaMA-3.1-8B #15 · open	Audio-LLM	72.09	4.55	3.90	4.12	47.17	66.88
Baichuan-Omni-1.5 #16 · open	omni_hd	71.32	4.50	4.05	4.06	57.25	54.54
MiniCPM-o #17 · open	omni_hd	71.23	4.42	4.15	3.94	54.78	49.25
Whisper-v3-turbo + LLaMA-3.2-3B #18 · open	Cascaded	71.02	4.45	3.82	4.04	51.37	69.71
Baichuan-Audio #19 · open	omni_hd	69.27	4.41	4.08	3.92	53.19	50.31
MERaLiON #20 · open	Audio-LLM	65.04	4.50	3.77	4.12	34.95	62.93
VITA-1.5 #21 · open	omni_hd	64.53	4.21	3.66	3.48	52.15	38.14
Phi-4-multimodal #22 · open	Vision + audio	64.32	3.81	3.82	3.56	42.19	45.35
Ola #23 · open	omni_hd	59.42	4.12	2.97	3.19	45.97	39.57
Lyra-Base #24 · open	omni_hd	59.00	3.85	3.50	3.42	49.74	36.28
Ultravox-v0.5-LLaMA-3.2-1B #26 · open	Audio-LLM	57.46	4.04	3.57	3.47	30.03	45.56
DiVA #27 · open	Audio-LLM	57.39	3.67	3.54	3.74	25.76	39.15
GLM-4-Voice #28 · open	omni_hd	56.48	3.97	3.42	3.18	39.75	25.92
Qwen2-Audio #29 · open	Audio-LLM	55.80	3.74	3.43	3.01	35.72	26.33
Step-Audio #31 · open	omni_hd	50.84	4.13	3.09	2.93	28.33	27.96
Megrez-3B-Omni #32 · open	omni_hd	46.76	3.50	2.95	2.34	27.03	25.71
Ichigo #33 · open	omni_hd	45.57	3.79	3.17	2.83	25.63	21.59
Lyra-Mini #34 · open	omni_hd	45.26	2.99	2.69	2.58	31.42	20.91
Mair-hub-0.5B-Omni #35 · open	omni_hd	44.59	3.06	2.87	2.48	25.60	14.85
LLaMA-Omni #36 · open	omni_hd	41.12	3.70	3.46	2.92	25.93	14.87
VITA-1.0 #37 · open	omni_hd	36.43	3.38	2.15	1.87	25.70	22.82
SLAM-Omni #38 · open	omni_hd	35.30	1.90	1.79	1.60	26.06	13.38
Mini-Omni2 #39 · open	omni_hd	33.49	2.32	2.18	1.79	24.27	11.56
Mini-Omni #40 · open	omni_hd	30.42	1.95	2.02	1.61	24.69	13.58

Interpretation limit

Open-ended answers use an automatic model judge. Its overall score combines heterogeneous task scales using the benchmark owner’s method.

Primary sources

Paper Code Dataset Owner page