Live source snapshot
EVA-Bench
Separates enterprise task accuracy from conversational experience across realistic service scenarios.
Source snapshots refreshed
Five measurement lanesVOICE / S2S
Reasoning, task completion, conversation dynamics, experience, and latency stay separate.
Benchmark profile
- Scope
- 213 scenarios; 3 enterprise domains; 12 systems
- Primary metric
- EVA-A accuracy and EVA-X experience
- Owner
- ServiceNow Research
- Available evidence
- Machine-readable benchmark-owner leaderboard, code, and public dataset
What it measures
Task completion
Can it follow policy, use tools, and finish a workflow?
Conversation dynamics
Can it handle turns, interruptions, ambiguity, and state?
Voice experience
Is the exchange natural, robust, and responsive?
Available results
| Rank | System | EVA-A pass | EVA-X pass | Response speed | Architecture |
|---|---|---|---|---|---|
| 1 | Cartesia Managed Agents - GPT 5.5 | 87.4% | 90.7% | 1.94s | cascade |
| 2 | Grok Voice 3.0 | 73.7% | 71.2% | 1.96s | s2s |
| 3 | Scribe v2.2 Realtime + GPT-5.4 + Eleven Flash v2 (ElevenAgents) | 67.2% | 13.0% | 3.56s | cascade |
| 4 | GPT Realtime 2.1 | 59.0% | 61.6% | 1.64s | s2s |
| 5 | Grok Voice 1.0 | 58.8% | 56.9% | 1.54s | s2s |
| 6 | Gemini 3.6 Flash + Gemini 3.1 Flash TTS Preview | 56.2% | 0.0% | 5.81s | 2-part |
| 7 | Gemini 3.8 Live Extended Thinking (High) | 54.8% | 73.7% | 1.72s | s2s |
| 8 | Universal 3.5 Pro + GPT-5.4 + Sonic 3.5 | 53.1% | 24.8% | 2.43s | cascade |
| 9 | Grok Voice 2.0 | 51.7% | 64.1% | 1.70s | s2s |
| 10 | GPT Realtime 2 | 50.4% | 39.0% | — | s2s |
| 11 | Gemini 3.8 Live Extended Thinking (Minimal) | 50.1% | 79.3% | 1.64s | s2s |
| 12 | Scribe v2.2 Realtime + Gemini 3 Flash + TTS Conversational v3 (ElevenAgents) | 49.0% | 2.4% | 4.16s | cascade |
| 13 | GPT Realtime 1.5 | 46.7% | 56.6% | — | s2s |
| 14 | Gemini 3.8 Live | 45.9% | 88.7% | 1.98s | s2s |
| 15 | Flux General Multi + GPT-5.4 + Sonic 3.5 | 43.1% | 17.6% | 2.93s | cascade |
| 16 | Scribe v2.2 Realtime + Claude Haiku 4.5 + Eleven Flash v2 (ElevenAgents) | 41.0% | 81.7% | 1.94s | cascade |
| 17 | GPT Realtime | 40.7% | 36.5% | — | s2s |
| 18 | Nova 3 + GPT-5.4-mini + Aura 2 | 39.5% | 9.6% | 3.42s | cascade |
| 19 | Ink 2 + Haiku 4.5 + Sonic 3.5 | 34.8% | 6.6% | 3.33s | cascade |
| 20 | Parakeet 1.1B CTC en-US + Gemma-4-26B + Kokoro | 33.1% | 39.5% | 2.58s | cascade |
| 21 | Gemini 3.1 Flash Live | 29.2% | 58.9% | — | s2s |
| 22 | Ultravox | 27.0% | 2.9% | — | 2-part |
| 23 | GPT Realtime 2.1 Mini | 21.6% | 29.2% | 2.14s | s2s |
| 24 | GPT Realtime Mini | 16.3% | 40.6% | — | s2s |
Interpretation limit
Accuracy and experience are deliberately separate axes; neither should be collapsed into the other.