When2Call (Decision Index)
We show this table for reference; we do not rank on it.
d1-ultra leads Artificial Analysis' October 15, 2026 snapshot of When2Call (Decision Index) at 92.6. NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.
Evaluation results published by Artificial Analysis.
Accuracy on When2Call (Decision Index) — October 15, 2026 snapshot
We mirror the published accuracy view for When2Call (Decision Index). The public snapshot contains 13 decision models. d1-ultra has the highest published score at 92.6%, but the available coverage and evaluation setups do not establish a market leader. We do not use these results to rank models overall.
d1-ultra
Liquid AI
$31.8 per 1M tasks · 93 ms per task
Celeris-1 Decision
Celeris
$31.57 per 1M tasks · 63 ms per task
GLiDE
Fastino
$119.02 per 1M tasks · 217 ms per task
13 decision modelsDecision ModelsCurrentDisplay onlyUpdated October 15, 2026 snapshot
| Rank | Model | Index | Knowledge accuracy | Non-hallucination | HLE-MultiChoice | When2Call | WildGuardTest | Cost per 1M tasks | Time per task |
|---|---|---|---|---|---|---|---|---|---|
| 1 | d1-ultraLiquid AI | 65.2 | 31.6 | 84.5 | 15.0 | 92.6 | 95.4 | $31.80 | 93 ms |
| 2 | Celeris-1 DecisionCeleris | 52.3 | 37.8 | 0.0 | 13.3 | 83.7 | 93.3 | $31.57 | 63 ms |
| 3 | GLiDEFastino | 53.9 | 36.4 | 16.4 | 10.5 | 83.7 | 95.1 | $119 | 217 ms |
| 4 | Microsoft-Decision-1Microsoft | 53.8 | 28.2 | 39.8 | 10.9 | 81.9 | 88.2 | $30.71 | 207 ms |
| 5 | Jev 1.14TypeSafe | 57.1 | 33.8 | 37.2 | 18.9 | 80.8 | 93.1 | $41.53 | 78 ms |
| 6 | Jev 1.13TypeSafe | 57.5 | 34.6 | 35.2 | 20.9 | 80.7 | 93.6 | $41.53 | 127 ms |
| 7 | Mercury DecideInception | 52.7 | 35.0 | 32.0 | 10.1 | 79.3 | 88.0 | $15.20 | 500 ms |
| 8 | d1Liquid AI | 50.3 | 41.0 | 1.7 | 11.5 | 77.4 | 90.9 | $30.29 | 299 ms |
| 9 | Decider v1.1 27BPerplexity | 53.1 | 39.8 | 13.3 | 15.4 | 76.5 | 93.8 | $16.53 | 170 ms |
| 10 | GPT-6 Luna (Decisions)OpenAI | 48.5 | 40.6 | 10.1 | 11.7 | 74.1 | 82.8 | $78.87 | 168 ms |
| 11 | ClefCloudflare | 48.6 | 39.0 | 4.3 | 11.1 | 71.0 | 90.7 | $199 | 309 ms |
| 12 | Clef-FlashCloudflare | 45.8 | 32.6 | 2.7 | 10.3 | 64.4 | 90.9 | $31.57 | 298 ms |
| 13 | Clef-OmniCloudflare | 44.3 | 36.0 | 1.6 | 9.0 | 62.1 | 87.2 | $123 | 172 ms |
Artificial Analysis runs every evaluation. The index weights AA-Omniscience-Decision accuracy and non-hallucination rate at 12.5% each and HLE-MultiChoice, When2Call and WildGuardTest at 25% each. Values are rounded to one decimal. Cost is the average token-priced cost of one pass through the index tasks, multiplied by one million; time is the median latency per task. The leaderboard also runs language models as structured-output baselines, which are not mirrored here.
How we show the Artificial Analysis Decision Index
Artificial Analysis publishes the Decision Index v1.0 as a live leaderboard of decision models, models that answer a typed question in one call with a probability for each option. NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.
Artificial Analysis runs every evaluation itself, so these are independent measurements and not provider reports. We mirror its decision-model rows and keep the language-model baselines it runs for comparison on its own page.
Scores are rounded to one decimal; Artificial Analysis charts print integers. The index is a different protocol from Decision Index 0.2.1 and 0.3, so values do not carry across.
Snapshot
The published When2Call (Decision Index) snapshot places d1-ultra first at 92.6%. The third row is 8.9 points behind. The broader top-10 range is 18.5 points, so the table still separates the published systems.
13 decision models are shown for When2Call (Decision Index). The benchmark falls in the Decision Models category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. When2Call (Decision Index) is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About When2Call (Decision Index)
Year
2026
Format
Accuracy
Difficulty
Tool-use decisions
Artificial Analysis runs every evaluation itself and poses each question as one typed decision with a probability for each option. We mirror its decision-model rows, round values to one decimal, and keep the language-model baselines it also runs on its own page. These results are separate from Decision Index 0.2.1, Decision Index 0.3, JevBench, and general model rankings.
Freshness and provenance
Version
Artificial Analysis Decision Index v1.0
Refresh cadence
Published leaderboard snapshots
Staleness state
Current
Question availability
HLE-MultiChoice, When2Call and WildGuardTest use public datasets; AA-Omniscience-Decision questions are private
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does When2Call (Decision Index) measure?
When2Call is one component of the Artificial Analysis Decision Index v1.0, run by Artificial Analysis. It measures accuracy on NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.
Which model leads the published When2Call (Decision Index) snapshot?
d1-ultra currently leads the published When2Call (Decision Index) snapshot with 92.6% accuracy. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on When2Call (Decision Index)?
The October 15, 2026 snapshot snapshot contains 13 decision models scored by Artificial Analysis.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.