# When2Call (Decision Index)

> d1-ultra leads Artificial Analysis' October 15, 2026 snapshot of When2Call (Decision Index) at 92.6. NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.

Canonical page: https://benchlm.ai/benchmarks/aa-decision-when2call

- Category: [Decision Models](/decision-models)
- Last updated: October 15, 2026 snapshot

Evaluation results published by [Artificial Analysis](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index). undefined

Artificial Analysis publishes the Decision Index v1.0 as a live leaderboard of decision models, models that answer a typed question in one call with a probability for each option. NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.

Artificial Analysis runs every evaluation itself, so these are independent measurements and not provider reports. We mirror its decision-model rows and keep the language-model baselines it runs for comparison on its own page.

Scores are rounded to one decimal; Artificial Analysis charts print integers. The index is a different protocol from Decision Index 0.2.1 and 0.3, so values do not carry across.

- [Artificial Analysis Decision Index](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index)
- [Launch article and methodology](https://artificialanalysis.ai/articles/artificial-analysis-decision-index)
- [Decision Models](/decision-models)
- [d1-ultra profile and pricing](/models/d1-ultra)

## About When2Call (Decision Index)

- Year: 2026
- Format: Accuracy
- Difficulty: Tool-use decisions
- Paper: [Artificial Analysis Decision Index v1.0](https://artificialanalysis.ai/articles/artificial-analysis-decision-index)

Artificial Analysis runs every evaluation itself and poses each question as one typed decision with a probability for each option. We mirror its decision-model rows, round values to one decimal, and keep the language-model baselines it also runs on its own page. These results are separate from Decision Index 0.2.1, Decision Index 0.3, JevBench, and general model rankings.

When2Call (Decision Index) is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (13 decision models)

| Rank | Model | Index | Knowledge accuracy | Non-hallucination | HLE-MultiChoice | When2Call | WildGuardTest | Cost per 1M tasks | Time per task |
|---|---|---|---|---|---|---|---|---|---|
| 1 | [d1-ultra](/models/d1-ultra) | 65.2 | 31.6 | 84.5 | 15.0 | 92.6 | 95.4 | $31.8 | 93 ms |
| 2 | [Celeris-1 Decision](/models/celeris-1-decision) | 52.3 | 37.8 | 0.0 | 13.3 | 83.7 | 93.3 | $31.57 | 63 ms |
| 3 | [GLiDE](/models/fastino-glide) | 53.9 | 36.4 | 16.4 | 10.5 | 83.7 | 95.1 | $119.02 | 217 ms |
| 4 | [Microsoft-Decision-1](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 53.8 | 28.2 | 39.8 | 10.9 | 81.9 | 88.2 | $30.71 | 207 ms |
| 5 | [Jev 1.14](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 57.1 | 33.8 | 37.2 | 18.9 | 80.8 | 93.1 | $41.53 | 78 ms |
| 6 | [Jev 1.13](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 57.5 | 34.6 | 35.2 | 20.9 | 80.7 | 93.6 | $41.53 | 127 ms |
| 7 | [Mercury Decide](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 52.7 | 35.0 | 32.0 | 10.1 | 79.3 | 88.0 | $15.2 | 500 ms |
| 8 | [d1](/models/liquid-d1) | 50.3 | 41.0 | 1.7 | 11.5 | 77.4 | 90.9 | $30.29 | 299 ms |
| 9 | [Decider v1.1 27B](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 53.1 | 39.8 | 13.3 | 15.4 | 76.5 | 93.8 | $16.53 | 170 ms |
| 10 | [GPT-6 Luna (Decisions)](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 48.5 | 40.6 | 10.1 | 11.7 | 74.1 | 82.8 | $78.87 | 168 ms |
| 11 | [Clef](/models/cloudflare-clef) | 48.6 | 39.0 | 4.3 | 11.1 | 71.0 | 90.7 | $199.41 | 309 ms |
| 12 | [Clef-Flash](/models/cloudflare-clef-flash) | 45.8 | 32.6 | 2.7 | 10.3 | 64.4 | 90.9 | $31.57 | 298 ms |
| 13 | [Clef-Omni](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index) | 44.3 | 36.0 | 1.6 | 9.0 | 62.1 | 87.2 | $123.08 | 172 ms |

Artificial Analysis runs every evaluation. The index weights AA-Omniscience-Decision accuracy and non-hallucination rate at 12.5% each and HLE-MultiChoice, When2Call and WildGuardTest at 25% each. Values are rounded to one decimal; language-model baselines on the source leaderboard are not mirrored.
## FAQ

### What does When2Call (Decision Index) measure?

When2Call is one component of the Artificial Analysis Decision Index v1.0, run by Artificial Analysis. It measures accuracy on NVIDIA's tool-calling evaluation: given a conversation and tools, choose to answer directly, call a tool, ask for more information, or decline.

### Which model leads the published When2Call (Decision Index) snapshot?

d1-ultra currently leads the published When2Call (Decision Index) snapshot with a score of 92.6%.

### How many models are evaluated on When2Call (Decision Index)?

The October 15, 2026 snapshot contains 13 decision models scored by Artificial Analysis.

### Does When2Call (Decision Index) affect BenchLM's overall score?

Not directly. When2Call (Decision Index) is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.