ClaimBench v1 (Vals AI) (ClaimBench)
We show this table for reference; we do not rank on it.
Vals AI finance benchmark that asks whether a 10-K excerpt supports a claim, contradicts it, or says nothing about it. It compares small decision models with frontier language models.
Evaluation results published by Vals AI. Published leaderboard aggregates and evaluation settings only. Dataset contents are not included.
Accuracy on ClaimBench — October 23, 2026
We mirror the published accuracy view for ClaimBench. d1-ultra leads the public snapshot at 100.00%, followed by Claude Sonnet 5.5 (100.00%) and Claude Haiku 5.5 (100.00%). We do not use these results to rank models overall.
d1-ultra
Liquid AI
Claude Sonnet 5.5
Anthropic
Claude Haiku 5.5
Anthropic
25 modelsDecision ModelsCurrentDisplay onlyUpdated October 23, 2026
Accuracy table (25 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | d1-ultraLiquid AI | 100.00% | Not reported | Closed |
| 2 | Claude Sonnet 5.5Anthropic | 100.00% | Not reported | Closed |
| 3 | Claude Haiku 5.5Anthropic | 100.00% | Not reported | Closed |
| 4 | Claude Opus 5.5Anthropic | 100.00% | Not reported | Closed |
| 5 | GPT-6.1 SolOpenAImax reasoning | 100.00% | Not reported | Closed |
| 6 | Perplexity Decider v1.2 27BPerplexity | 99.69% | Not reported | Open |
| 7 | Drex MaxNace.AI via OpenRouter | 99.18% | Not reported | Unknown |
| 8 | GPT-6 LunaOpenAImax reasoning | 99.18% | Not reported | Closed |
| 9 | Mercury DecideInception | 99.08% | Not reported | Closed |
| 10 | Perplexity Decider v1.1 (27B)Perplexity | 98.78% | Not reported | Open |
| 11 | d1Liquid AI | 98.67% | Not reported | Closed |
| 12 | Microsoft-Decision-1Microsoft | 98.21% | Not reported | Closed |
| 13 | Jev 1.14TypeSafe AI | 98.06% | Not reported | Closed |
| 14 | Drex v1.5Nace.AI | 98.01% | Not reported | Open |
| 15 | Jev 1.13.0TypeSafe AI | 97.65% | Not reported | Closed |
| 16 | Drex v2.0Nace.AI | 97.30% | Not reported | Open |
| 17 | GPT-6 Luna DecisionsOpenAI | 96.89% | Not reported | Unknown |
| 18 | Perplexity Decider Flash v1.2 4BPerplexity | 96.53% | Not reported | Open |
| 19 | ClefCloudflare | 95.10% | Not reported | Open |
| 20 | Solar DecideUpstage via OpenRouter | 94.39% | Not reported | Unknown |
| 21 | Tev1-4B-experimentaltogethercomputer | 89.80% | Not reported | Open |
| 22 | Clef-OmniCloudflare | 87.70% | Not reported | Open |
| 23 | ScaleDown ClassifyScaleDown | 85.00% | Not reported | Closed |
| 24 | Clef-FlashCloudflare | 84.85% | Not reported | Open |
| 25 | Solar Decide FlashUpstage via OpenRouter | 82.45% | Not reported | Unknown |
How to read this leaderboard
Operator receipt: 25 sourced rows are currently displayable on this page; the leading published row is d1-ultra at 100.00%.
Honest limit: Four language models reach 100% overall, so the board separates decision models more than frontier models. Three of the five categories use edited or generated claims, which can carry stylistic quirks. The class mix is uneven, so always guessing one label scores about 40%.
How ClaimBench is shown here
We mirror Vals AI's ClaimBench v1, released October 20, 2026. Each of its 1,960 claims comes with a 10-K excerpt, and a system labels the claim supported, contradicted or lacking evidence, judging only from the excerpt.
Vals built the board to test small decision models, which return one label with a probability, against frontier language models. Four language models score 100%, so the spread sits among the decision models. The results stay display only and do not enter model rankings.
Snapshot
The published ClaimBench snapshot places d1-ultra first at 100.00%. The third row is 0.00 points behind. The broader top-10 range is 1.22 points, so many of the published results sit in a relatively narrow band.
25 models have been evaluated on ClaimBench. The benchmark falls in the Decision Models category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. ClaimBench is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About ClaimBench
Year
2026
Tasks
1,960 claims
Format
Three-label classification accuracy
Difficulty
Claim verification against SEC 10-K excerpts
Vals released v1 on October 20, 2026. It has 1,960 claims, each paired with a 10-K excerpt and five construction categories of about 392 claims: direct support, subtle contradiction, numeric discrepancy, missing evidence and non-supporting citation. Gold labels come from the construction rule, not from a model. We mirror overall and per-category accuracy as a display-only board, outside overall and category rankings.
Freshness and provenance
Version
ClaimBench 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does ClaimBench measure?
Vals AI finance benchmark that asks whether a 10-K excerpt supports a claim, contradicts it, or says nothing about it. It compares small decision models with frontier language models.
Which model leads the published ClaimBench snapshot?
d1-ultra currently leads the published ClaimBench snapshot with 100.00% accuracy. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on ClaimBench?
The October 23, 2026 snapshot contains 25 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.