Skip to main content
BenchLM
Data
JevBench v1.5.4 · retrieved November 16, 2026 · 112 systems · 106 ranked

Best AI decision models

Cygnet and Winnow-12B Q8 are joint leaders in JevBench v1.5.4, with a published statistical tie. Compare Jev for hosted access and Liquid d1 for small local deployments. JevBench balances quality, confidence calibration, speed, and cost across 1,624 decisions; your workload can favor a different model.
  1. JevBench rank 1 Tied with rank 2

    73.70

    Cygnet

    blockbrain · 95% CI 72.36–74.46

  2. JevBench rank 2

    73.23

    Winnow-12B Q8

    Eldan Ring · 95% CI 72.02–73.99

  3. JevBench rank 3

    72.13

    Jev 1.13.0

    TypeSafe AI · 95% CI 71.01–72.61

  4. JevBench rank 4

    71.90

    JevK5 v0.3 4B

    allebee · 95% CI 69.39–72.95

JevBench v1.5.4 · retrieved November 16, 2026All 112 systems

Which decision model should you try?

Compare candidates for ticket routing, classification, yes/no checks, and rubric scoring. Choose by deployment needs, then test your own labeled cases. JevBench points, Decision Index skill points, and provider accuracy percentages measure different things, so the shortlist groups candidates by need.

AI decision model shortlist, access, evidence, and trade-offs
Starting pointModelAccess and priceEvidence and trade-off
Hosted typed decisions with optional explanationsHosted API: $0.00 / 1M input tokens; free outputOfficial pricing · checked 2026-11-10

Independent Decision Index v1.0 results and the fastest measured time per task; Celeris-reported accuracy and Brier scores for Celeris-1 Decision

Closed weights. Its decision-benching report does not establish an official JevBench rank. Test confidence and latency on your own workload.

Hosted multimodal decisions over text, images, video, audio and PDFsHosted API: $0.10 / 1M input tokens; free outputOfficial pricing · checked 2026-11-11

Independent Decision Index v2.1 results; Google-reported results on nine benchmarks, including 92.2% on the JevBench public split, and the lowest top-1 calibration error of the four systems Google compared

Closed weights. At $0.10 per million input tokens, its Decision Index cost per task is about three times Drex Max v1.1’s. The Decisions API is a v1beta endpoint without Batch API or caching.

Hosted decisions over text, images and videoHosted API; check official pricingOfficial pricing · checked 2026-11-13

HeyGen documentation only: up to 400 named questions per request over up to four images and one video, with a probability for every answer

Closed weights and no published benchmark results or independent board yet. The Turbo model is in preview. A public per-token price is not posted yet.

Long inputs in one decision callHosted API: $0.04 / 1M input tokens; free outputOfficial pricing · checked 2026-11-11

Independent Decision Index v2.1 results; Subquadratic-reported speed (100K tokens in about a second) and a provider-run Decision Index 0.2.1 subset

Closed weights and no published API model ID in the reviewed sources. The long-context claims are Subquadratic’s own; test them on your inputs.

Typed decisions inside an existing OpenAI stackHosted API: $0.10 / 1M input tokens; free outputOfficial pricing · checked 2026-11-11

Independent Decision Index v2.1 results

Runs only on gpt-6-luna through POST /v1/decisions. Closed weights; GPT-6 Luna language-model scores do not carry over to this configuration.

Strongest published JevBench resultOpen weights; self-hosted evaluation

Cygnet: 73.70; Winnow-12B Q8: 73.23 JevBench points; statistical tie.

Requires your own serving setup. Source cost and speed assumptions are not a production bill or latency guarantee.

Hosted API with a strong JevBench resultProprietary hosted API

JevBench #3, 72.13 points; operator API evaluation.

Closed weights. Sealed questions were sent to the operator, and the listed decision cost retains benchmark pricing assumptions.

Small local model with image or audio inputOpen weights; hardware costs apply

Liquid-reported Decision Index 0.2.1 public split for d1-3B and d1-omni-600M; Liquid-run Decision Index 0.3 public suite, Decision Bench, JevBench and own-board results for the 2B models

No shared JevBench result here. The 600M checkpoint is experimental, cannot combine image and audio input, and has tighter input limits. LFM license conditions apply.

Hosted decisions with the strongest independent index resultHosted API: $0.042 / 1M input tokens, no output tokensOfficial pricing · checked October 15, 2026

Independent Decision Index v1.0 results on the benchmark page (November 6, 2026 snapshot)

Closed weights and no published context limit. Its knowledge accuracy trails d1, and the index is a different protocol from JevBench and Decision Index 0.2.1.

Low token tariff with text and image inputHosted API: $0.02 / 1M input tokens; free outputOfficial pricing · checked 2026-11-12

Decision Index board for v1.3 and v1.2; Vals ClaimBench for v1.2; the 11-task Perplexity panel covers v1

The provider panel and launch-post evaluations do not establish a JevBench rank or probability calibration. Probabilities can change between v1.2 and v1.3, so retune thresholds before switching. Apache 2.0 weights are available, but local serving has separate costs.

Open-weight decisions in one forward passHosted API: $0.04 / 1M input tokens; free outputOfficial pricing · checked 2026-11-05

Decision Index board and independent Decision Index v1.0 results; Fastino-reported Decision Index 0.2.1 for the first GLiDE

The hosted fastino/GLiDE id now serves v1.2. Weights carry the Fastino Community License, and local serving has separate costs.

Open-weight multimodal decisions at a low input tariffHosted API: $0.03 / 1M input tokens; free outputOfficial pricing · checked 2026-11-05

Decision Index board and independent Decision Index v1.0 results

Drex Max v1.1 is hosted only; the open checkpoint is the original Drex Max under the Nace.AI Open RAIL-M license. Each Max model is a pinned id, and drex-latest still points to Drex v2.0.

JevBench decision-model leaderboard

JevBench v1.5.4 covers 1624 decisions. Its official score combines Intelligence, Calibration, Speed, and Cost; it is an index, rather than percent accuracy. The 112 configurations include 106 ranked systems and retain the source's unranked entries.

The official headline is option A, selected after the method owner reviewed the results. Published composite intervals and adjacent-pair statistical ties remain attached to each configuration. Cost per 1,000 decisions retains source tariffs or estimates. This composite stays outside general model rankings.

Option A · joint leaders (statistical tie). The ≈ marker identifies a published statistical tie with the next row. Missing pairwise markers establish neither a tie nor a separation; 95% intervals below describe individual composite scores.

Official JevBench 1.5 results (112 systems)
RankSystemScoreIntelligenceCalibrationSpeedCostSealed IntelligenceUSD / 1,000 decisionsWho ran itParameters (B)Open / closed
1 ≈Cygnetblockbrain · system-one-open
Evaluation setup

Source entry: Cygnet (blockbrain, frozen Gemma-4-12B-it). Published attribution: blockbrain.

Model author or lab

evaluator-owned Lium GPU pod (RTXPRO6000), offline read-only container

Adjusted latency: p50 0.230 s; p95 0.348 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; no exact base-model floor applies

Secondary composites: B 73.16; C 73.70.

choice: open 82.92; sealed 80.29 (chance-corrected competence).

noul: open 57.86; sealed 55.67 (chance-corrected competence).

score: open 76.60; sealed 73.20 (chance-corrected competence).

73.7095% CI 72.36–74.4671.0987.0190.9756.4369.720.02834estimateRun by JevBenchNot reportedOpen
2Winnow-12B Q8Eldan Ring · jev-rebuild
Evaluation setup

Source entry: Winnow-12B Q8. Published attribution: Eldan Ring.

Model author or lab

evaluator-owned Lium GPU pod (RTX6000), offline read-only container

Adjusted latency: p50 0.340 s; p95 0.715 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; no exact base-model floor applies

Secondary composites: B 73.47; C 73.23.

choice: open 77.61; sealed 82.04 (chance-corrected competence).

noul: open 58.61; sealed 66.15 (chance-corrected competence).

score: open 81.21; sealed 80.99 (chance-corrected competence).

73.2395% CI 72.02–73.9974.4384.0786.1456.5676.390.02806estimateRun by JevBenchNot reportedOpen
3Jev 1.13.0TypeSafe AI · jevAPI: sealed item text sent to the operator
Evaluation setup

Source entry: Jev 1.13.0 (TypeSafe AI). Published attribution: TypeSafe AI.

Model author or lab

the operator's hosted API

Adjusted latency: p50 0.616 s; p95 0.674 s. none (hosted API, measured as is)

operator standard launch list price (interpretation I-1); no exact base-model floor applies

Secondary composites: B 72.11; C 72.13.

choice: open 85.73; sealed 87.59 (chance-corrected competence).

noul: open 47.75; sealed 48.62 (chance-corrected competence).

score: open 81.16; sealed 81.14 (chance-corrected competence).

72.1395% CI 71.01–72.6172.0088.0383.8154.7372.450.03230estimateRun by JevBenchNot reportedClosed
4JevK5 v0.3 4Ballebee · unclassified
Evaluation setup

Source entry: JevK5 v0.3 (4B). Published attribution: unknown.

Model author or lab

evaluator-owned Lium GPU pod (H100), offline read-only container

Adjusted latency: p50 0.182 s; p95 0.243 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; reference deepinfra:Qwen/Qwen3.5-4B frozen at the 25 Sep cut-off; the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen/Qwen3.5-9B; the frozen snapshot rates remain in use (I-3)

Secondary composites: B 68.11; C 63.24.

choice: open 80.25; sealed 74.25 (chance-corrected competence).

noul: open 42.33; sealed 14.98 (chance-corrected competence).

score: open 62.22; sealed 63.59 (chance-corrected competence).

v1.5 roster addendum A1. No pairwise comparison is inferred for this addition.

71.9095% CI 69.39–72.9556.2788.3493.5563.0750.940.01702estimateRun by JevBenchNot reportedUnknown
5Plumb-4Bcrh225 · unclassified
Evaluation setup

Source entry: Plumb-4B (crh225, JevK5 v0.2 + LoRA). Published attribution: unknown.

Model author or lab

evaluator-owned Lium GPU pod (H100), offline read-only container

Adjusted latency: p50 0.185 s; p95 0.245 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; reference deepinfra:Qwen/Qwen3.5-4B frozen at the 25 Sep cut-off; the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen/Qwen3.5-9B; the frozen snapshot rates remain in use (I-3)

Secondary composites: B 67.75; C 62.00.

choice: open 80.43; sealed 73.30 (chance-corrected competence).

noul: open 42.95; sealed 16.58 (chance-corrected competence).

score: open 59.26; sealed 62.57 (chance-corrected competence).

v1.5 roster addendum A1. No pairwise comparison is inferred for this addition.

71.5695% CI 69.20–72.7355.8587.4493.4563.0750.810.01702estimateRun by JevBenchNot reportedOpen
6 ≈Jev-Omniakhilaaa3 · jev-rebuild
Evaluation setup

Source entry: Jev-Omni (akhilaaa3, Gemma-4-12B merged). Published attribution: akhilaaa3.

Model author or lab

evaluator-owned Lium GPU pod (RTX6000), offline read-only container

Adjusted latency: p50 0.375 s; p95 0.908 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; no exact base-model floor applies

Secondary composites: B 71.30; C 71.50.

choice: open 83.37; sealed 78.70 (chance-corrected competence).

noul: open 55.28; sealed 56.73 (chance-corrected competence).

score: open 72.79; sealed 76.07 (chance-corrected competence).

71.5095% CI 70.21–72.4070.4982.6084.6856.0570.500.02917estimateRun by JevBenchNot reportedOpen
7decider-4b v2Mapika · system-one-open
Evaluation setup

Source entry: decider-4b v2 (Mapika). Published attribution: Mapika.

Model author or lab

evaluator-owned Lium GPU pod (RTX5090), offline read-only container

Adjusted latency: p50 0.203 s; p95 0.405 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; reference deepinfra:Qwen/Qwen3.5-4B frozen at the 25 Sep cut-off; the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen/Qwen3.5-9B; the frozen snapshot rates remain in use (I-3)

Secondary composites: B 67.53; C 61.59.

choice: open 80.43; sealed 77.31 (chance-corrected competence).

noul: open 42.92; sealed 18.73 (chance-corrected competence).

score: open 61.88; sealed 53.36 (chance-corrected competence).

71.2895% CI 69.11–72.3555.7785.5990.8664.5449.800.01520estimateRun by JevBenchNot reportedOpen
8Decision 4B v1.2FlyMy.AI · unclassified
Evaluation setup

Source entry: Decision 4B v1.2 (FlyMyJev, Qwen3.5-4B + LoRA). Published attribution: unknown.

Model author or lab

evaluator-owned Lium GPU pod (H100), offline read-only container

Adjusted latency: p50 0.183 s; p95 0.244 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; reference deepinfra:Qwen/Qwen3.5-4B frozen at the 25 Sep cut-off; the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen/Qwen3.5-9B; the frozen snapshot rates remain in use (I-3)

Secondary composites: B 66.57; C 56.66.

choice: open 80.87; sealed 73.76 (chance-corrected competence).

noul: open 20.13; sealed 16.44 (chance-corrected competence).

score: open 67.79; sealed 63.00 (chance-corrected competence).

v1.5 roster addendum A1. No pairwise comparison is inferred for this addition.

70.8395% CI 68.52–71.9753.6688.5693.5063.0751.060.01702estimateRun by JevBenchNot reportedUnknown
9Imajev-4B (RTX 5090)mohit67890 · unclassified
Evaluation setup

Source entry: Imajev-4B (RTX 5090). Published attribution: unknown.

Model author or lab

evaluator-owned Lium GPU pod (RTX 5090), offline read-only container

Adjusted latency: p50 0.234 s; p95 0.329 s. x2 + 0.15 s (assumption, not measured)

ESTIMATE (I-2): measured input tokens; zero generated output tokens for signed logits readout. M2 floor uses the 25 Sep 2026 DeepInfra Qwen3.5-4B snapshot rates (USD 0.03/M input, USD 0.15/M output); the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen3.5-9B.

Secondary composites: B 66.20; C 55.91.

choice: open 80.12; sealed 82.33 (chance-corrected competence).

noul: open 21.82; sealed 7.56 (chance-corrected competence).

score: open 66.80; sealed 62.21 (chance-corrected competence).

v1.5 roster addendum A2. No pairwise comparison is inferred for this addition.

70.3995% CI 67.80–71.6153.4788.1291.1363.2650.700.01677estimateRun by JevBenchNot reportedUnknown
10Decision 4B v1.1FlyMy.AI · unclassified
Evaluation setup

Source entry: Decision 4B v1.1 (FlyMyJev, Qwen3.5-4B + LoRA). Published attribution: unknown.

Model author or lab

evaluator-owned Lium GPU pod (H100), offline read-only container

Adjusted latency: p50 0.183 s; p95 0.243 s. x2 + 0.15 s (assumption, not measured)

documented hosted-model estimate; reference deepinfra:Qwen/Qwen3.5-4B frozen at the 25 Sep cut-off; the snapshot records deprecation on 11 Jun 2026 and replacement by Qwen/Qwen3.5-9B; the frozen snapshot rates remain in use (I-3)

Secondary composites: B 66.09; C 55.14.

choice: open 80.12; sealed 74.77 (chance-corrected competence).

noul: open 18.02; sealed 15.75 (chance-corrected competence).

score: open 65.82; sealed 64.17 (chance-corrected competence).

v1.5 roster addendum A1. No pairwise comparison is inferred for this addition.

70.3995% CI 66.86–71.5753.1187.3193.5263.0751.560.01702estimateRun by JevBenchNot reportedUnknown

Showing the first 10 ranked systems. Search and filters check the full roster.

Read the JevBench method and result notes, or inspect the pinned official results.

JevBench by Florian Standhartinger and contributors. MIT license and copyright notice.

Choose a system for your own decisions

Match the decision to the test

Check the rubric, input length, answer options, and model configuration. A reranker adapted to pick an option and a native decision model can answer the same question through different serving paths.

Check confidence against errors

A calibrated model that reports 80% confidence should be correct about eight times in ten on comparable cases. Test an escalation threshold on your own labeled cases before letting a high-confidence answer approve an action or bypass a larger model.

Price the whole decision

Compare cost per completed decision alongside latency and accuracy. JevBench estimates some hosting costs from base-model reference prices and adjusts self-hosted latency. Those assumptions can change which system fits your budget.

Typed decisions your code can use

Decision models take application state, a written question, and an allowed answer space. Their output fits a check your code can act on. A bounded answer can still be wrong, and reported confidence needs testing on the decisions your application actually makes.

The OpenRouter Jev explainer shows each primitive with a real API response.

Decision primitives and example tasks
PrimitiveExample taskOutput
ChoiceRoute a support ticket to billing, technical support, or account support.An option and its probability distribution.
NoulCheck whether a request meets a written approval rule.A probability that a yes/no proposition holds.
ScoreRate a bug against an ordered severity rubric.A score and probabilities over the allowed levels.

Provider reports answer narrower questions

Celeris, Liquid, Perplexity, and Fastino publish useful comparisons of their own models. Keep each report’s task set and serving conditions attached to its result. A lead in one report does not establish a lead on JevBench.

Celeris-1 Decision returns probabilities over defined answers

Celeris released Celeris-1 Decision on October 8, 2026, as a hosted diffusion model for Noul, Choice, and Score questions. It accepts text, JSON, and images through a System One-compatible API, with optional one-sentence explanations. Celeris lists $0.04 per million input tokens, including cached input, and free output. Reported input usage includes the internal request representation and images.

Show Celeris’s reported results and evaluation limits

Celeris reports 80.0% accuracy on 22 jev-bench datasets totaling 3,210 items and 76.5% on 400 typed-decisions cases with five questions each. The table preserves its October 2026 comparison, run through each provider’s decision endpoint with one scorer. Lower Brier scores mean smaller errors in the reported probabilities.

These are provider-reported results from decision-benching harness 0cfd276. This jev-bench accuracy is separate from the official JevBench composite and Perplexity’s 11-task panel. The two Brier scores use different gold labels: hard labels for jev-bench and soft distributions for typed decisions. Failed requests or missing distributions were scored as uniform. Latency was measured end to end from AWS us-east-1; optional explanations add latency. We have not rerun this evaluation, and these results stay outside general model rankings.

Celeris-reported decision-model accuracy, Brier scores, and latency
MetricCeleris-1 DecisionDecider v1.1 27BMercury DecideJev 1.13.0GPT-6 Luna
jev-bench accuracy · 3,210 items80.0%79.0%75.3%73.1%69.0%
jev-bench Brier · lower is better0.2720.2880.3720.3800.455
Typed-decisions accuracy · 400 × 5 questions76.5%73.8%71.5%73.6%71.8%
Typed-decisions Brier · lower is better0.0790.1730.2700.1490.204
Median response · 1 question67 ms122 ms147 ms135 ms110 ms
Median response · 50 questions81 ms888 ms460 ms153 ms175 ms

Open d1 makes decisions from text and media

Liquid AI released d1-3B and the experimental d1-omni-600M on October 7, 2026. Both return Noul, Choice, or Score answers in one forward pass with zero generated tokens. d1-3B has 3.12B parameters, text and image input, and a 32,768-token context. d1-omni-600M has 587M parameters and accepts text with images or text with up to 30 seconds of speech. Its combined context is 16,384 positions; image requests cap state and question text at 896 tokens. Images and audio cannot share one request. On October 27, 2026, Liquid added d1-2B and d1-Omni-2B on the LFM3 backbone, with 128K contexts, video input, spoken questions on d1-Omni-2B, and new Point, Read, Estimate and Pick answer types; Liquid says d1-2B replaces d1-3B. Their Liquid-run results are on each model page, not on the 0.2.1 rows.

Show Liquid’s reported scores and evaluation limits
Liquid-reported Decision Index 0.2.1 skill points
Metricd1-3Bd1-omni-600M
Decision Index 0.2.148.5715.95
Knowledge23.88.3
Language56.412.9
Retrieval52.835.0
Tools74.515.1
Arts36.36.8

Liquid AI evaluated these checkpoints with the official Decision Index 0.2.1 scorer on the public split. These are provider-reported runs, not public leaderboard submissions. The index and five areas are skill points, not percent accuracy. They remain separate from Decision Index 0.3, JevBench, and general model rankings; no private vision-split result is reported. Zero token prices on these profiles describe self-hosting; hardware costs and the LFM Open License v1.0 commercial-use conditions still apply.

d1-ultra is Liquid’s hosted decision model

Liquid AI released d1-ultra on October 15, 2026, as a hosted decision model on the Liquid API. It reads text, images, or both and returns Noul, Choice, or Score answers in one forward pass without generating tokens. Liquid lists it at $0.042 per million input tokens, billed on input only, with Vercel and OpenRouter listed as coming soon. Liquid publishes no weights, parameter count, or context limit and says only that it builds on a larger open-weight model than d1.

Independent Decision Index v1.0 results for d1-ultra and other decision models are on the benchmark page (November 6, 2026 snapshot). d1-ultra pairs the highest index there (65.2) with an 84.5 non-hallucination rate, while its 31.6 knowledge accuracy sits below d1 at 41.0. Liquid’s own 112-evaluation comparison with d1 and Jev 1.14 is a provider-run suite without published tasks, so it is not mirrored. The index is a different protocol from Decision Index 0.2.1 and 0.3.

Perplexity Decider v1 27B

Perplexity released Decider v1 27B on October 1, 2026, as a Qwen3.8-27B fine-tune with Apache 2.0 weights. It reads text, JSON, and images, then returns yes/no probabilities, choices, or rubric scores. The hosted Decisions API bills input tokens only, with free output and no per-request fee. Since October 16, requests that name v1 run Decider v1.2, which has its own page with the 4B Flash v1.2.

GLiDE makes structured decisions with adaptive reasoning

Fastino released GLiDE on September 30, 2026. It returns Noul, Choice, and Score answers with probabilities and confidence, and spends additional reasoning on uncertain decisions. The API accepts 40,000 tokens per rendered question prompt. Input usage sums internal passes across questions. The current hosted tariff is shown in the shortlist and official pricing page.

Show Fastino’s comparison with Jev

GLiDE on Decision Index 0.2.1

Fastino reports 64.81 skill points for GLiDE on Decision Index 0.2.1, compared with 57.91 for the published Jev 1.13.0 reference. Its September 30, 2026 announcement uses the official scorer; GLiDE is not listed on the public board.

Fastino-reported GLiDE results and published Jev reference on Decision Index 0.2.1
MetricUnitGLiDEParameters (B): Not reportedOpen / closed: UnknownTypeSafe Jev 1.13.0Parameters (B): Not reportedOpen / closed: Unknown
Decision Index 0.2.1Skill points64.8157.91
Knowledge and ReasoningSkill points62.951.4
Language UnderstandingSkill points63.862.0
Retrieval and ClassificationSkill points60.955.4
Tools and AutomationSkill points83.575.1
Arts and Human TasteSkill points46.037.7
CLadderAccuracy88.7%72.6%
CRUXEvalAccuracy92.6%73.0%

Other overall references in Fastino’s launch chart

Additional overall skill scores quoted in the Fastino GLiDE launch chart
Evaluated systemSkill pointsSource qualification
d1Parameters (B): Not reported · Open / closed: Unknown58.9Liquid AI self-reported reproduction, quoted by Fastino
Surogate Rune 26B-A4B v3Parameters (B): Not reported · Open / closed: Unknown57.44Published Decision Index reference quoted by Fastino
Decider chat · Gemma-4-31BParameters (B): Not reported · Open / closed: Unknown57.33Inference technique on Gemma-4-31B; published reference quoted by Fastino
AutoJev-27BParameters (B): Not reported · Open / closed: Unknown56.40Published Decision Index reference quoted by Fastino
simple-jev · Qwen3.8-27BParameters (B): Not reported · Open / closed: Unknown55.74Featherless inference technique; published reference quoted by Fastino

The five area scores and overall index are chance-adjusted skill points. CLadder and CRUXEval are raw accuracies. Fastino reports a complete run, but publishes exact GLiDE values for only these eight metrics. Radar-chart gaps do not supply the other individual benchmark accuracies. We have not rerun the evaluation, and these results stay outside general model rankings. The launch chart also quotes five other overall references, including Liquid AI’s self-reported d1 reproduction; their category and task scores are not inferred.

Perplexity’s 11-task decision panel

Perplexity reports 85.71% accuracy for Decider v1 27B, 84.51% for TypeSafe Jev 1.13.0, and 74.76% for Qwen3.8-27B on a fixed 7,210-row panel. The chart covers September 2026 and was published October 1. Decider was measured through the Perplexity API. These sampled accuracy results do not establish a shared leaderboard rank or confidence calibration.

Show all 11 tasks and panel limits

These are provider-reported results. The task samples have different sizes, so the published overall weights them differently. Prompts, sampled item identities, label mappings, baseline inference settings, and uncertainty are not specified in the model card. This panel stays separate from full-dataset results and general model rankings; JevBench public-hard accuracy is separate from its full composite.

Perplexity-reported accuracy on the September 2026 decision panel
BenchmarkRowsJev 1.13.0Qwen3.8-27BPerplexity Decider
WinoGrande1,00090.70%73.10%83.30%
FinancialPhraseBank99976.98%75.68%84.18%
RAGTruth1,50077.27%61.53%88.80%
JudgeBench35078.57%68.86%78.29%
BBH75094.27%72.80%82.80%
JevBench public hard10173.27%72.28%70.30%
TabFact50089.80%78.60%90.60%
ContractNLI51077.45%80.78%80.78%
Circa50084.60%87.00%89.20%
Belebele50095.00%93.20%94.00%
TruthfulQA binary50092.00%82.80%85.40%
Overall7,21084.51%74.76%85.71%

Emphasized values mark the best percentage in each row, including ties. Read the panel result notes, the pinned Perplexity model card, or the official launch and chart.

Benchmarks with a different scope

DecisionBench by Hanno Labs. A separate benchmark for typed decisions. Its applied suite and reasoning track stay separate; results are linked at the source rather than imported into this table.

Find the benchmark closest to your task

The seven original benchmark pages include 40 source tables, with full numeric metrics and profiles for identified evaluated configurations. Each page provides a JSON download. Anonymous submissions and conflicting source values are labeled.

These datasets cover different decisions, from sentiment labels to evidence checks. Compare scores only when the sample, label mapping, language, and evaluation setup match. A subset or binary conversion keeps its own result.

WinoGrande

Resolve an ambiguous reference in a sentence.

RAGTruth

Detect unsupported claims against retrieved evidence.

JudgeBench

Choose the objectively better response from a pair.

BBH

Solve challenging reasoning tasks from BIG-Bench.

JevBench

Evaluate decision systems; public-subset accuracy is separate from the full composite.

TabFact

Check whether a table supports or refutes a statement.

ContractNLI

Classify a contract hypothesis and identify supporting evidence.

Circa

Interpret an indirect answer to a yes/no question.

Belebele

Answer passage-comprehension questions across language variants.

TruthfulQA

Test resistance to common misconceptions; a binary conversion is a separate setup.

Questions

What is the best AI decision model?

Cygnet and Winnow-12B Q8 are joint leaders in JevBench v1.5.4, with a published statistical tie. That composite combines quality, calibration, speed, and cost on one task set. Choose from the shortlist by deployment needs, then test your own labeled cases before trusting confidence thresholds or automating consequential actions.

Which decision models can run locally?

Cygnet and Winnow-12B Q8 have open weights and self-hosted JevBench evaluations. Liquid d1-3B supports text and images, while the experimental d1-omni-600M adds audio under separate input limits. Perplexity Decider v1 27B also has open weights. Check each checkpoint’s license, runtime, and hardware needs; open weights do not make serving free.

What does Perplexity Decider’s 85.71% measure?

The 85.71% is Perplexity’s reported accuracy across a fixed 7,210-row panel of 11 tasks from September 2026. Task samples have different sizes, and Decider was measured through the Perplexity API. The percentage measures correctness on those samples; it does not establish probability calibration or replace JevBench’s full composite.

What does Perplexity Decider cost?

Perplexity bills pplx-decider-v1.3 and v1.2 at $0.02 per million input tokens, and the 4B Flash versions at $0.01, with free output and no per-request fee. The latest aliases select v1.3, and requests that still name v1 or v1.1 run v1.2 at its rate. State, images, and every question count as input, so price a completed decision from the returned input-token count.

How are decision models different from reasoning models?

Decision models answer within a defined set of choices or score levels, often with probabilities your code can use. Reasoning models are evaluated on solving broader problems. A general-purpose model can act as a decision system, but its result depends on the prompt, adapter, and serving setup.

Which benchmarks evaluate AI decision models?

JevBench evaluates typed decision systems across intelligence, calibration, speed, and cost. DecisionBench from Hanno Labs publishes task-level accuracy and probability-quality measures, with applied decisions and reasoning in separate tracks. This page mirrors the pinned JevBench release and links to DecisionBench directly; the two scores are not interchangeable.

Does the Decision Models category change overall rankings?

The category groups decision-system evidence for discovery. JevBench retains its official ranking and remains outside overall and category score calculations because its composite includes serving latency, pricing assumptions, and adapters. Model pages keep the evaluated configuration separate from its base model, so scores are not inherited between them.