Cloudflare Decision Index 0.2.1: VAST
We show this table for reference; we do not rank on it.
Published Cloudflare and community decision-model configurations.
Evaluation results published by Cloudflare and upstream community contributors.
macro-F1 on Cloudflare Decision Index 0.2.1: VAST — October 1, 2026 capture
We mirror the published macro-f1 view for Cloudflare Decision Index 0.2.1: VAST. Surogate Rune 26B-A4B v3 [bf16] leads the public snapshot at 76.6%, followed by Decider chat · Gemma-4-31B (73.2%) and simple-jev · Qwen3.8-27B (featherless) (71.9%). We do not use these results to rank models overall.
Surogate Rune 26B-A4B v3 [bf16]
Upstream community result; full fine-tune; 38/38 index tasks; median 120.5 ms; p95 682.8 ms
Decider chat · Gemma-4-31B
Upstream community result; inference technique; 38/38 index tasks; median 108.5 ms; p95 1114.7 ms
simple-jev · Qwen3.8-27B (featherless)
Upstream community result; inference technique; 38/38 index tasks; median 373 ms; p95 2796.8 ms
73 configurationsDecision ModelsDated mixed-source configurationsDisplay onlyUpdated October 1, 2026 capture
macro-F1 table (73 configurations)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Surogate Rune 26B-A4B v3 [bf16]Creator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 120.5 ms; p95 682.8 ms | 76.6% | Not reported | Unknown |
| 2 | Decider chat · Gemma-4-31BCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 108.5 ms; p95 1114.7 ms | 73.2% | Not reported | Unknown |
| 3 | simple-jev · Qwen3.8-27B (featherless)Creator not reportedUpstream community result; inference technique; 38/38 index tasks; median 373 ms; p95 2796.8 ms | 71.9% | Not reported | Unknown |
| 4 | Eikos-27B [FP8]Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 129.7 ms; p95 509.8 ms | 71.3% | Not reported | Unknown |
| 5 | AutoJev-27BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 101.4 ms; p95 1052.5 ms | 70.8% | Not reported | Unknown |
| 6 | Solomon v1.1Creator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 223.9 ms; p95 1176.8 ms | 69.4% | Not reported | Unknown |
| 7 | XorCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 139.9 ms; p95 419.9 ms | 68.8% | Not reported | Unknown |
| 8 | Winnow-12B [Q8_0]Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 72.5 ms; p95 356.2 ms | 68.6% | Not reported | Unknown |
| 9 | JevTypeSafeUpstream community result; Hosted API (closed); 38/38 index tasks; median 524.1 ms; p95 536 ms | 64.6% | Not reported | Unknown |
| 10 | djevCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 84.4 ms; p95 211.2 ms | 64.6% | Not reported | Unknown |
| 11 | JPT-9BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 148.6 ms; p95 461.9 ms | 64.4% | Not reported | Unknown |
| 12 | Jebadiah 27BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 110.4 ms; p95 591.1 ms | 63.8% | Not reported | Unknown |
| 13 | JPT-4BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 139.1 ms; p95 405.7 ms | 63.4% | Not reported | Unknown |
| 14 | Winnow-E4B [Q8_0]Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 45 ms; p95 188.3 ms | 63.1% | Not reported | Unknown |
| 15 | mmastrac diffusiongemma (vLLM PR 57250)Creator not reportedUpstream community result; inference technique; 38/38 index tasks; median 126.7 ms; p95 274.6 ms | 60.9% | Not reported | Unknown |
| 16 | JevfireCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 89.6 ms; p95 447.5 ms | 60.8% | Not reported | Unknown |
| 17 | reflex 27B [FP8 · wide choice]Creator not reportedUpstream community result; inference technique; 38/38 index tasks; median 108.3 ms; p95 1239.7 ms | 60.5% | Not reported | Unknown |
| 18 | ClefCloudflareCloudflare self-reported; Routing head + LoRA; 36/38 index tasks; median 209.3 ms; p95 238.6 ms | 59.5% | Not reported | Unknown |
| 19 | razorback16 openjev diffusiongemma (NVFP4, vLLM) [one read, NVFP4]Creator not reportedUpstream community result; inference technique; 38/38 index tasks; median 37.7 ms; p95 117.8 ms | 59.2% | Not reported | Unknown |
| 20 | Decider chat · Qwen3.6-27BCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 83.6 ms; p95 965.6 ms | 58.4% | Not reported | Unknown |
| 21 | JoshuaSP diffusiongemma (open-jev)Creator not reportedUpstream community result; inference technique; 38/38 index tasks; median 260.4 ms; p95 537.3 ms | 55.7% | Not reported | Unknown |
| 22 | Kev 9BCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 51.4 ms; p95 187.9 ms | 55.4% | Not reported | Unknown |
| 23 | Jev-OmniCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 54.9 ms; p95 627.9 ms | 54.6% | Not reported | Unknown |
| 24 | Decider 35B-A3B [NVFP4]Creator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 101.4 ms; p95 210.9 ms | 54.5% | Not reported | Unknown |
| 25 | Decider 4BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 12.6 ms; p95 188.5 ms | 53.5% | Not reported | Unknown |
| 26 | Decider 2B [FP8]Creator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 8.1 ms; p95 134.9 ms | 51.0% | Not reported | Unknown |
| 27 | Kev 4BCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 52.1 ms; p95 141.1 ms | 50.9% | Not reported | Unknown |
| 28 | Decision 1.0 SolCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 38.1 ms; p95 101 ms | 50.0% | Not reported | Unknown |
| 29 | this-that 1.2Creator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 44.3 ms; p95 57.2 ms | 50.0% | Not reported | Unknown |
| 30 | Clef-flashCloudflareCloudflare self-reported; Routing head + LoRA; 36/38 index tasks; median 38.8 ms; p95 122.4 ms | 49.6% | Not reported | Unknown |
| 31 | levCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 70.8 ms; p95 687.7 ms | 48.4% | Not reported | Unknown |
| 32 | Jet v6.2Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 46.8 ms; p95 371.1 ms | 47.9% | Not reported | Unknown |
| 33 | Tev1-4B-experimentalCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 35.8 ms; p95 166 ms | 47.7% | Not reported | Unknown |
| 34 | NeoHorse-Jev-4BCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 49 ms; p95 273.9 ms | 47.4% | Not reported | Unknown |
| 35 | Decision 1.0 NoxCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 53.2 ms; p95 242.1 ms | 47.3% | Not reported | Unknown |
| 36 | JevK5Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 22 ms; p95 226.3 ms | 45.7% | Not reported | Unknown |
| 37 | Tev1-0.8B-experimentalCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 28.5 ms; p95 133.9 ms | 44.6% | Not reported | Unknown |
| 38 | Hopper (G) 1.2Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 23.3 ms; p95 238.2 ms | 44.3% | Not reported | Unknown |
| 39 | JPT-0.8BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 117.5 ms; p95 341.2 ms | 44.0% | Not reported | Unknown |
| 40 | SemIfCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 113.1 ms; p95 214.6 ms | 44.0% | Not reported | Unknown |
| 41 | Decision 1.0 LuxCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 51 ms; p95 347.5 ms | 43.9% | Not reported | Unknown |
| 42 | JobeCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 53.5 ms; p95 2910.3 ms | 43.7% | Not reported | Unknown |
| 43 | Decision 1.0 EosCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 39.7 ms; p95 84.3 ms | 43.1% | Not reported | Unknown |
| 44 | mini-jevCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 66.9 ms; p95 178 ms | 43.1% | Not reported | Unknown |
| 45 | Metask-Jev-4BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 57.3 ms; p95 369.4 ms | 43.0% | Not reported | Unknown |
| 46 | openvonsCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 21.2 ms; p95 113.3 ms | 42.5% | Not reported | Unknown |
| 47 | MoJevCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 46.4 ms; p95 55.1 ms | 40.9% | Not reported | Unknown |
| 48 | Intern-Decision-4BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 44.2 ms; p95 115.4 ms | 40.7% | Not reported | Unknown |
| 49 | LayaCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 5.8 ms; p95 222.5 ms | 40.5% | Not reported | Unknown |
| 50 | jeffCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 21.8 ms; p95 67.6 ms | 40.5% | Not reported | Unknown |
| 51 | Decision 1.0 LexCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 29.8 ms; p95 106 ms | 40.4% | Not reported | Unknown |
| 52 | Kev 0.8BCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 41.6 ms; p95 95.9 ms | 39.6% | Not reported | Unknown |
| 53 | Intern-Decision-2BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 33.5 ms; p95 58 ms | 39.3% | Not reported | Unknown |
| 54 | Bespoke Nimble 9B v2Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 77.3 ms; p95 621.4 ms | 39.1% | Not reported | Unknown |
| 55 | open-jev (pngwn)Creator not reportedUpstream community result; LoRA; 38/38 index tasks; median 112.3 ms; p95 724.1 ms | 38.3% | Not reported | Unknown |
| 56 | GLiNER 2.5 smallCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 13.6 ms; p95 35.2 ms | 37.1% | Not reported | Unknown |
| 57 | Bosun v3.1 1.7BCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 40.3 ms; p95 173.5 ms | 36.4% | Not reported | Unknown |
| 58 | LFM2.5-2.6B-RLCDCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 39.3 ms; p95 95.4 ms | 36.3% | Not reported | Unknown |
| 59 | GLiNER 2.5 baseCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 14.1 ms; p95 62.1 ms | 35.8% | Not reported | Unknown |
| 60 | GLiNER2.5-DecideCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 23.3 ms; p95 228.6 ms | 35.3% | Not reported | Unknown |
| 61 | LavoirCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 20 ms; p95 49.5 ms | 35.1% | Not reported | Unknown |
| 62 | system-one-gemmaCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 31.9 ms; p95 748 ms | 34.9% | Not reported | Unknown |
| 63 | Lumma-Fev-0.6BCreator not reportedUpstream community result; LoRA; 38/38 index tasks; median 21.3 ms; p95 219.7 ms | 33.7% | Not reported | Unknown |
| 64 | VerdictCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 11.4 ms; p95 60.1 ms | 33.4% | Not reported | Unknown |
| 65 | Julia 1Creator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 5.8 ms; p95 15.3 ms | 32.9% | Not reported | Unknown |
| 66 | Qwen-2.5-1B-RLCDCreator not reportedUpstream community result; inference technique; 38/38 index tasks; median 43.8 ms; p95 75.2 ms | 31.4% | Not reported | Unknown |
| 67 | Intern-Decision-0.8BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 33.7 ms; p95 39.2 ms | 28.0% | Not reported | Unknown |
| 68 | Lumma-Fev-0.1BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 20.9 ms; p95 112.5 ms | 27.3% | Not reported | Unknown |
| 69 | GLiNER 2.5 multilingualCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 14 ms; p95 92.4 ms | 26.4% | Not reported | Unknown |
| 70 | Bosun v3.1 0.6BCreator not reportedUpstream community result; LoRA + head; 38/38 index tasks; median 38.7 ms; p95 107.3 ms | 26.3% | Not reported | Unknown |
| 71 | LFM2.5-350M-RLCDCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 26.7 ms; p95 174.3 ms | 25.3% | Not reported | Unknown |
| 72 | CLM-v0.1-8BCreator not reportedUpstream community result; full fine-tune; 38/38 index tasks; median 46.8 ms; p95 130.9 ms | 22.7% | Not reported | Unknown |
| 73 | Decision 1.0 KaiCreator not reportedUpstream community result; head / adapter; 38/38 index tasks; median 30.4 ms; p95 106.8 ms | 20.8% | Not reported | Unknown |
How to read this source comparison
The source combines Cloudflare self-reported Clef runs with upstream community configurations. Each row retains its source engine and evidence label; model names alone do not establish equivalent serving settings.
The 0.2.1 composite uses 38 panel benchmarks grouped into five weighted areas. It applies chance correction; ForecastBench uses its Brier transformation. Additional task views retain the source metric and panel-inclusion metadata. The snapshot lists 48 task metrics, while the launch refers to 43 evaluations including latency rows.
Clef and Clef-Flash each cover 36 of 38 index tasks; HLE and iSarcasmEval are missing. Missing values are not inserted as zero. The composite is retained as published without recomputing it or filling gaps.
Community results use the upstream September 28 snapshot, which declares one NVIDIA RTX PRO 6000. Cloudflare run hardware is not established by this artifact. Latency comparisons across these configurations do not isolate model speed. The October 1 date marks capture, not evaluation.
These views remain separate from JevBench and canonical provider benchmark results. They do not enter overall or category rankings. Equal scores do not establish statistical ties.
The published Cloudflare Decision Index 0.2.1: VAST snapshot places Surogate Rune 26B-A4B v3 [bf16] first at 76.6%. The third row is 4.8 points behind. The broader top-10 range is 12.0 points, so the table still separates the published systems.
73 configurations are shown for Cloudflare Decision Index 0.2.1: VAST. The benchmark falls in the Decision Models category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. Cloudflare Decision Index 0.2.1: VAST is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
Freshness and provenance
Version
Cloudflare comparison on Decision Index 0.2.1
Refresh cadence
Reviewed dated source capture
Staleness state
Dated mixed-source configurations
Question availability
Upstream tasks and terms; aggregate scores only
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Cloudflare Decision Index 0.2.1: VAST measure?
Published Cloudflare and community decision-model configurations.
Which model leads the published Cloudflare Decision Index 0.2.1: VAST snapshot?
Surogate Rune 26B-A4B v3 [bf16] currently leads the published Cloudflare Decision Index 0.2.1: VAST snapshot with 76.6% macro-f1. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on Cloudflare Decision Index 0.2.1: VAST?
The October 1, 2026 capture snapshot contains 73 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.