Brevity Benchmark (NVIDIA) (Brevity Benchmark)
We show this table for reference; we do not rank on it.
NVIDIA benchmark that scores conversational answers on quality and conciseness together.
Benchmark score on Brevity Benchmark — October 29, 2026
We compile the Brevity Benchmark rows from provider self-reports. The table contains 2 models. Nemotron 3.5 Bolt E4B has the highest published score at 82.84%, but the available coverage and evaluation setups do not establish a market leader. We do not use these results to rank models overall.
Nemotron 3.5 Bolt E4B
NVIDIA
Nemotron 3.5 Bolt E4B BF16
NVIDIA
2 modelsInstruction FollowingCurrentDisplay onlyUpdated October 29, 2026
Benchmark score table (2 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Nemotron 3.5 Bolt E4BNVIDIA | 82.84% | Not reported | Open |
| 2 | Nemotron 3.5 Bolt E4B BF16NVIDIA | 82.52% | Not reported | Open |
How to read this leaderboard
Operator receipt: 2 sourced rows are currently displayable on this page; the highest published score among these 2 models is Nemotron 3.5 Bolt E4B at 82.84%, which does not establish a market leader.
Honest limit: Unreleased and built by the model's own publisher; the task set and judge cannot be checked until NVIDIA publishes them. Only NVIDIA-run results are stored so far, from one harness. Display-only and excluded from weighted ranking.
About Brevity Benchmark
Year
2026
Tasks
Conversational prompts (not yet published)
Format
Average of quality and conciseness scores
Difficulty
Concise, speech-ready answers
NVIDIA reports it for Nemotron 3.5 Bolt E4B (October 23, 2026), run in its NeMo Gym harness, as the average of answer quality and conciseness in reasoning mode. NVIDIA says the benchmark will be released later, so its tasks, judge and scoring are not public yet. We store the launch runs on their own display-only key.
Freshness and provenance
Version
Brevity Benchmark 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does Brevity Benchmark measure?
NVIDIA benchmark that scores conversational answers on quality and conciseness together.
Which model scores highest on Brevity Benchmark?
Nemotron 3.5 Bolt E4B by NVIDIA currently leads with a score of 82.84% on Brevity Benchmark.
How many models are evaluated on Brevity Benchmark?
2 AI models have published results on Brevity Benchmark in the BenchLM catalog.
Compare top models on Brevity Benchmark
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.