RoleBench (RoleLLM) (RoleBench)
We show this table for reference; we do not rank on it.
Character role-play benchmark from the RoleLLM project that tests whether a model speaks in a given character's style.
Benchmark score on RoleBench — October 29, 2026
We compile the RoleBench rows from provider self-reports. The table contains 2 models. Nemotron 3.5 Bolt E4B has the highest published score at 71.21%, but the available coverage and evaluation setups do not establish a market leader. We do not use these results to rank models overall.
Nemotron 3.5 Bolt E4B
NVIDIA
Nemotron 3.5 Bolt E4B BF16
NVIDIA
2 modelsInstruction FollowingStaleDisplay onlyUpdated October 29, 2026
Benchmark score table (2 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Nemotron 3.5 Bolt E4BNVIDIA | 71.21% | Not reported | Open |
| 2 | Nemotron 3.5 Bolt E4B BF16NVIDIA | 67.05% | Not reported | Open |
How to read this leaderboard
Operator receipt: 2 sourced rows are currently displayable on this page; the highest published score among these 2 models is Nemotron 3.5 Bolt E4B at 71.21%, which does not establish a market leader.
Honest limit: The win-rate scoring is NVIDIA's adaptation of the benchmark. Only NVIDIA-run results are stored so far, from one harness. Display-only and excluded from weighted ranking.
About RoleBench
Year
2023
Tasks
Character role-play prompts
Format
Win rate against reference responses
Difficulty
Character speaking style
NVIDIA reports it for Nemotron 3.5 Bolt E4B (October 23, 2026), run in its NeMo Gym harness, as a custom win rate against the reference responses with reasoning off. That is NVIDIA's own scoring, not the metrics in the RoleLLM paper, so it is stored on its own display-only key.
Freshness and provenance
Version
RoleBench 2023
Refresh cadence
Static
Staleness state
Stale
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does RoleBench measure?
Character role-play benchmark from the RoleLLM project that tests whether a model speaks in a given character's style.
Which model scores highest on RoleBench?
Nemotron 3.5 Bolt E4B by NVIDIA currently leads with a score of 71.21% on RoleBench.
How many models are evaluated on RoleBench?
2 AI models have published results on RoleBench in the BenchLM catalog.
Compare top models on RoleBench
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.