# RoleBench (RoleLLM) (RoleBench)

> Character role-play benchmark from the RoleLLM project that tests whether a model speaks in a given character's style.

Canonical page: https://benchlm.ai/benchmarks/rolebench

- Category: [Instruction Following](/instruction-following)
- Last updated: October 29, 2026

## About RoleBench

- Year: 2023
- Tasks: Character role-play prompts
- Format: Win rate against reference responses
- Difficulty: Character speaking style
- Paper: [RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models](https://github.com/InteractiveNLP-Team/RoleLLM-public)

NVIDIA reports it for Nemotron 3.5 Bolt E4B (October 23, 2026), run in its NeMo Gym harness, as a custom win rate against the reference responses with reasoning off. That is NVIDIA's own scoring, not the metrics in the RoleLLM paper, so it is stored on its own display-only key.

RoleBench is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (2 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Nemotron 3.5 Bolt E4B](/models/nemotron-3-5-bolt-e4b) | NVIDIA | 71.21% |
| 2 | [Nemotron 3.5 Bolt E4B BF16](/models/nemotron-3-5-bolt-e4b-bf16) | NVIDIA | 67.05% |

## FAQ

### What does RoleBench measure?

Character role-play benchmark from the RoleLLM project that tests whether a model speaks in a given character's style.

### Which model scores highest on RoleBench?

Nemotron 3.5 Bolt E4B by NVIDIA currently leads with a score of 71.21% on RoleBench.

### How many models are evaluated on RoleBench?

2 AI models have published results on RoleBench in the BenchLM catalog.

### Does RoleBench affect BenchLM's overall score?

Not directly. RoleBench is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Compare Top Models on RoleBench

- [Nemotron 3.5 Bolt E4B vs Nemotron 3.5 Bolt E4B BF16](/compare/nemotron-3-5-bolt-e4b-vs-nemotron-3-5-bolt-e4b-bf16)