Testing the Limits of Chain-of-thought with Multistep Soft Reasoning (MuSR)
We show this table for reference; we do not rank on it.
A dataset for evaluating language models on multistep soft reasoning tasks specified in natural language narratives. Tests the ability to perform complex, structured reasoning.
Benchmark score on MuSR — October 19, 2026
We compile the MuSR rows from provider self-reports. Ternary Bonsai 8B leads the table at 56.2%, followed by Ternary Bonsai 1.7B (50.8%) and 1-bit Bonsai 8B (50%). We do not use these results to rank models overall.
Ternary Bonsai 8B
Prism ML
Ternary Bonsai 1.7B
Prism ML
1-bit Bonsai 8B
Prism ML
6 modelsReasoningStaleDisplay onlyUpdated October 19, 2026
Benchmark score table (6 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Ternary Bonsai 8BPrism ML | 56.2% | Not reported | Open |
| 2 | Ternary Bonsai 1.7BPrism ML | 50.8% | Not reported | Open |
| 3 | 1-bit Bonsai 8BPrism ML | 50% | Not reported | Open |
| 4 | Ternary Bonsai 4BPrism ML | 45.1% | Not reported | Open |
| 5 | 1-bit Bonsai 1.7BPrism ML | 45.1% | Not reported | Open |
| 6 | 1-bit Bonsai 4BPrism ML | 41.4% | Not reported | Open |
Among the reported MuSR rows, Ternary Bonsai 8B is first at 56.2%. The third row is 6.2 points behind. The broader top-10 range is 14.8 points, so the table still separates the published systems.
6 models have been evaluated on MuSR. The benchmark falls in the Reasoning category. MuSR is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About MuSR
Year
2023
Tasks
Multi-step reasoning
Format
Narrative-based reasoning
Difficulty
Complex reasoning tasks
MuSR challenges models to perform multistep reasoning over complex narratives. Unlike simple factual questions, it requires models to track multiple entities, relationships, and logical steps across extended contexts.
Freshness and provenance
Version
MuSR 2023
Refresh cadence
Static
Staleness state
Stale
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MuSR measure?
A dataset for evaluating language models on multistep soft reasoning tasks specified in natural language narratives. Tests the ability to perform complex, structured reasoning.
Which model scores highest on MuSR?
Ternary Bonsai 8B by Prism ML currently leads with a score of 56.2% on MuSR.
How many models are evaluated on MuSR?
6 AI models have published results on MuSR in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.