# Testing the Limits of Chain-of-thought with Multistep Soft Reasoning (MuSR)

> A dataset for evaluating language models on multistep soft reasoning tasks specified in natural language narratives. Tests the ability to perform complex, structured reasoning.

Canonical page: https://benchlm.ai/benchmarks/musr

- Category: [Reasoning](/reasoning)
- Last updated: October 19, 2026

## About MuSR

- Year: 2023
- Tasks: Multi-step reasoning
- Format: Narrative-based reasoning
- Difficulty: Complex reasoning tasks
- Paper: [MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning](https://arxiv.org/abs/2310.16049)

MuSR challenges models to perform multistep reasoning over complex narratives. Unlike simple factual questions, it requires models to track multiple entities, relationships, and logical steps across extended contexts.

MuSR is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (6 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [Ternary Bonsai 8B](/models/ternary-bonsai-8b) | Prism ML | 56.2% |
| 2 | [Ternary Bonsai 1.7B](/models/ternary-bonsai-1-7b) | Prism ML | 50.8% |
| 3 | [1-bit Bonsai 8B](/models/bonsai-8b) | Prism ML | 50% |
| 4 | [Ternary Bonsai 4B](/models/ternary-bonsai-4b) | Prism ML | 45.1% |
| 5 | [1-bit Bonsai 1.7B](/models/bonsai-1-7b) | Prism ML | 45.1% |
| 6 | [1-bit Bonsai 4B](/models/bonsai-4b) | Prism ML | 41.4% |

## FAQ

### What does MuSR measure?

A dataset for evaluating language models on multistep soft reasoning tasks specified in natural language narratives. Tests the ability to perform complex, structured reasoning.

### Which model scores highest on MuSR?

Ternary Bonsai 8B by Prism ML currently leads with a score of 56.2% on MuSR.

### How many models are evaluated on MuSR?

6 AI models have published results on MuSR in the BenchLM catalog.

### Does MuSR affect BenchLM's overall score?

Not directly. MuSR is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Compare Top Models on MuSR

- [Ternary Bonsai 8B vs Ternary Bonsai 1.7B](/compare/ternary-bonsai-1-7b-vs-ternary-bonsai-8b)
- [Ternary Bonsai 1.7B vs 1-bit Bonsai 8B](/compare/bonsai-8b-vs-ternary-bonsai-1-7b)
- [1-bit Bonsai 8B vs Ternary Bonsai 4B](/compare/bonsai-8b-vs-ternary-bonsai-4b)
- [Ternary Bonsai 4B vs 1-bit Bonsai 1.7B](/compare/bonsai-1-7b-vs-ternary-bonsai-4b)