MMLU-Pro, Vals AI run (MMLU-Pro (Vals))
Vals AI’s independent run of MMLU-Pro across fourteen academic subjects.
Data verified 31 confirmed releases in the last 30 daysSee provider release alertsHow BenchLM shows Vals MMLU-Pro mirror
BenchLM mirrors the public Vals AI Vals MMLU-Pro mirror leaderboard captured from https://www.vals.ai/benchmarks/mmlu_pro and updated by Vals on September 1, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.
Vals MMLU-Pro mirror is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.
Snapshot
Vals MMLU-Pro mirror score on MMLU-Pro (Vals) — September 1, 2026
We mirror the published vals mmlu-pro mirror score view for MMLU-Pro (Vals). Claude Fable 5.1 leads the public snapshot at 92.4%, followed by Claude Opus 5 (91.6%) and Claude Fable 5 (91.5%). We do not use these results to rank models overall.
Claude Fable 5.1
Anthropic
Claude Opus 5
Anthropic
Claude Fable 5
Anthropic
138 modelsKnowledge10% of category scoreCurrentUpdated September 1, 2026
Vals MMLU-Pro mirror score table (138 models)
ScoreThe published MMLU-Pro (Vals) snapshot places Claude Fable 5.1 first at 92.4%. The third row is 0.9 points behind. The broader top-10 range is 2.9 points, so many of the published results sit in a relatively narrow band.
138 models have been evaluated on MMLU-Pro (Vals). The benchmark falls in the Knowledge category. This category carries a 12% weight in BenchLM.ai's overall scoring system. Within that category, MMLU-Pro (Vals) contributes 10% of the category score, so strong performance here directly affects a model's overall ranking.
About MMLU-Pro (Vals)
Year
2026
Tasks
Academic multiple-choice questions
Format
Accuracy
Difficulty
Broad academic knowledge
BenchLM mirrors the Vals AI board on a dedicated key so a provider-run row on the canonical key is never overwritten. Vals publishes per-task accuracy with standard error, latency, and cost for every model it runs. Admitted as independent third-party evidence in methodology v5.5 (2026-09-04).
BenchLM freshness & provenance
Version
MMLU-Pro (Vals) 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
FAQ
What does MMLU-Pro (Vals) measure?
Vals AI’s independent run of MMLU-Pro across fourteen academic subjects.
Which model leads the published MMLU-Pro (Vals) snapshot?
Claude Fable 5.1 currently leads the published MMLU-Pro (Vals) snapshot with 92.4% vals mmlu-pro mirror score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on MMLU-Pro (Vals)?
The September 1, 2026 snapshot contains 138 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.