Massive Multitask Language Understanding (MMLU)
We show this table for reference; we do not rank on it.
A comprehensive multiple-choice question answering test covering 57 tasks including elementary mathematics, US history, computer science, law, and more. Tests knowledge across diverse academic subjects from high school to professional level.
Benchmark score on MMLU — October 17, 2026
We compile the MMLU rows from provider self-reports and secondary reports. o1 leads the table at 91.8%, followed by DeepSeek-R1 (90.8%) and GPT-4.5 (90.8%). We do not use these results to rank models overall.
o1
OpenAI
DeepSeek-R1
DeepSeek
GPT-4.5
OpenAI
34 modelsKnowledgeMMLU is displayed for reference and does not enter the active public ranking.StaleSaturatedDisplay onlyUpdated October 17, 2026
Benchmark score table (34 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | o1OpenAI | 91.8% | Not reported | Closed |
| 2 | DeepSeek-R1DeepSeek | 90.8% | Not reported | Open |
| 3 | GPT-4.5OpenAI | 90.8% | Not reported | Closed |
| 4 | Sarvam 105BSarvam | 90.6% | Not reported | Open |
| 5 | GPT-4.1OpenAI | 90.2% | Not reported | Closed |
| 6 | DeepSeek V4 Pro BaseDeepSeek | 90.1% | Not reported | Open |
| 7 | o4-miniOpenAI | 90% | Not reported | Closed |
| 8 | GPT-OSS 120BOpenAI | 90% | Not reported | Open |
| 9 | Kimi K2Moonshot AI | 89.5% | Not reported | Closed |
| 10 | DeepSeek V4 Flash BaseDeepSeek | 88.7% | Not reported | Open |
| 11 | Grok 2xAI | 87.5% | Not reported | Closed |
| 12 | GPT-4.1 miniOpenAI | 87.5% | Not reported | Closed |
| 13 | Trinity-Large-PreviewArcee AI | 87.2% | Not reported | Open |
| 14 | o3-miniOpenAI | 86.9% | Not reported | Closed |
| 15 | Grok 2 minixAI | 86.2% | Not reported | Closed |
| 16 | Llama 3.3 70BMeta | 86% | Not reported | Open |
| 17 | Llama 3.1 70BMeta | 86% | Not reported | Open |
| 18 | LongCat-Flash-Lite-SparseMeituan | 85.3% | Not reported | Open |
| 19 | GPT-OSS 20BOpenAI | 85.3% | Not reported | Open |
| 20 | o1-miniOpenAI | 85.2% | Not reported | Closed |
| 21 | Sarvam 30BSarvam | 85.1% | Not reported | Open |
| 22 | Phi-4Microsoft | 84.8% | Not reported | Open |
| 23 | GPT-4o miniOpenAI | 82% | Not reported | Closed |
| 24 | Mistral Small 3.1Mistral | 80.6% | Not reported | Open |
| 25 | Claude 3.5 HaikuAnthropic | 80.3% | Not reported | Closed |
| 26 | GPT-4.1 nanoOpenAI | 80.1% | Not reported | Closed |
| 27 | Mixtral 8x22B Instruct v0.1Mistral | 77.8% | Not reported | Open |
| 28 | Gemma 2 27BGoogle | 76.2% | Not reported | Open |
| 29 | Llama 3.1 8BMeta | 73% | Not reported | Open |
| 30 | Phi-3.5-miniMicrosoft | 69% | Not reported | Open |
| 31 | Granite-4.0-H-1BIBM | 59.7% | Not reported | Open |
| 32 | Granite-4.0-1BIBM | 59.4% | Not reported | Open |
| 33 | Granite-4.0-H-350MIBM | 36.2% | Not reported | Open |
| 34 | Granite-4.0-350MIBM | 35.0% | Not reported | Open |
How to read this leaderboard
Original MMLU is a four-choice benchmark covering 57 academic and professional tasks. Its subject mix ranges from elementary material to advanced domains, but it is now a saturated historical reference rather than a strong separator of current frontier models.
Variant check: MMLU-Pro is a separate benchmark with more than 12,000 questions across 14 domains and ten choices per question. Its results are not interchangeable with scores stored under the original MMLU key.
Use this table as a source receipt for original MMLU results. It does not expose a complete prompt and scoring setup for every row, so consult the original provider report before comparing results. Do not transfer MMLU-Pro's design or scores into this lane.
Operator receipt: 34 sourced rows are currently displayable on this page; the leading published row is o1 at 91.8%.
Provenance: At the September 22, 2026 editorial check, the displayed values came from provider reports in the source ledger. The original repository's legacy leaderboard did not supply that table.
Source freshness: The original paper was submitted on September 7, 2020 and published at ICLR 2021. The MMLU-Pro paper was last revised on November 6, 2024. Those source dates are separate from this page's data refresh date.
Honest limit: MMLU is display-only here because the lane is stale and saturated. Small differences do not establish a current model winner without a controlled, like-for-like evaluation on a benchmark that still separates the systems being considered.
Among the reported MMLU rows, o1 is first at 91.8%. The third row is 1.0 points behind. The broader top-10 range is 3.1 points, so many of the published results sit in a relatively narrow band.
34 models have been evaluated on MMLU. The benchmark falls in the Knowledge category. MMLU is displayed for reference and does not enter the active public ranking.
About MMLU
Year
2020
Tasks
57 subjects
Format
Multiple choice questions
Difficulty
Elementary to professional level
MMLU evaluates models on 57 subjects spanning humanities, social sciences, STEM, and other areas. Questions range from elementary to advanced professional level, making it a comprehensive test of world knowledge and reasoning ability.
Freshness and provenance
Version
MMLU
Refresh cadence
Static
Staleness state
Stale
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MMLU measure?
Original MMLU tests knowledge and reasoning across 57 academic and professional tasks. Each question has four answer choices. The subjects range from elementary material to advanced professional domains, but the public lane is now saturated enough that BenchLM keeps these sourced rows as a display-only historical reference.
Is MMLU-Pro the same benchmark as MMLU?
MMLU-Pro is a separate benchmark, not a directly interchangeable score update. It contains more than 12,000 questions across 14 domains, expands each question from four choices to ten, and removes some noisy or trivial original items. Compare models within one named benchmark and evaluation setup.
Does MMLU affect the current public ranking?
No. The MMLU table is display-only and does not enter the active public ranking. Its rows remain useful as source receipts for historically important evaluations, but small differences should not decide a current model choice without a controlled, like-for-like run on a more discriminating benchmark.
Compare top models on MMLU
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.