Skip to main content
Radar

Five or fewer confirmed AI changes, with original sources, on mornings when something changed.A free source-linked morning brief.

Start free brief
BenchLM
Advertise

Benchmark profile

Massive Multitask Language Understanding Professional (MMLU-Pro)

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

Data verified 24 confirmed releases in the last 30 daysStart free brief

Top models on MMLU-Pro — August 12, 2026

As of August 12, 2026, Qwen3.7 Max leads the MMLU-Pro leaderboard with 89.6% , followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%).

1Closed

Qwen3.7 Max

Alibaba

qwen3-7-max

89.6%
Overall 71.52Context 1M
2Closed

Claude Opus 4.5

Anthropic

claude-opus-4-5

89.5%
Overall 63.52Context 200K
3Closed

Qwen3.7 Plus

Alibaba

qwen3-7-plus

88.5%
Overall 65.8Context 1M
44 modelsKnowledge30% of category scoreRefreshingUpdated August 12, 2026

Leaderboard (44 models)

Score
1
Qwen3.7 MaxAlibaba · Closed
89.6%
2
Claude Opus 4.5Anthropic · Closed
89.5%
3
Qwen3.7 PlusAlibaba · Closed
88.5%
4
Qwen3.6 PlusAlibaba · Closed
88.5%
5
Qwen3.5 397BAlibaba · Open weight
87.8%
6
DeepSeek V4 Pro (Max)DeepSeek · Open weight
87.5%
7
DeepSeek V4 Pro (High)DeepSeek · Open weight
87.1%
8
Kimi K2.5Moonshot AI · Open weight
87.1%
9
Kimi K2.5 (Reasoning)Moonshot AI · Closed
87.1%
10
Nemotron 3 UltraNVIDIA · Open weight
86.8%
11
Qwen3.5-122B-A10BAlibaba · Open weight
86.7%
12
DeepSeek V4 Flash (High)DeepSeek · Closed
86.4%
13
Qwen3.6-27BAlibaba · Open weight
86.2%
14
DeepSeek V4 Flash (Max)DeepSeek · Closed
86.2%
15
Qwen3.5-27BAlibaba · Open weight
86.1%
16
GLM-5Z.AI · Open weight
85.7%
17
Qwen3.5-35B-A3BAlibaba · Open weight
85.3%
18
Qwen3.6-35B-A3BAlibaba · Open weight
85.2%
19
Gemma 4 31BGoogle · Open weight
85.2%
20
MAI-Thinking-1Microsoft · Closed
85%
21
MiMo-V2-FlashXiaomi · Open weight
84.9%
22
GLM-4.7Z.AI · Open weight
84.3%
23
Qwen3 235B 2507Alibaba · Open weight
83%
24
DeepSeek V4 FlashDeepSeek · Closed
83%
25
DeepSeek V4 ProDeepSeek · Open weight
82.9%
26
Gemma 4 26B A4BGoogle · Open weight
82.6%
27
Claude Opus 4.6Anthropic · Closed
82%
28
Exaone 4.0 32BLG AI Research · Open weight
81.8%
29
Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIA · Open weight
81.6%
30
Claude Sonnet 4.6Anthropic · Closed
79.2%
31
Nemotron 3 Nano Omni 30B A3BNVIDIA · Open weight
77.3%
32
Gemma 4 12BGoogle · Open weight
77.2%
33
Celeris-1Celeris · Closed
75.9%
34
DeepSeek V3DeepSeek · Open weight
75.9%
35
ZAYA1-8BZyphra · Open weight
74.2%
36
DeepSeek V4 Pro BaseDeepSeek · Open weight
73.5%
37
Gemma 4 E4BGoogle · Open weight
69.4%
38
DeepSeek V4 Flash BaseDeepSeek · Open weight
68.3%
39
ZAYA1-74B-PreviewZyphra · Open weight
68.1%
40
Gemma 4 E2BGoogle · Open weight
60%
41
Soofi S 30B-A3BSoofi Project · Open weight
51.4%
42
MiniCPM5-1BOpenBMB · Open weight
48.9%
43
LFM2.5-230MLiquidAI · Open weight
20.3%
44
LFM2.5-VL-450MLiquidAI · Open weight
19.3%

According to BenchLM.ai, Qwen3.7 Max leads the MMLU-Pro benchmark with a score of 89.6%, followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%). The top models are clustered within 1.1 points, suggesting this benchmark is nearing saturation for frontier models.

44 models have been evaluated on MMLU-Pro. The benchmark falls in the Knowledge category. This category carries a 12% weight in BenchLM.ai's overall scoring system. Within that category, MMLU-Pro contributes 30% of the category score, so strong performance here directly affects a model's overall ranking.

About MMLU-Pro

Year

2024

Tasks

Multiple subjects

Format

10-way multiple choice

Difficulty

Professional level

MMLU-Pro increases the number of choices from 4 to 10 and integrates more reasoning-focused problems, reducing the chance of correct guessing and better evaluating true understanding. It serves as a more robust discriminator of model capabilities.

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

BenchLM freshness & provenance

Version

MMLU-Pro

Refresh cadence

Static

Staleness state

Refreshing

Question availability

Public benchmark set

Refreshing

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

FAQ

What does MMLU-Pro measure?

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

Which model scores highest on MMLU-Pro?

Qwen3.7 Max by Alibaba currently leads with a score of 89.6% on MMLU-Pro.

How many models are evaluated on MMLU-Pro?

44 AI models have been evaluated on MMLU-Pro on BenchLM.

Compare Top Models on MMLU-Pro

Qwen3.7 Max vs Claude Opus 4.5Claude Opus 4.5 vs Qwen3.7 PlusQwen3.7 Plus vs Qwen3.6 PlusQwen3.6 Plus vs Qwen3.5 397B

Learn More

Read our explainer: MMLU-Pro benchmark deep dive
Last updated: August 12, 2026 · BenchLM version MMLU-Pro

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 2,000+ readers.

One email each week. Unsubscribe anytime.