Massive Multitask Language Understanding Professional (MMLU-Pro)
Data verified
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
Top models on MMLU-Pro — October 29, 2026
As of October 29, 2026, Qwen3.7 Max leads the MMLU-Pro leaderboard with 89.6% , followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%).
1Closed
Qwen3.7 Max
Alibaba
89.6%
Overall 62.32Context 1M
2Closed
Claude Opus 4.5
Anthropic
89.5%
Overall 53.35Context 200K
3Closed
Qwen3.7 Plus
Alibaba
88.5%
Overall 53.79Context 1M
80 modelsKnowledge6% of Knowledge reference weightRefreshingUpdated October 29, 2026
Leaderboard (80 models)
80 of 80 rows
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Qwen3.7 MaxAlibaba | 89.6% | Not reported | Closed |
| 2 | Claude Opus 4.5Anthropic | 89.5% | Not reported | Closed |
| 3 | Qwen3.7 PlusAlibaba | 88.5% | Not reported | Closed |
| 4 | Qwen3.6 PlusAlibaba | 88.5% | Not reported | Closed |
| 5 | Qwen3.5 397BAlibaba | 87.8% | Not reported | Open |
| 6 | DeepSeek V4 Pro 0813DeepSeek | 87.5% | Not reported | Open |
| 7 | Kimi K2.5 (Reasoning)Moonshot AI | 87.1% | Not reported | Closed |
| 8 | Kimi K2.5Moonshot AI | 87.1% | Not reported | Open |
| 9 | DeepSeek V4 Pro (High)DeepSeek | 87.1% | Not reported | Open |
| 10 | Nemotron 3 UltraNVIDIA | 86.8% | Not reported | Open |
| 11 | Qwen3.5-122B-A10BAlibaba | 86.7% | Not reported | Open |
| 12 | DeepSeek V4 Flash (High)DeepSeek | 86.4% | Not reported | Open |
| 13 | Solar Pro 4Upstage | 86.3% | Not reported | Closed |
| 14 | Qwen3.6-27BAlibaba | 86.2% | Not reported | Open |
| 15 | DeepSeek V4 Flash 0731DeepSeek | 86.2% | Not reported | Open |
| 16 | Solar Open 2Upstage | 86.2% | Not reported | Open |
| 17 | Qwen3.5-27BAlibaba | 86.1% | Not reported | Open |
| 18 | GLM-5Z.AI | 85.7% | Not reported | Open |
| 19 | Qwen3.5-35B-A3BAlibaba | 85.3% | Not reported | Open |
| 20 | Gemma 4 31BGoogle | 85.2% | Not reported | Open |
| 21 | Qwen3.6-35B-A3BAlibaba | 85.2% | Not reported | Open |
| 22 | DeepSeek-R1-0528DeepSeek | 85% | Not reported | Open |
| 23 | MAI-Thinking-1Microsoft | 85% | Not reported | Closed |
| 24 | MiMo-V2-FlashXiaomi | 84.9% | Not reported | Open |
| 25 | DeepSeek V3.1 (Reasoning)DeepSeek | 84.8% | Not reported | Open |
| 26 | Qwen3 235B 2507 (Reasoning)Alibaba | 84.4% | Not reported | Open |
| 27 | GLM-4.7Z.AI | 84.3% | Not reported | Open |
| 28 | DeepSeek-R1DeepSeek | 84% | Not reported | Open |
| 29 | Nemotron 3 Super 100BNVIDIA | 83.7% | Not reported | Open |
| 30 | DeepSeek V3.1DeepSeek | 83.7% | Not reported | Open |
| 31 | K-EXAONE 2.0LG AI Research | 83.5% | Not reported | Open |
| 32 | Qwen3 235B 2507Alibaba | 83% | Not reported | Open |
| 33 | DeepSeek V4 FlashDeepSeek | 83% | Not reported | Open |
| 34 | DeepSeek V4 ProDeepSeek | 82.9% | Not reported | Open |
| 35 | Gemma 4 26B A4BGoogle | 82.6% | Not reported | Open |
| 36 | Claude Opus 4.6Anthropic | 82% | Not reported | Closed |
| 37 | Exaone 4.0 32BLG AI Research | 81.8% | Not reported | Open |
| 38 | Sarvam 105BSarvam | 81.7% | Not reported | Open |
| 39 | Nemotron 3.5 Lightning 30B A3B NVFP4NVIDIA | 81.6% | Not reported | Open |
| 40 | Kimi K2Moonshot AI | 81.1% | Not reported | Closed |
| 41 | Sarvam 30BSarvam | 80% | Not reported | Open |
| 42 | Grok 3 [Beta]xAI | 79.9% | Not reported | Closed |
| 43 | LongCat-Flash-Lite-SparseMeituan | 79.2% | Not reported | Open |
| 44 | Claude Sonnet 4.6Anthropic | 79.2% | Not reported | Closed |
| 45 | Nemotron 3 Nano 30BNVIDIA | 78.3% | Not reported | Open |
| 46 | Granite 4.2 30BIBM | 77.6% | Not reported | Open |
| 47 | Nemotron 3 Nano Omni 30B A3BNVIDIA | 77.3% | Not reported | Open |
| 48 | Gemma 4 12BGoogle | 77.2% | Not reported | Open |
| 49 | Qwen2.5-MaxAlibaba | 76.1% | Not reported | Closed |
| 50 | Phi-4-reasoning-plusMicrosoft | 76% | Not reported | Open |
| 51 | Celeris-1Celeris | 75.9% | Not reported | Closed |
| 52 | DeepSeek V3DeepSeek | 75.9% | Not reported | Open |
| 53 | Grok 2xAI | 75.5% | Not reported | Closed |
| 54 | Phi-4-reasoningMicrosoft | 74.3% | Not reported | Open |
| 55 | ZAYA1-8BZyphra | 74.2% | Not reported | Open |
| 56 | Granite 4.2 8BIBM | 74.0% | Not reported | Open |
| 57 | DeepSeek V4 Pro BaseDeepSeek | 73.5% | Not reported | Open |
| 58 | Grok 2 minixAI | 72% | Not reported | Closed |
| 59 | MiniCPM5-2BOpenBMB | 70.8% | Not reported | Open |
| 60 | Gemma 4 E4BGoogle | 69.4% | Not reported | Open |
| 61 | Llama 3.3 70BMeta | 68.9% | Not reported | Open |
| 62 | Qwen2.5-VL-32BAlibaba | 68.8% | Not reported | Open |
| 63 | DeepSeek V4 Flash BaseDeepSeek | 68.3% | Not reported | Open |
| 64 | ZAYA1-74B-PreviewZyphra | 68.1% | Not reported | Open |
| 65 | Granite 4.2 3BIBM | 67.8% | Not reported | Open |
| 66 | Mistral Small 3.1Mistral | 66.8% | Not reported | Open |
| 67 | Llama 3.1 70BMeta | 66.4% | Not reported | Open |
| 68 | Claude 3.5 HaikuAnthropic | 65% | Not reported | Closed |
| 69 | Gemma 4 E2BGoogle | 60% | Not reported | Open |
| 70 | Soofi S 30B-A3BSoofi Project | 51.4% | Not reported | Open |
| 71 | MiniCPM5-1BOpenBMB | 48.9% | Not reported | Open |
| 72 | Llama 3.1 8BMeta | 48.3% | Not reported | Open |
| 73 | Phi-3.5-miniMicrosoft | 47.4% | Not reported | Open |
| 74 | Granite-4.0-1BIBM | 34.0% | Not reported | Open |
| 75 | Granite-4.0-H-1BIBM | 32.9% | Not reported | Open |
| 76 | LFM2.5-230MLiquidAI | 20.3% | Not reported | Open |
| 77 | LFM2.5-350MLiquidAI | 20.0% | Not reported | Open |
| 78 | LFM2.5-VL-450MLiquidAI | 19.3% | Not reported | Open |
| 79 | Granite-4.0-H-350MIBM | 14.4% | Not reported | Open |
| 80 | Granite-4.0-350MIBM | 12.1% | Not reported | Open |
AA MMLU-Pro
Stored separately from the core MMLU-Pro lane because Artificial Analysis controls the evaluation configuration.
15 of 15 rows
| Source rank | Source model or configuration | Score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Gemini 3 ProGoogle | 89.8% | Artificial AnalysisOriginal result | Not reported | Closed |
| 2 | Claude Opus 4.5 ThinkingAnthropic | 89.5% | Artificial AnalysisOriginal result | Not reported | Closed |
| 3 | Claude 4.1 Opus ThinkingAnthropic | 88.0% | Artificial AnalysisOriginal result | Not reported | Closed |
| 4 | GPT-5.2OpenAI | 87.4% | Artificial AnalysisOriginal result | Not reported | Closed |
| 5 | Claude 4 Opus ThinkingAnthropic | 87.3% | Artificial AnalysisOriginal result | Not reported | Closed |
| 6 | GPT-5 (high)OpenAI | 87.1% | Artificial AnalysisOriginal result | Not reported | Closed |
| 7 | GPT-5.1OpenAI | 87.0% | Artificial AnalysisOriginal result | Not reported | Closed |
| 8 | Grok 4xAI | 86.6% | Artificial AnalysisOriginal result | Not reported | Closed |
| 9 | Gemini 2.5 ProGoogle | 86.2% | Artificial AnalysisOriginal result | Not reported | Closed |
| 10 | GLM-4.7Z.AI | 85.6% | Artificial AnalysisOriginal result | Not reported | Open |
| 11 | Grok 4.1 Fast (Reasoning)xAI | 85.4% | Artificial AnalysisOriginal result | Not reported | Closed |
| 12 | o3OpenAI | 85.3% | Artificial AnalysisOriginal result | Not reported | Closed |
| 13 | DeepSeek V3.1 (Reasoning)DeepSeek | 85.1% | Artificial AnalysisOriginal result | Not reported | Open |
| 14 | Grok 4 Fast (Reasoning)xAI | 85.0% | Artificial AnalysisOriginal result | Not reported | Closed |
| 15 | DeepSeek-R1DeepSeek | 84.9% | Artificial AnalysisOriginal result | Not reported | Open |
MMLU-Pro (Arcee)
BenchLM stores this chart-specific MMLU-Pro row separately so it does not overwrite the standardized weighted MMLU-Pro benchmark values.
6 of 6 rows
| Source rank | Source model or configuration | Score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.6Anthropic | 89.1% | Arcee AIOriginal result | Not reported | Closed |
| 2 | Kimi K2.5Moonshot AI | 87.1% | Arcee AIOriginal result | Not reported | Open |
| 3 | GLM-5Z.AI | 85.8% | Arcee AIOriginal result | Not reported | Open |
| 4 | Trinity-Large-ThinkingArcee AI | 83.4% | Arcee AIOriginal result | Not reported | Open |
| 5 | MiniMax M2.7MiniMax | 80.8% | Arcee AIOriginal result | Not reported | Open |
| 6 | Trinity-Large-PreviewArcee AI | 75.2% | Arcee AIOriginal result | Not reported | Open |
MMLU-Pro (Vals)
BenchLM mirrors this Vals AI leaderboard for reference only. It is excluded from BenchLM overall and category rankings.
138 of 138 rows
| Source rank | Source model or configuration | Vals MMLU-Pro mirror score | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Claude Fable 5.1Anthropic | 92.4% | Vals AIOriginal result | Not reported | Closed |
| 2 | Claude Opus 5Anthropic | 91.6% | Vals AIOriginal result | Not reported | Closed |
| 3 | Claude Fable 5Anthropic | 91.5% | Vals AIOriginal result | Not reported | Closed |
| 4 | Gemini 3.1 Pro PreviewGoogle | 91.0% | high reasoningOriginal result | Not reported | Unknown |
| 5 | Gemini 3.8 FlashGoogle | 90.2% | high reasoningOriginal result | Not reported | Closed |
| 6 | Gemini 3.7 FlashGoogle | 90.1% | high reasoningOriginal result | Not reported | Closed |
| 7 | Gemini 3 Pro PreviewGoogle | 90.1% | Vals AIOriginal result | Not reported | Unknown |
| 8 | Claude Opus 4.7Anthropic | 89.9% | Vals AIOriginal result | Not reported | Closed |
| 9 | Claude Opus 4.8Anthropic | 89.6% | Vals AIOriginal result | Not reported | Closed |
| 10 | Gemini 3.5 FlashGoogle | 89.5% | high reasoningOriginal result | Not reported | Closed |
| 11 | Grok 4.6xAI | 89.4% | high reasoningOriginal result | Not reported | Closed |
| 12 | Qwen3.7 MaxAlibaba | 89.3% | Vals AIOriginal result | Not reported | Closed |
| 13 | Gemini 3.6 FlashGoogle | 89.3% | high reasoningOriginal result | Not reported | Closed |
| 14 | Grok 4.5xAI | 89.2% | high reasoningOriginal result | Not reported | Closed |
| 15 | Claude Opus 4.6 (Adaptive)Anthropic | 89.1% | Vals AIOriginal result | Not reported | Closed |
| 16 | GPT-5.6 SolOpenAI | 89.1% | max reasoningOriginal result | Not reported | Closed |
| 17 | Muse Spark 1.1Meta | 88.7% | xhigh reasoningOriginal result | Not reported | Closed |
| 18 | Qwen3.8 MaxAlibaba | 88.6% | Vals AIOriginal result | Not reported | Open |
| 19 | Gemini 3 Flash PreviewGoogle | 88.6% | high reasoningOriginal result | Not reported | Unknown |
| 20 | Muse Spark 1.2Meta | 88.3% | xhigh reasoningOriginal result | Not reported | Closed |
| 21 | GPT-5.5OpenAI | 88.1% | xhigh reasoningOriginal result | Not reported | Closed |
| 22 | Kimi K3Moonshot AI | 88.0% | max reasoningOriginal result | Not reported | Open |
| 23 | Claude Opus 4.1 20250805 ThinkingAnthropic | 87.9% | Vals AIOriginal result | Not reported | Unknown |
| 24 | Qwen3.6 PlusAlibaba | 87.7% | Vals AIOriginal result | Not reported | Closed |
| 25 | Kimi K2.6Moonshot AI | 87.6% | Vals AIOriginal result | Not reported | Open |
| 26 | Claude Sonnet 5Anthropic | 87.5% | Vals AIOriginal result | Not reported | Closed |
| 27 | GPT-5.4OpenAI | 87.5% | xhigh reasoningOriginal result | Not reported | Closed |
| 28 | Claude Sonnet 4.5 ThinkingAnthropic | 87.4% | Vals AIOriginal result | Not reported | Closed |
| 29 | Claude Sonnet 4.6Anthropic | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 30 | Muse SparkMeta | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 31 | Claude Opus 4.5 ThinkingAnthropic | 87.3% | Vals AIOriginal result | Not reported | Closed |
| 32 | DeepSeek V4 Pro 0813DeepSeek | 87.2% | max reasoningOriginal result | Not reported | Open |
| 33 | Claude Opus 4.1Anthropic | 87.2% | Vals AIOriginal result | Not reported | Unknown |
| 34 | Qwen3.5 Plus ThinkingAlibaba | 87.2% | Vals AIOriginal result | Not reported | Unknown |
| 35 | MiniMax M2.1MiniMax | 87.0% | Vals AIOriginal result | Not reported | Unknown |
| 36 | DeepSeek V4 Pro 0813DeepSeek | 87.0% | max reasoningOriginal result | Not reported | Open |
| 37 | GLM-5.1Z.AI | 86.9% | Vals AIOriginal result | Not reported | Open |
| 38 | GLM-5.3Z.AI | 86.8% | max reasoningOriginal result | Not reported | Open |
| 39 | GLM-5.2Z.AI | 86.7% | Vals AIOriginal result | Not reported | Open |
| 40 | GPT-5.6 TerraOpenAI | 86.7% | xhigh reasoningOriginal result | Not reported | Closed |
| 41 | GPT-5OpenAI | 86.5% | high reasoningOriginal result | Not reported | Unknown |
| 42 | GPT-5.1OpenAI | 86.4% | high reasoningOriginal result | Not reported | Closed |
| 43 | InklingThinking Machines Lab | 86.3% | 0.99 reasoningOriginal result | Not reported | Open |
| 44 | Grok 4.20 0309 ReasoningxAI | 86.3% | Vals AIOriginal result | Not reported | Unknown |
| 45 | Gemini 3.1 Flash Lite PreviewGoogle | 86.2% | Vals AIOriginal result | Not reported | Unknown |
| 46 | GPT-5.2OpenAI | 86.2% | xhigh reasoningOriginal result | Not reported | Closed |
| 47 | DeepSeek V4 Flash 0731DeepSeek | 86.2% | high reasoningOriginal result | Not reported | Open |
| 48 | Claude Opus 4Anthropic | 86.2% | Vals AIOriginal result | Not reported | Unknown |
| 49 | GLM-5.3-FlashZ.AI | 86.1% | max reasoningOriginal result | Not reported | Open |
| 50 | GPT-5.6 LunaOpenAI | 86.0% | max reasoningOriginal result | Not reported | Closed |
| 51 | GLM 5 ThinkingZhipu AI | 86.0% | Vals AIOriginal result | Not reported | Unknown |
| 52 | Kimi K2.5 ThinkingMoonshot AI | 85.9% | Vals AIOriginal result | Not reported | Unknown |
| 53 | Grok 4.3xAI | 85.8% | high reasoningOriginal result | Not reported | Closed |
| 54 | Gemini 3.5 Flash-LiteGoogle | 85.8% | high reasoningOriginal result | Not reported | Closed |
| 55 | Nemotron 3 Ultra 550b A55bNvidia | 85.8% | Vals AIOriginal result | Not reported | Unknown |
| 56 | o3OpenAI | 85.6% | high reasoningOriginal result | Not reported | Closed |
| 57 | Claude Opus 4.5Anthropic | 85.6% | Vals AIOriginal result | Not reported | Closed |
| 58 | Inkling-SmallThinking Machines Lab | 85.6% | 0.99 reasoningOriginal result | Not reported | Open |
| 59 | Grok 4 0709xAI | 85.3% | Vals AIOriginal result | Not reported | Unknown |
| 60 | Qwen3 MaxAlibaba | 85.0% | Vals AIOriginal result | Not reported | Closed |
| 61 | DeepSeek V3p2 ThinkingFireworks AI | 84.9% | Vals AIOriginal result | Not reported | Unknown |
| 62 | MiMo-V2.5-ProXiaomi | 84.6% | Vals AIOriginal result | Not reported | Closed |
| 63 | GPT-5.4 miniOpenAI | 84.6% | xhigh reasoningOriginal result | Not reported | Closed |
| 64 | Qwen3 MaxAlibaba | 84.4% | Vals AIOriginal result | Not reported | Closed |
| 65 | Qwen3.8-27BAlibaba | 84.3% | xhigh reasoningOriginal result | Not reported | Open |
| 66 | MiniMax M3MiniMax | 84.2% | Vals AIOriginal result | Not reported | Open |
| 67 | Grok 4.1 Fast (Reasoning)xAI | 84.2% | Vals AIOriginal result | Not reported | Closed |
| 68 | Qwen3.5 FlashAlibaba | 84.1% | Vals AIOriginal result | Not reported | Closed |
| 69 | Gemini 2.5 Pro Exp 03 25Google | 84.1% | Vals AIOriginal result | Not reported | Unknown |
| 70 | Claude Sonnet 4 20250514 ThinkingAnthropic | 83.9% | Vals AIOriginal result | Not reported | Unknown |
| 71 | Gemini 2.5 Flash Preview 09 2025Google | 83.7% | Vals AIOriginal result | Not reported | Unknown |
| 72 | Gemini 2.5 Flash Preview 09 2025 ThinkingGoogle | 83.7% | Vals AIOriginal result | Not reported | Unknown |
| 73 | Qwen3 Max PreviewAlibaba | 83.5% | Vals AIOriginal result | Not reported | Unknown |
| 74 | o1OpenAI | 83.5% | high reasoningOriginal result | Not reported | Closed |
| 75 | DeepSeek-R1DeepSeek | 83.2% | Vals AIOriginal result | Not reported | Open |
| 76 | DeepSeek V3p2Fireworks AI | 83.1% | Vals AIOriginal result | Not reported | Unknown |
| 77 | MiMo-V2.5Xiaomi | 82.9% | Vals AIOriginal result | Not reported | Closed |
| 78 | GLM-4.7Z.AI | 82.7% | Vals AIOriginal result | Not reported | Open |
| 79 | Claude 3.7 Sonnet ThinkingAnthropic | 82.7% | Vals AIOriginal result | Not reported | Closed |
| 80 | GPT-5 miniOpenAI | 82.2% | high reasoningOriginal result | Not reported | Closed |
| 81 | GLM-4.6Z.AI | 82.2% | Vals AIOriginal result | Not reported | Open |
| 82 | Ling 3.0 Flash 2607Ant | 82.0% | Vals AIOriginal result | Not reported | Unknown |
| 83 | Grok 3 Mini Fast High ReasoningxAI | 81.4% | high reasoningOriginal result | Not reported | Unknown |
| 84 | Qwen3 235B-A22BAlibaba | 81.2% | Vals AIOriginal result | Not reported | Open |
| 85 | GLM-4.5Z.AI | 81.2% | Vals AIOriginal result | Not reported | Closed |
| 86 | Kimi K2 ThinkingMoonshot AI | 81.1% | Vals AIOriginal result | Not reported | Unknown |
| 87 | Claude 3.7 SonnetAnthropic | 80.7% | Vals AIOriginal result | Not reported | Closed |
| 88 | o4-miniOpenAI | 80.6% | high reasoningOriginal result | Not reported | Closed |
| 89 | GPT-4.1OpenAI | 80.5% | high reasoningOriginal result | Not reported | Closed |
| 90 | MiniMax M2.7MiniMax | 80.4% | Vals AIOriginal result | Not reported | Open |
| 91 | MiniMax M2.5MiniMax | 80.1% | Vals AIOriginal result | Not reported | Closed |
| 92 | Grok 3 Mini Fast Low ReasoningxAI | 80.0% | low reasoningOriginal result | Not reported | Unknown |
| 93 | Grok 3xAI | 79.9% | Vals AIOriginal result | Not reported | Unknown |
| 94 | Mistral Large 2512Mistral AI | 79.8% | Vals AIOriginal result | Not reported | Unknown |
| 95 | Grok 4 Fast (Reasoning)xAI | 79.7% | Vals AIOriginal result | Not reported | Closed |
| 96 | DeepSeek V3 0324DeepSeek | 79.5% | Vals AIOriginal result | Not reported | Open |
| 97 | Claude Sonnet 4Anthropic | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 98 | Llama4 Maverick Instruct BasicFireworks AI | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 99 | Moonshotai Kimi K2 InstructTogether AI | 79.4% | Vals AIOriginal result | Not reported | Unknown |
| 100 | GPT-OSS 120BOpenAI | 79.2% | Vals AIOriginal result | Not reported | Open |
| 101 | Gemini 2.5 Flash Lite Preview 09 2025 ThinkingGoogle | 79.1% | Vals AIOriginal result | Not reported | Unknown |
| 102 | Claude Haiku 4.5 ThinkingAnthropic | 78.7% | Vals AIOriginal result | Not reported | Closed |
| 103 | o3-miniOpenAI | 78.7% | high reasoningOriginal result | Not reported | Closed |
| 104 | Gemini 2.5 Flash Lite Preview 09 2025Google | 78.6% | Vals AIOriginal result | Not reported | Unknown |
| 105 | Claude 3.5 SonnetAnthropic | 78.4% | Vals AIOriginal result | Not reported | Closed |
| 106 | Gemini 2.0 Flash 001Google | 77.4% | Vals AIOriginal result | Not reported | Unknown |
| 107 | GPT-4.1 miniOpenAI | 77.2% | high reasoningOriginal result | Not reported | Closed |
| 108 | GPT-5.4 nanoOpenAI | 77.2% | high reasoningOriginal result | Not reported | Closed |
| 109 | GPT-5 nanoOpenAI | 76.1% | high reasoningOriginal result | Not reported | Closed |
| 110 | Grok 2 1212xAI | 75.5% | Vals AIOriginal result | Not reported | Unknown |
| 111 | Mistral Medium 3.5Mistral AI | 75.3% | high reasoningOriginal result | Not reported | Unknown |
| 112 | Gemini 1.5 Pro 002Google | 75.3% | Vals AIOriginal result | Not reported | Unknown |
| 113 | Mistral Medium 2505Mistral AI | 75.3% | Vals AIOriginal result | Not reported | Unknown |
| 114 | Grok 4.1 Fast Non ReasoningxAI | 75.2% | Vals AIOriginal result | Not reported | Unknown |
| 115 | GPT-4oOpenAI | 74.1% | high reasoningOriginal result | Not reported | Closed |
| 116 | DeepSeek V3DeepSeek | 73.8% | Vals AIOriginal result | Not reported | Open |
| 117 | GPT-4oOpenAI | 72.6% | high reasoningOriginal result | Not reported | Closed |
| 118 | GPT-OSS 20BOpenAI | 71.6% | Vals AIOriginal result | Not reported | Open |
| 119 | Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561Together AI | 70.8% | Vals AIOriginal result | Not reported | Unknown |
| 120 | Grok 4 Fast Non ReasoningxAI | 70.3% | Vals AIOriginal result | Not reported | Unknown |
| 121 | Meta Llama Llama 3.3 70B Instruct TurboTogether AI | 69.9% | Vals AIOriginal result | Not reported | Unknown |
| 122 | Mistral Large 2411Mistral AI | 69.7% | Vals AIOriginal result | Not reported | Unknown |
| 123 | Meta Llama Llama 4 Scout 17B 16E InstructTogether AI | 69.6% | Vals AIOriginal result | Not reported | Unknown |
| 124 | Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561 ThinkingTogether AI | 69.6% | Vals AIOriginal result | Not reported | Unknown |
| 125 | Command A 03 2025Cohere | 69.2% | Vals AIOriginal result | Not reported | Unknown |
| 126 | Laguna XS.2Poolside | 69.0% | Vals AIOriginal result | Not reported | Open |
| 127 | Laguna M.1Poolside | 68.8% | Vals AIOriginal result | Not reported | Closed |
| 128 | Magistral Medium 2509Mistral AI | 68.7% | Vals AIOriginal result | Not reported | Unknown |
| 129 | Mistral Small 2503Mistral AI | 66.0% | Vals AIOriginal result | Not reported | Unknown |
| 130 | Gemini 1.5 Flash 002Google | 65.6% | Vals AIOriginal result | Not reported | Unknown |
| 131 | Mistral Small 2402Mistral AI | 64.4% | Vals AIOriginal result | Not reported | Unknown |
| 132 | Claude 3.5 HaikuAnthropic | 64.1% | Vals AIOriginal result | Not reported | Closed |
| 133 | GPT-4.1 nanoOpenAI | 63.5% | high reasoningOriginal result | Not reported | Closed |
| 134 | GPT-4o miniOpenAI | 62.7% | high reasoningOriginal result | Not reported | Closed |
| 135 | Magistral Small 2509Mistral AI | 62.1% | Vals AIOriginal result | Not reported | Unknown |
| 136 | Jamba Large 1.6AI21 Labs | 49.8% | Vals AIOriginal result | Not reported | Unknown |
| 137 | Command R PlusCohere | 44.0% | Vals AIOriginal result | Not reported | Unknown |
| 138 | Jamba Mini 1.6AI21 Labs | 30.3% | Vals AIOriginal result | Not reported | Unknown |
MMLU-Pro (Kaggle)
Native proportions are displayed as percentages. Source profiles, task versions, evaluation dates, and symmetric intervals stay attached to each row. Missing results are not zero. How Kaggle Benchmarks reruns tests.
64 of 64 rows
| Source rank | Source model or configuration | Accuracy (%) | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | 93.2% | Task version 12; Evaluated 2026-10-09Original result | Not reported | Unknown |
| 2 | Claude Opus 5 | 91.7% | Task version 12; Evaluated 2026-07-27Original result | Not reported | Unknown |
| 3 | Gemini 3.1 Pro Preview | 91.3% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 4 | Gemini 3.8 Flash | 90.6% | Task version 12; Evaluated 2026-09-04Original result | Not reported | Unknown |
| 5 | Gemini 3.7 Flash | 90.3% | Task version 12; Evaluated 2026-08-28Original result | Not reported | Unknown |
| 6 | GPT-6 Astra | 89.8% | Task version 12; Evaluated 2026-10-21Original result | Not reported | Unknown |
| 7 | Gemini 3.6 Flash | 89.8% | Task version 12; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 8 | Gemini 3.5 Flash | 89.6% | Task version 12; Evaluated 2026-06-01Original result | Not reported | Unknown |
| 9 | Grok 4.5 | 89.5% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 10 | GPT-6.1 Sol | 89.4% | Task version 12; Evaluated 2026-10-05Original result | Not reported | Unknown |
| 11 | Gemini 3 Flash Preview | 89.4% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 12 | Claude Opus 4.8 | 88.7% | Task version 12; Evaluated 2026-05-29Original result | Not reported | Unknown |
| 13 | GPT-6 Sol | 88.1% | Task version 12; Evaluated 2026-10-01Original result | Not reported | Unknown |
| 14 | Claude Opus 4.1 | 87.9% | Task version 12; Evaluated 2025-08-21Original result | Not reported | Unknown |
| 15 | GPT-5 | 87.1% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 16 | Claude Opus 4 | 86.9% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 17 | Gemma 4 31B | 86.7% | Task version 12; Evaluated 2026-06-02Original result | Not reported | Unknown |
| 18 | Gemini 2.5 Pro Preview | 86.7% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 19 | Grok 4 | 86.6% | Task version 12; Evaluated 2025-09-04Original result | Not reported | Unknown |
| 20 | o3 | 86.4% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 21 | GPT-5.6 Terra | 86.3% | Task version 12; Evaluated 2026-07-20Original result | Not reported | Unknown |
| 22 | Deepseek V4.1 Flash | 86.3% | Task version 12; Evaluated 2026-10-20Original result | Not reported | Unknown |
| 23 | Claude Haiku 5.5 | 85.8% | Task version 12; Evaluated 2026-10-09Original result | Not reported | Unknown |
| 24 | GPT-6 Luna | 85.4% | Task version 12; Evaluated 2026-10-05Original result | Not reported | Unknown |
| 25 | Gemini 3.1 Flash-Lite Preview | 84.5% | Task version 12; Evaluation date not reportedOriginal result | Not reported | Unknown |
| 26 | GPT-5.6 Luna | 84.4% | Task version 12; Evaluated 2026-07-20Original result | Not reported | Unknown |
| 27 | GPT-5.6 Sol | 84.4% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 28 | DeepSeek-R1 | 84.0% | Task version 12; Evaluated 2025-07-02Original result | Not reported | Unknown |
| 29 | o1 | 83.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 30 | Claude Sonnet 4 | 83.7% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 31 | Gemini 3.5 Flash-Lite | 83.2% | Task version 12; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 32 | deepseek-v3.1 | 83.1% | Task version 12; Evaluated 2025-08-27Original result | Not reported | Unknown |
| 33 | Gemini 2.5 Flash Preview | 82.9% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 34 | Grok 3 Mini | 81.2% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 35 | GPT-4.1 | 81.0% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 36 | gpt-oss-120b | 78.5% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 37 | DeepSeek-V3 | 78.2% | Task version 12; Evaluated 2025-07-09Original result | Not reported | Unknown |
| 38 | o4 mini | 77.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 39 | o3 mini | 77.7% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 40 | Claude 3.5 Sonnet | 77.6% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 41 | Grok 3 | 77.2% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 42 | Gemini 2.0 Flash | 76.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 43 | Grok 2 | 74.1% | Task version 12; Evaluated 2025-06-23Original result | Not reported | Unknown |
| 44 | GPT-4o | 73.0% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 45 | o1 mini | 72.0% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 46 | Claude 3.7 Sonnet | 71.9% | Task version 12; Evaluated 2025-07-09Original result | Not reported | Unknown |
| 47 | Gemini 1.5 Pro | 70.1% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 48 | Mistral Large 2 | 70.0% | Task version 12; Evaluated 2025-07-07Original result | Not reported | Unknown |
| 49 | gpt-oss-20b | 67.1% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 50 | Gemma 3 27B | 66.2% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 51 | Qwen 3 235B A22B Instruct 2506 | 66.1% | Task version 12; Evaluated 2026-07-21Original result | Not reported | Unknown |
| 52 | Gemini 1.5 Flash | 64.7% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 53 | GPT-4o mini | 62.4% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 54 | Claude 3.5 Haiku | 61.3% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 55 | Qwen 3 235B A22B Thinking | 58.4% | Task version 12; Evaluated 2025-08-20Original result | Not reported | Unknown |
| 56 | Gemma 3 12B | 58.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 57 | Granite 4.0 Small | 57.4% | Task version 12; Evaluated 2025-09-30Original result | Not reported | Unknown |
| 58 | Mixtral 8x22B | 55.1% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 59 | Codestral | 51.6% | Task version 12; Evaluated 2025-07-07Original result | Not reported | Unknown |
| 60 | Gemini 1.5 Flash 8B | 48.2% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 61 | GPT-3.5 Turbo | 46.9% | Task version 12; Evaluated 2025-06-13Original result | Not reported | Unknown |
| 62 | Ministral 8B | 42.8% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
| 63 | Gemma 3 4B | 41.1% | Task version 12; Evaluated 2025-06-12Original result | Not reported | Unknown |
| 64 | Ministral 3B | 38.3% | Task version 12; Evaluated 2025-07-10Original result | Not reported | Unknown |
According to BenchLM.ai, Qwen3.7 Max leads the MMLU-Pro benchmark with a score of 89.6%, followed by Claude Opus 4.5 (89.5%) and Qwen3.7 Plus (88.5%). The top models are clustered within 1.1 points, suggesting this benchmark is nearing saturation for frontier models.
80 models have been evaluated on MMLU-Pro. The benchmark falls in the Knowledge category. BenchAlign v5.8 gives MMLU-Pro 6% of the Knowledge reference weight, so it moves the Knowledge leaderboard and the overall ranking. Reference weights are relative weights in the calibrated model, not fixed shares of a score.
About MMLU-Pro
Year
2024
Tasks
Multiple subjects
Format
10-way multiple choice
Difficulty
Professional level
MMLU-Pro increases the number of choices from 4 to 10 and integrates more reasoning-focused problems, reducing the chance of correct guessing and better evaluating true understanding. It serves as a more robust discriminator of model capabilities.
Freshness and provenance
Version
MMLU-Pro
Refresh cadence
Static
Staleness state
Refreshing
Question availability
Public benchmark set
Refreshing
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MMLU-Pro measure?
An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.
Which model scores highest on MMLU-Pro?
Qwen3.7 Max by Alibaba currently leads with a score of 89.6% on MMLU-Pro.
How many models are evaluated on MMLU-Pro?
80 AI models have published results on MMLU-Pro in the BenchLM catalog.
Compare top models on MMLU-Pro
Last updated: October 29, 2026 · BenchLM version MMLU-Pro
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.