MATH-500 Problem Set (MATH-500)

We show this table for reference; we do not rank on it.

Data verified October 29, 2026

A curated subset of 500 problems from the MATH dataset, covering algebra, counting and probability, geometry, intermediate algebra, number theory, prealgebra, and precalculus.

Benchmark score on MATH-500 — October 29, 2026

We compile the MATH-500 rows from provider self-reports. Ternary Bonsai 2 27B leads the table at 98.8%, followed by OrcaCyber-Nano-1.0 (98.6%) and Sarvam 105B (98.6%). We do not use these results to rank models overall.

1Open weights

Ternary Bonsai 2 27B

Prism ML

98.8%

Overall 53.95Context 262K

2Open weights

OrcaCyber-Nano-1.0

OrcaRouter

98.6%

Context 262K

3Open weights

Sarvam 105B

Sarvam

98.6%

Overall 30.79Context 128K

27 modelsMathematicsStaleDisplay onlyUpdated October 29, 2026

Benchmark score table (27 models)

Benchmark score results for MATH-500
Rank	Model / configuration	Score	Parameters (B)	Open / closed
1	Ternary Bonsai 2 27BPrism ML	98.8%	Not reported	Open
2	OrcaCyber-Nano-1.0OrcaRouter	98.6%	Not reported	Open
3	Sarvam 105BSarvam	98.6%	Not reported	Open
4	Qwen3 235B-A22B (Reasoning)Alibaba	98%	Not reported	Open
5	QwQ-32BAlibaba	98%	Not reported	Open
6	o4-miniOpenAI	97.5%	Not reported	Closed
7	Kimi K2Moonshot AI	97.4%	Not reported	Closed
8	DeepSeek-R1DeepSeek	97.3%	Not reported	Open
9	Qwen3 32B (Reasoning)Alibaba	97.2%	Not reported	Open
10	Nemotron Ultra 253BNVIDIA	97%	Not reported	Open
11	Sarvam 30BSarvam	97%	Not reported	Open
12	MiniMax M1 80kMiniMax	96.8%	Not reported	Closed
13	Claude 3.7 Sonnet ThinkingAnthropic	96.2%	Not reported	Closed
14	LFM2.5-24B-A2BLiquidAI	96.0%	Not reported	Open
15	Magistral SmallMistral	95.9%	Not reported	Open
16	LongCat-Flash-Lite-SparseMeituan	95.8%	Not reported	Open
17	MiniCPM5-2BOpenBMB	94.6%	Not reported	Open
18	Magistral MediumMistral	94.3%	Not reported	Closed
19	MiniCPM5-1BOpenBMB	91.6%	Not reported	Open
20	Qwen3 235B-A22BAlibaba	91.2%	Not reported	Open
21	DeepSeek V3DeepSeek	90.2%	Not reported	Open
22	o1-miniOpenAI	90%	Not reported	Closed
23	LFM2.5-8B-A1BLiquidAI	88.8%	Not reported	Open
24	Qwen3 32BAlibaba	88.6%	Not reported	Open
25	Claude 3.7 SonnetAnthropic	82.2%	Not reported	Closed
26	Kanana-2 1.3B InstructKakao	61.4%	Not reported	Open
27	Kanana-2 3B InstructKakao	61.2%	Not reported	Open

AA MATH-500

Stored separately from the core MATH-500 lane because Artificial Analysis controls the evaluation configuration.

AA MATH-500; source configurations remain separate from the canonical benchmark table.
Source rank	Source model or configuration	Score	Source setup	Parameters (B)	Open / closed
1	GPT-5 (high)OpenAI	99.4%	Artificial AnalysisOriginal result	Not reported	Closed
2	o3OpenAI	99.2%	Artificial AnalysisOriginal result	Not reported	Closed
3	Grok 4xAI	99.0%	Artificial AnalysisOriginal result	Not reported	Closed
4	o4-miniOpenAI	98.9%	Artificial AnalysisOriginal result	Not reported	Closed
5	DeepSeek-R1DeepSeek	98.3%	Artificial AnalysisOriginal result	Not reported	Open
6	Claude 4 Opus ThinkingAnthropic	98.2%	Artificial AnalysisOriginal result	Not reported	Closed
7	MiniMax M1 80kMiniMax	98.0%	Artificial AnalysisOriginal result	Not reported	Closed
8	Kimi K2Moonshot AI	97.1%	Artificial AnalysisOriginal result	Not reported	Closed
9	o1OpenAI	97.0%	Artificial AnalysisOriginal result	Not reported	Closed
10	Gemini 2.5 ProGoogle	96.7%	Artificial AnalysisOriginal result	Not reported	Closed

MATH 500

BenchLM mirrors this Vals AI leaderboard for reference only. It is excluded from BenchLM overall and category rankings.

MATH 500; source configurations remain separate from the canonical benchmark table.
Source rank	Source model or configuration	MATH 500 score	Source setup	Parameters (B)	Open / closed
1	Gemini 3 Pro PreviewGoogle	96.40%	high reasoningOriginal result	Not reported	Unknown
2	Grok 4 0709xAI	96.20%	Vals AIOriginal result	Not reported	Unknown
3	GPT-5OpenAI	96.00%	high reasoningOriginal result	Not reported	Unknown
4	Claude Opus 4.1 20250805 ThinkingAnthropic	95.40%	Vals AIOriginal result	Not reported	Unknown
5	Gemini 2.5 Pro Exp 03 25Google	95.20%	Vals AIOriginal result	Not reported	Unknown
6	GPT-5 miniOpenAI	94.80%	high reasoningOriginal result	Not reported	Closed
7	GPT-OSS 120BOpenAI	94.80%	Vals AIOriginal result	Not reported	Open
8	o3OpenAI	94.60%	high reasoningOriginal result	Not reported	Closed
9	Qwen3 235B-A22BAlibaba	94.60%	Vals AIOriginal result	Not reported	Open
10	o4-miniOpenAI	94.20%	high reasoningOriginal result	Not reported	Closed
11	Grok 3 Mini Fast High ReasoningxAI	94.20%	high reasoningOriginal result	Not reported	Unknown
12	GPT-OSS 20BOpenAI	94.20%	Vals AIOriginal result	Not reported	Open
13	Moonshotai Kimi K2 InstructTogether AI	94.20%	Vals AIOriginal result	Not reported	Unknown
14	GLM-4.5Z.AI	94.00%	Vals AIOriginal result	Not reported	Closed
15	GPT-5 nanoOpenAI	93.80%	high reasoningOriginal result	Not reported	Closed
16	Claude Sonnet 4 20250514 ThinkingAnthropic	93.80%	Vals AIOriginal result	Not reported	Unknown
17	Claude Opus 4.1Anthropic	93.00%	Vals AIOriginal result	Not reported	Unknown
18	DeepSeek-R1DeepSeek	92.20%	Vals AIOriginal result	Not reported	Open
19	o3-miniOpenAI	91.80%	high reasoningOriginal result	Not reported	Closed
20	Gemini 2.5 Flash Preview 04 17 ThinkingGoogle	91.80%	Vals AIOriginal result	Not reported	Unknown
21	Gemini 2.5 Flash Preview 04 17Google	91.60%	Vals AIOriginal result	Not reported	Unknown
22	Claude 3.7 Sonnet ThinkingAnthropic	91.60%	Vals AIOriginal result	Not reported	Closed
23	Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561 ThinkingTogether AI	91.40%	Vals AIOriginal result	Not reported	Unknown
24	Claude Opus 4Anthropic	90.40%	Vals AIOriginal result	Not reported	Unknown
25	o1OpenAI	90.40%	high reasoningOriginal result	Not reported	Closed
26	Claude Sonnet 4Anthropic	90.32%	Vals AIOriginal result	Not reported	Unknown
27	Grok 3xAI	89.80%	Vals AIOriginal result	Not reported	Unknown
28	Gemini 2.0 Flash ExpGoogle	89.00%	Vals AIOriginal result	Not reported	Unknown
29	MiniMax M2.1MiniMax	89.00%	Vals AIOriginal result	Not reported	Unknown
30	DeepSeek V3 0324DeepSeek	88.60%	Vals AIOriginal result	Not reported	Open
31	Gemini 2.0 Flash 001Google	88.00%	Vals AIOriginal result	Not reported	Unknown
32	GPT-4.1 miniOpenAI	88.00%	high reasoningOriginal result	Not reported	Closed
33	GPT-4.1OpenAI	87.20%	high reasoningOriginal result	Not reported	Closed
34	Mistral Medium 2505Mistral AI	87.00%	Vals AIOriginal result	Not reported	Unknown
35	Llama4 Maverick Instruct BasicFireworks	85.20%	Vals AIOriginal result	Not reported	Unknown
36	Gemini 2.0 Flash Thinking Exp 01 21Google	84.60%	Vals AIOriginal result	Not reported	Unknown
37	Gemini 1.5 Pro 002Google	82.80%	Vals AIOriginal result	Not reported	Unknown
38	DeepSeek V3DeepSeek	80.40%	Vals AIOriginal result	Not reported	Open
39	GPT-4.1 nanoOpenAI	80.20%	high reasoningOriginal result	Not reported	Closed
40	Meta Llama Llama 4 Scout 17B 16E InstructTogether AI	79.20%	Vals AIOriginal result	Not reported	Unknown
41	Gemini 1.5 Flash 002Google	78.80%	Vals AIOriginal result	Not reported	Unknown
42	Grok 2 1212xAI	78.40%	Vals AIOriginal result	Not reported	Unknown
43	Claude 3.7 SonnetAnthropic	76.80%	Vals AIOriginal result	Not reported	Closed
44	Command A 03 2025Cohere	76.20%	Vals AIOriginal result	Not reported	Unknown
45	GPT-4oOpenAI	75.20%	high reasoningOriginal result	Not reported	Closed
46	Mistral Large 2411Mistral AI	74.40%	Vals AIOriginal result	Not reported	Unknown
47	GPT-4oOpenAI	74.00%	high reasoningOriginal result	Not reported	Closed
48	Meta Llama Llama 3.3 70B Instruct TurboTogether AI	73.40%	Vals AIOriginal result	Not reported	Unknown
49	GPT-4o miniOpenAI	72.60%	high reasoningOriginal result	Not reported	Closed
50	Claude 3.5 SonnetAnthropic	72.40%	Vals AIOriginal result	Not reported	Closed
51	Meta Llama Meta Llama 3.1 405B Instruct TurboTogether AI	71.40%	Vals AIOriginal result	Not reported	Unknown
52	Langston Nim Nvidia Llama 3.3 Nemotron Super 49b V1 42e84561Together AI	71.20%	Vals AIOriginal result	Not reported	Unknown
53	Mistral Small 2402Mistral AI	70.60%	Vals AIOriginal result	Not reported	Unknown
54	Grok 3 Mini Fast Low ReasoningxAI	70.20%	low reasoningOriginal result	Not reported	Unknown
55	Mistral Small 2503Mistral AI	68.40%	Vals AIOriginal result	Not reported	Unknown
56	Meta Llama Meta Llama 3.1 70B Instruct TurboTogether AI	65.00%	Vals AIOriginal result	Not reported	Unknown
57	Claude 3.5 HaikuAnthropic	64.20%	Vals AIOriginal result	Not reported	Closed
58	Jamba Large 1.6AI21 Labs	54.80%	Vals AIOriginal result	Not reported	Unknown
59	Meta Llama Meta Llama 3.1 8B Instruct TurboTogether AI	44.40%	Vals AIOriginal result	Not reported	Unknown
60	Jamba Mini 1.6AI21 Labs	25.40%	Vals AIOriginal result	Not reported	Unknown

MATH-500 (Kaggle)

Native proportions are displayed as percentages. Source profiles, task versions, evaluation dates, and symmetric intervals stay attached to each row. Missing results are not zero. How Kaggle Benchmarks reruns tests.

MATH-500 (Kaggle); source configurations remain separate from the canonical benchmark table.
Source rank	Source model or configuration	Accuracy (%)	Source setup	Parameters (B)	Open / closed
1	Claude Sonnet 5.5	99.4%	Task version 8; Evaluated 2026-09-30Original result	Not reported	Unknown
2	Gemini 3.8 Flash	99.1%	Task version 8; Evaluated 2026-09-04Original result	Not reported	Unknown
3	Grok 4.6	99.1%	Task version 8; Evaluated 2026-10-06Original result	Not reported	Unknown
4	GPT-6.1 Sol	99.0%	Task version 8; Evaluated 2026-10-05Original result	Not reported	Unknown
5	Gemini 3.7 Flash	99.0%	Task version 8; Evaluated 2026-08-27Original result	Not reported	Unknown
6	Gemini 4 Argon	99.0%	Task version 8; Evaluated 2026-10-09Original result	Not reported	Unknown
7	Claude Haiku 5.5	98.7%	Task version 8; Evaluated 2026-10-09Original result	Not reported	Unknown
8	GPT-6 Sol	98.6%	Task version 8; Evaluated 2026-10-01Original result	Not reported	Unknown
9	Claude Opus 5	97.6%	Task version 8; Evaluated 2026-07-27Original result	Not reported	Unknown
10	GPT-5.6 Sol	96.7%	Task version 8; Evaluated 2026-07-21Original result	Not reported	Unknown
11	Gemini 3.5 Flash	96.4%	Task version 8; Evaluated 2026-07-21Original result	Not reported	Unknown
12	GPT-5.6 Terra	95.9%	Task version 8; Evaluated 2026-07-20Original result	Not reported	Unknown
13	Gemini 3.6 Flash	95.9%	Task version 8; Evaluated 2026-07-23Original result	Not reported	Unknown
14	Gemini 3 Flash Preview	94.1%	Task version 8; Evaluated 2026-07-13Original result	Not reported	Unknown
15	GPT-5.6 Luna	93.7%	Task version 8; Evaluated 2026-07-21Original result	Not reported	Unknown
16	Claude Opus 4.8	93.5%	Task version 8; Evaluated 2026-07-21Original result	Not reported	Unknown
17	Grok 4.5	92.9%	Task version 8; Evaluated 2026-07-14Original result	Not reported	Unknown
18	Gemini 3.5 Flash-Lite	92.8%	Task version 8; Evaluated 2026-07-23Original result	Not reported	Unknown
19	Claude Sonnet 5	92.7%	Task version 8; Evaluated 2026-07-21Original result	Not reported	Unknown
20	GPT-5.5	92.7%	Task version 8; Evaluated 2026-07-14Original result	Not reported	Unknown
21	Claude Sonnet 4.6	90.4%	Task version 8; Evaluated 2026-07-14Original result	Not reported	Unknown
22	gpt-oss-120b	76.5%	Task version 8; Evaluated 2026-07-16Original result	Not reported	Unknown

Among the reported MATH-500 rows, Ternary Bonsai 2 27B is first at 98.8%. The third row is 0.2 points behind. The broader top-10 range is 1.8 points, so many of the published results sit in a relatively narrow band.

27 models have been evaluated on MATH-500. The benchmark falls in the Mathematics category. MATH-500 is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About MATH-500

Year

2021

Tasks

500 problems

Format

Free-form mathematical answers

Difficulty

High school to undergraduate

MATH-500 is one of the most widely cited math benchmarks. It is nearing saturation with top reasoning models scoring 96-99%, making it less useful for differentiating frontier models but still a standard baseline.

Measuring Mathematical Problem Solving With the MATH Dataset

Freshness and provenance

Version

MATH-500 2021

Refresh cadence

Static

Staleness state

Stale

Question availability

Public benchmark set

StaleDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does MATH-500 measure?

A curated subset of 500 problems from the MATH dataset, covering algebra, counting and probability, geometry, intermediate algebra, number theory, prealgebra, and precalculus.

Which model scores highest on MATH-500?

Ternary Bonsai 2 27B by Prism ML currently leads with a score of 98.8% on MATH-500.

How many models are evaluated on MATH-500?

27 AI models have published results on MATH-500 in the BenchLM catalog.

Compare top models on MATH-500

Ternary Bonsai 2 27B vs OrcaCyber-Nano-1.0 OrcaCyber-Nano-1.0 vs Sarvam 105B Sarvam 105B vs Qwen3 235B-A22B (Reasoning)Qwen3 235B-A22B (Reasoning) vs QwQ-32B

Last updated: October 29, 2026 · BenchLM version MATH-500 2021

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 6,100+ subscribers.

One email each week. Unsubscribe anytime.