Multilingual Grade School Math (MGSM)

We show this table for reference; we do not rank on it.

Data verified November 13, 2026

A multilingual benchmark that translates 250 grade school math problems from GSM8K into 10 typologically diverse languages: Bengali, German, Spanish, French, Japanese, Russian, Swahili, Telugu, Thai, and Chinese.

Benchmark score on MGSM — November 13, 2026

We compile the MGSM rows from provider self-reports. o4-mini leads the table at 93.7%, followed by Llama 3.3 70B (91.1%) and o1-mini (89.9%). We do not use these results to rank models overall.

1Closed

o4-mini

OpenAI

Llama 3.3 70B

Benchmark score table (15 models)

Benchmark score results for MGSM
Rank	Model / configuration	Score	Who ran it	Parameters (B)	Open / closed
1	o4-miniOpenAIReported by OpenAI	93.7%	Reported by OpenAI	Not reported	Closed
2	Llama 3.3 70BMetaReported by Meta	91.1%	Reported by Meta	Not reported	Open
3	o1-miniOpenAIReported by OpenAI	89.9%	Reported by OpenAI	Not reported	Closed
4	GPT-4.5OpenAIReported by OpenAI	86.9%	Reported by OpenAI	Not reported	Closed
5	Llama 3.1 70BMetaReported by Meta	86.9%	Reported by Meta	Not reported	Open
6	DeepSeek V4 Flash BaseDeepSeekReported by DeepSeek	85.7%	Reported by DeepSeek	Not reported	Open
7	Claude 3.5 HaikuAnthropicReported by Anthropic	85.6%	Reported by Anthropic	Not reported	Closed
8	DeepSeek V4 Pro BaseDeepSeekReported by DeepSeek	84.4%	Reported by DeepSeek	Not reported	Open
9	Phi-4MicrosoftReported by Microsoft	80.6%	Reported by Microsoft	Not reported	Open
10	Llama 3.1 8BMetaReported by Meta	68.9%	Reported by Meta	Not reported	Open
11	Phi-3.5-miniMicrosoftReported by Microsoft	47.9%	Reported by Microsoft	Not reported	Open
12	Granite-4.0-1BIBMReported by IBM	37.8%	Reported by IBM	Not reported	Open
13	Granite-4.0-H-1BIBMReported by IBM	27.5%	Reported by IBM	Not reported	Open
14	Granite-4.0-H-350MIBMReported by IBM	16.2%	Reported by IBM	Not reported	Open
15	Granite-4.0-350MIBMReported by IBM	14.7%	Reported by IBM	Not reported	Open

Lab-reported
Benchmark-run
Independent

What the labels mean

MGSM

BenchLM mirrors this Vals AI leaderboard for reference only. It is excluded from BenchLM overall and category rankings.

MGSM; source configurations remain separate from the canonical benchmark table.
Source rank	Source model or configuration	MGSM score	Source setup	Parameters (B)	Open / closed
1	Claude Opus 4.5 ThinkingAnthropic	95.20%	Vals AIOriginal result	Not reported	Closed
2	Claude Opus 4.5Anthropic	94.76%	Vals AIOriginal result	Not reported	Closed
3	Claude Opus 4.1 20250805 ThinkingAnthropic	94.44%	Vals AIOriginal result	Not reported	Unknown
4	Claude Sonnet 4.5 ThinkingAnthropic	94.33%	Vals AIOriginal result	Not reported	Closed
5	Claude Opus 4.1Anthropic	94.22%	Vals AIOriginal result	Not reported	Unknown
6	GPT-5.2OpenAI	94.00%	xhigh reasoningOriginal result	Not reported	Closed
7	Gemini 3 Pro PreviewGoogle	93.93%	high reasoningOriginal result	Not reported	Unknown
8	Claude Opus 4Anthropic	93.78%	Vals AIOriginal result	Not reported	Unknown
9	o4-miniOpenAI	93.42%	high reasoningOriginal result	Not reported	Closed
10	Gemini 3 Flash PreviewGoogle	93.31%	high reasoningOriginal result	Not reported	Unknown
11	Claude Sonnet 4Anthropic	93.02%	Vals AIOriginal result	Not reported	Unknown
12	GPT-5.1OpenAI	92.98%	high reasoningOriginal result	Not reported	Closed
13	Claude 3.7 Sonnet ThinkingAnthropic	92.98%	Vals AIOriginal result	Not reported	Closed
14	GPT-5OpenAI	92.84%	high reasoningOriginal result	Not reported	Unknown
15	Claude 3.5 SonnetAnthropic	92.58%	Vals AIOriginal result	Not reported	Closed
16	GPT-5 miniOpenAI	92.58%	high reasoningOriginal result	Not reported	Closed
17	Qwen3 235B-A22BAlibaba	92.47%	Vals AIOriginal result	Not reported	Open
18	Llama4 Maverick Instruct BasicFireworks	92.44%	Vals AIOriginal result	Not reported	Unknown
19	Claude 3.7 SonnetAnthropic	92.40%	Vals AIOriginal result	Not reported	Closed
20	DeepSeek-R1DeepSeek	92.25%	Vals AIOriginal result	Not reported	Open
21	Claude Haiku 4.5 ThinkingAnthropic	92.15%	Vals AIOriginal result	Not reported	Closed
22	DeepSeek V3DeepSeek	92.15%	Vals AIOriginal result	Not reported	Open
23	Qwen3 Max PreviewAlibaba	92.15%	Vals AIOriginal result	Not reported	Unknown
24	GPT-OSS 120BOpenAI	92.04%	Vals AIOriginal result	Not reported	Open
25	Qwen3 MaxAlibaba	91.82%	Vals AIOriginal result	Not reported	Closed
26	o3OpenAI	91.75%	high reasoningOriginal result	Not reported	Closed
27	DeepSeek V3 0324DeepSeek	91.67%	Vals AIOriginal result	Not reported	Open
28	Grok 3xAI	91.35%	Vals AIOriginal result	Not reported	Unknown
29	o3-miniOpenAI	91.35%	high reasoningOriginal result	Not reported	Closed
30	Meta Llama Llama 3.3 70B Instruct TurboTogether AI	91.09%	Vals AIOriginal result	Not reported	Unknown
31	Moonshotai Kimi K2 InstructTogether AI	90.95%	Vals AIOriginal result	Not reported	Unknown
32	Grok 4 0709xAI	90.91%	Vals AIOriginal result	Not reported	Unknown
33	Grok 4 Fast (Reasoning)xAI	90.87%	Vals AIOriginal result	Not reported	Closed
34	Mistral Medium 2505Mistral AI	90.87%	Vals AIOriginal result	Not reported	Unknown
35	DeepSeek V3p2Fireworks	90.87%	Vals AIOriginal result	Not reported	Unknown
36	Claude Sonnet 4 20250514 ThinkingAnthropic	90.87%	Vals AIOriginal result	Not reported	Unknown
37	GLM-4.5Z.AI	90.84%	Vals AIOriginal result	Not reported	Closed
38	GPT-4oOpenAI	90.69%	high reasoningOriginal result	Not reported	Closed
39	Grok 3 Mini Fast High ReasoningxAI	90.44%	high reasoningOriginal result	Not reported	Unknown
40	GPT-4oOpenAI	90.36%	high reasoningOriginal result	Not reported	Closed
41	Grok 3 Mini Fast Low ReasoningxAI	90.36%	low reasoningOriginal result	Not reported	Unknown
42	Kimi K2 ThinkingMoonshot AI	90.15%	Vals AIOriginal result	Not reported	Unknown
43	Gemini 2.5 Flash Preview 09 2025Google	89.85%	Vals AIOriginal result	Not reported	Unknown
44	Gemini 2.5 Flash Preview 09 2025 ThinkingGoogle	89.82%	Vals AIOriginal result	Not reported	Unknown
45	GLM-4.6Z.AI	89.75%	Vals AIOriginal result	Not reported	Open
46	Gemini 2.5 Flash Lite Preview 09 2025Google	89.53%	Vals AIOriginal result	Not reported	Unknown
47	Grok 4.1 Fast (Reasoning)xAI	89.53%	Vals AIOriginal result	Not reported	Closed
48	o1OpenAI	89.31%	high reasoningOriginal result	Not reported	Closed
49	GPT-5 nanoOpenAI	89.31%	high reasoningOriginal result	Not reported	Closed
50	Gemini 1.5 Pro 002Google	89.20%	Vals AIOriginal result	Not reported	Unknown
51	Gemini 2.0 Flash 001Google	89.02%	Vals AIOriginal result	Not reported	Unknown
52	GPT-OSS 20BOpenAI	89.02%	Vals AIOriginal result	Not reported	Open
53	Gemini 2.5 Flash Lite Preview 09 2025 ThinkingGoogle	88.40%	Vals AIOriginal result	Not reported	Unknown
54	GLM-4.7Z.AI	88.18%	Vals AIOriginal result	Not reported	Open
55	Grok 4 Fast Non ReasoningxAI	88.00%	Vals AIOriginal result	Not reported	Unknown
56	Meta Llama Llama 4 Scout 17B 16E InstructTogether AI	87.96%	Vals AIOriginal result	Not reported	Unknown
57	MiniMax M2.1MiniMax	87.85%	Vals AIOriginal result	Not reported	Unknown
58	GPT-4.1 miniOpenAI	87.78%	high reasoningOriginal result	Not reported	Closed
59	GPT-4.1OpenAI	87.67%	high reasoningOriginal result	Not reported	Closed
60	Grok 4.1 Fast Non ReasoningxAI	87.56%	Vals AIOriginal result	Not reported	Unknown
61	Mistral Large 2411Mistral AI	87.24%	Vals AIOriginal result	Not reported	Unknown
62	Gemini 1.5 Flash 002Google	86.58%	Vals AIOriginal result	Not reported	Unknown
63	Magistral Small 2509Mistral AI	86.25%	Vals AIOriginal result	Not reported	Unknown
64	GPT-4o miniOpenAI	86.18%	high reasoningOriginal result	Not reported	Closed
65	Grok 2 1212xAI	86.15%	Vals AIOriginal result	Not reported	Unknown
66	DeepSeek V3p2 ThinkingFireworks	86.04%	Vals AIOriginal result	Not reported	Unknown
67	Command A 03 2025Cohere	85.71%	Vals AIOriginal result	Not reported	Unknown
68	Mistral Large 2512Mistral AI	85.42%	Vals AIOriginal result	Not reported	Unknown
69	Claude 3.5 HaikuAnthropic	84.62%	Vals AIOriginal result	Not reported	Closed
70	Mistral Small 2503Mistral AI	84.22%	Vals AIOriginal result	Not reported	Unknown
71	Mistral Small 2402Mistral AI	83.96%	Vals AIOriginal result	Not reported	Unknown
72	Magistral Medium 2509Mistral AI	74.62%	Vals AIOriginal result	Not reported	Unknown
73	Jamba Large 1.6AI21 Labs	71.24%	Vals AIOriginal result	Not reported	Unknown
74	GPT-4.1 nanoOpenAI	69.27%	high reasoningOriginal result	Not reported	Closed
75	Jamba Mini 1.6AI21 Labs	41.71%	Vals AIOriginal result	Not reported	Unknown

MGSM Bengali (Kaggle)

Native accuracy proportions are displayed as percentages. Source profiles stay separate from canonical model identities and from multilingual aggregate results. How Kaggle Benchmarks reruns tests.

MGSM Bengali (Kaggle); source configurations remain separate from the canonical benchmark table.
Source rank	Source model or configuration	Accuracy	Source setup	Parameters (B)	Open / closed
1	Claude Opus 4.8	95.6%	Evaluation date not reportedOriginal result	Not reported	Unknown
2	Claude Opus 4	94.4%	Evaluated 2025-07-01Original result	Not reported	Unknown
3	GPT-5	94.0%	Evaluated 2025-08-19Original result	Not reported	Unknown
4	Gemini 2.5 Pro Preview	92.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
5	Claude Opus 4.1	92.8%	Evaluated 2025-08-20Original result	Not reported	Unknown
6	Gemma 4 26B A4B	92.4%	Evaluation date not reportedOriginal result	Not reported	Unknown
7	o4 mini	91.6%	Evaluated 2025-06-30Original result	Not reported	Unknown
8	gpt-oss-120b	91.2%	Evaluated 2025-08-19Original result	Not reported	Unknown
9	gpt-oss-20b	91.2%	Evaluated 2025-08-19Original result	Not reported	Unknown
10	Claude 3.7 Sonnet	90.4%	Evaluated 2025-07-01Original result	Not reported	Unknown
11	DeepSeek-V3	89.2%	Evaluated 2025-06-30Original result	Not reported	Unknown
12	deepseek-v3.1	89.2%	Evaluated 2025-08-27Original result	Not reported	Unknown
13	Gemini 1.5 Pro	88.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
14	GPT-4o	88.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
15	DeepSeek-R1	88.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
16	Claude 3.5 Sonnet	88.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
17	o3 mini	88.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
18	Grok 3 Mini	88.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
19	Gemini 3 Pro Preview	88.4%	Evaluated 2025-11-14Original result	Not reported	Unknown
20	Gemini 1.5 Flash	87.6%	Evaluated 2025-06-30Original result	Not reported	Unknown
21	Gemini 2.5 Flash Preview	87.6%	Evaluated 2025-06-30Original result	Not reported	Unknown
22	o3	87.6%	Evaluated 2025-07-01Original result	Not reported	Unknown
23	Claude Sonnet 4	85.6%	Evaluated 2025-07-01Original result	Not reported	Unknown
24	Gemini 3.5 Flash	83.5%	Evaluation date not reportedOriginal result	Not reported	Unknown
25	Mistral Large 2	83.2%	Evaluated 2025-07-01Original result	Not reported	Unknown
26	Grok 4	82.8%	Evaluated 2025-09-02Original result	Not reported	Unknown
27	GPT-4o mini	82.0%	Evaluated 2025-06-30Original result	Not reported	Unknown
28	Claude 3.5 Haiku	78.8%	Evaluated 2025-07-01Original result	Not reported	Unknown
29	Gemini 1.5 Flash 8B	78.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
30	Grok 3	78.0%	Evaluated 2025-06-30Original result	Not reported	Unknown
31	Gemma 3 12B	74.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
32	Codestral	74.0%	Evaluated 2025-07-01Original result	Not reported	Unknown
33	Gemini 2.0 Flash	72.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
34	o1	66.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
35	Qwen 3 235B A22B Thinking	65.6%	Evaluated 2025-08-19Original result	Not reported	Unknown
36	Gemma 3 4B	64.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
37	Mixtral 8x22B	58.8%	Evaluated 2025-07-01Original result	Not reported	Unknown
38	Gemma 4 31B	57.2%	Evaluation date not reportedOriginal result	Not reported	Unknown
39	Ministral 8B	56.0%	Evaluated 2025-07-01Original result	Not reported	Unknown
40	GPT-4.1	54.0%	Evaluated 2025-06-30Original result	Not reported	Unknown
41	Granite 4.0 Small	53.2%	Evaluated 2025-09-30Original result	Not reported	Unknown
42	Grok 2	50.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
43	Gemma 3 27B	48.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
44	o1 mini	46.4%	Evaluated 2025-06-30Original result	Not reported	Unknown
45	GPT-3.5 Turbo	32.8%	Evaluated 2025-06-30Original result	Not reported	Unknown
46	Ministral 3B	24.8%	Evaluated 2025-07-01Original result	Not reported	Unknown

Among the reported MGSM rows, o4-mini is first at 93.7%. The third row is 3.8 points behind. The broader top-10 range is 24.8 points, so the table still separates the published systems.

15 models have been evaluated on MGSM. The benchmark falls in the Multilingual category. MGSM is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.

About MGSM

Year

2022

Tasks

250 problems × 11 languages

Format

Math word problems

Difficulty

Grade school math, multilingual

MGSM evaluates mathematical reasoning across languages, revealing that performance can vary significantly across languages, with lower-resource languages (Bengali, Swahili, Telugu) typically showing the largest gaps.

Language Models are Multilingual Chain-of-Thought Reasoners

Freshness and provenance

Version

MGSM 2022

Refresh cadence

Static

Staleness state

Stale

Question availability

Public benchmark set

StaleDisplay only

BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.

Questions

What does MGSM measure?

Which model scores highest on MGSM?

o4-mini by OpenAI currently leads with a score of 93.7% on MGSM.

How many models are evaluated on MGSM?

15 AI models have published results on MGSM in the BenchLM catalog.

Compare top models on MGSM

o4-mini vs Llama 3.3 70B Llama 3.3 70B vs o1-mini o1-mini vs GPT-4.5 GPT-4.5 vs Llama 3.1 70B

Last updated: November 13, 2026 · BenchLM version MGSM 2022

Know when it’s worth switching models

The model to choose, the cheaper alternative, and the release we would wait on.

Read a sample issue

Join 7,000+ subscribers.

One email each week. Unsubscribe anytime.