benchlm.ai

Nemotron Ultra 253B vs o1-preview

Side-by-side benchmark comparison across knowledge, coding, math, and reasoning.

Quick Verdict

o1-preview wins overall with a score of 71 vs 40 (31 point difference).o1-preview wins 4 out of 4 categories.

Nemotron Ultra 253B

CreatorNVIDIA
TypeOpen Weight
Context32K
Score40

o1-preview

CreatorOpenAI
TypeProprietary
Context200K
Score71

Knowledge

o1-preview

Nemotron Ultra 253B

46.8

o1-preview

89

49
MMLU
92
48
GPQA
90
46
SuperGPQA
88
44
OpenBookQA
86

Coding

o1-preview

Nemotron Ultra 253B

41

o1-preview

86

41
HumanEval
86

Mathematics

o1-preview

Nemotron Ultra 253B

48

o1-preview

93

49
AIME 2023
94
51
AIME 2024
96
50
AIME 2025
95
45
HMMT Feb 2023
90
47
HMMT Feb 2024
92
46
HMMT Feb 2025
91
48
BRUMO 2025
93

Reasoning

o1-preview

Nemotron Ultra 253B

46

o1-preview

87

47
SimpleQA
88
45
MuSR
86

Frequently Asked Questions

Which is better, Nemotron Ultra 253B or o1-preview?

o1-preview scores higher overall with 71 vs 40, a difference of 31 points across all benchmarks.

Which is better for knowledge tasks, Nemotron Ultra 253B or o1-preview?

o1-preview leads in knowledge tasks with an average score of 89 vs 46.8.

Which is better for coding, Nemotron Ultra 253B or o1-preview?

o1-preview leads in coding with an average score of 86 vs 41.

Which is better for math, Nemotron Ultra 253B or o1-preview?

o1-preview leads in math with an average score of 93 vs 48.

Which is better for reasoning, Nemotron Ultra 253B or o1-preview?

o1-preview leads in reasoning with an average score of 87 vs 46.

Related Comparisons

Nemotron Ultra 253B vs GPT-5.4Nemotron Ultra 253B vs Gemini 3.1 ProNemotron Ultra 253B vs Claude Opus 4.6Nemotron Ultra 253B vs GPT-5.3 CodexNemotron Ultra 253B vs Grok 4.1Nemotron Ultra 253B vs GPT-5.2