benchlm.ai

Llama 3.1 405B vs o1-preview

Side-by-side benchmark comparison across knowledge, coding, math, and reasoning.

Quick Verdict

o1-preview wins overall with a score of 71 vs 58 (13 point difference).o1-preview wins 4 out of 4 categories.

Llama 3.1 405B

CreatorMeta
TypeOpen Weight
Context128K
Score58

o1-preview

CreatorOpenAI
TypeProprietary
Context200K
Score71

Knowledge

o1-preview

Llama 3.1 405B

68.5

o1-preview

89

70
MMLU
92
70
GPQA
90
68
SuperGPQA
88
66
OpenBookQA
86

Coding

o1-preview

Llama 3.1 405B

62

o1-preview

86

62
HumanEval
86

Mathematics

o1-preview

Llama 3.1 405B

69

o1-preview

93

70
AIME 2023
94
72
AIME 2024
96
71
AIME 2025
95
66
HMMT Feb 2023
90
68
HMMT Feb 2024
92
67
HMMT Feb 2025
91
69
BRUMO 2025
93

Reasoning

o1-preview

Llama 3.1 405B

67

o1-preview

87

68
SimpleQA
88
66
MuSR
86

Frequently Asked Questions

Which is better, Llama 3.1 405B or o1-preview?

o1-preview scores higher overall with 71 vs 58, a difference of 13 points across all benchmarks.

Which is better for knowledge tasks, Llama 3.1 405B or o1-preview?

o1-preview leads in knowledge tasks with an average score of 89 vs 68.5.

Which is better for coding, Llama 3.1 405B or o1-preview?

o1-preview leads in coding with an average score of 86 vs 62.

Which is better for math, Llama 3.1 405B or o1-preview?

o1-preview leads in math with an average score of 93 vs 69.

Which is better for reasoning, Llama 3.1 405B or o1-preview?

o1-preview leads in reasoning with an average score of 87 vs 67.

Related Comparisons

Llama 3.1 405B vs GPT-5.4Llama 3.1 405B vs Gemini 3.1 ProLlama 3.1 405B vs Claude Opus 4.6Llama 3.1 405B vs GPT-5.3 CodexLlama 3.1 405B vs Grok 4.1Llama 3.1 405B vs GPT-5.2