Head-to-head comparison across 0benchmark categories. Overall scores shown here use BenchLM's provisional ranking lane.
Gemini 1.5 Pro
35
Llama 3.1 405B
41
Benchmark data for Gemini 1.5 Pro and Llama 3.1 405B is coming soon on BenchLM.
Gemini 1.5 Pro
Llama 3.1 405B
$1.25 / $5
$0 / $0
N/A
29 t/s
N/A
2.19s
2M
128K
Benchmark data for Gemini 1.5 Pro and Llama 3.1 405B is coming soon on BenchLM.
BenchLM has partial data for these models, but not enough overlapping benchmark coverage to produce a fair score-level comparison yet.
Gemini 1.5 Pro is priced at $1.25 input / $5.00 output per 1M tokens, versus $0.00 input / $0.00 output per 1M tokens for Llama 3.1 405B. Gemini 1.5 Pro has the larger context window at 2M, compared with 128K for Llama 3.1 405B.
Not fully yet. BenchLM is tracking both models, but the sourced benchmark breakdown for this comparison is still coming soon.
BenchLM only shows category winners and benchmark-level calls when we have sourced results that can be compared fairly. For these models, the public benchmark coverage is not complete enough yet.
Gemini 1.5 Pro: $1.25 input / $5.00 output per 1M tokens Llama 3.1 405B: $0.00 input / $0.00 output per 1M tokens Both model pages still include creator, context window, reasoning mode, and other metadata while benchmark coverage fills in.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.