Data as of October 17, 2026 · How the score is built
Released Jul 23, 2024128K context
Llama 3.1 405B
Decision readingLlama 3.1 405B scores 28.2 out of 100 and ranks #197 of 224. This profile shows 1 source-displayable benchmark rows. Published weights can be self-hosted, but infrastructure cost varies and is not a comparable API token rate.
Released Jul 23, 2024 — see all recent releases
Decision snapshot
Each value carries a field reference instead of floating alone. Markers compare this model with the current ranked and priced catalog; they are not absolute quality thresholds.
Capability
28.2/100
field median 49.6#197 of 224 ranked models
Public
#197of 224
Verified —
Price
Self-hosted; infrastructure cost varies
input median $0.75No comparable first-party hosted token rate
Speed
29tok/s
field median 92 tok/sFirst token 2.19 s
Context
128Ktokens
field median 256,000Reported for this model; direct source link not stored
Strongest published evidence
Published rows are visible, but no category has enough eligible evidence for a comparative rank.
Validate before choosing
1 published rows leave some tracked benchmark slots empty. No comparable first-party API token rate is published.
Source-linked · 1 displayable benchmark row
Category score record
Scores and ranks appear only where published evidence can be displayed. The table keeps the score, weight, cohort, and evidence state together.
| Category | Score | Rank | Percentile | Weight | Benchmarks | Evidence |
|---|---|---|---|---|---|---|
| AgenticWeight 22%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| CodingWeight 20%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| ReasoningWeight 17%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| MultimodalWeight 12%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| KnowledgeWeight 12%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| MultilingualWeight 7%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| Inst. FollowingRank #148 of 148Percentile 0thWeight 5%1 benchmarkVerified | 0.0 | 1 benchmark | Verified | |||
| MathWeight 5%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured |
1 of 681 tracked benchmark slots have displayable evidence · bars run 0–100
Coverage detailsHow much of this is verified
Coverage is split by category so a strong number never hides a thin evidence base. Verified means the row is tied to a published source; provisional rows remain visible but separate.
- AgenticNot measured
- CodingNot measured
- ReasoningNot measured
- MultimodalNot measured
- KnowledgeNot measured
- MultilingualNot measured
- Inst. Following1/1 verified
- MathNot measured
Capability shape
Each axis shows percentile within that category’s eligible cohort. The comparison outline is the median of the six nearest public-score peers; a collapsed vertex means the category is not rank-eligible.
Llama 3.1 405B category percentile values
- AgenticNot eligible
- CodingNot eligible
- ReasoningNot eligible
- MultimodalNot eligible
- KnowledgeNot eligible
- MultilingualNot eligible
- Instruction following0th percentile
- MathNot eligible
The dashed outline is median of 6 nearest peers.
Eligible category ranks
- AgenticNot ranked
- CodingNot ranked
- ReasoningNot ranked
- MultimodalNot ranked
- KnowledgeNot ranked
- MultilingualNot ranked
- Inst. Following#148/148
- MathNot ranked
Benchmark ledger
Inst. Following opens by default. The marker compares each value with the best source-verified result in the catalog; provisional leaders do not set the reference. Expand the remaining categories for every published row.
Inst. Following1 row
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| IFEvalInstruction-Following Eval | Score88.6% | Versus best verified row Best verified: LFM2.5-24B-A2B · 96.8% | Gap8.2 behind | WeightDisplay only | Provider exact |
Bars run 0–100; the dark tick marks the best source-verified value
All 1 rowsLineage
The sequence follows explicit supersedes links. Each score is estimated for that model; a relative can inform a sparse estimate but never sets a floor, so a newer release can score below an earlier one. Scores and prices remain blank when the corresponding public row or first-party rate is unavailable.
Base entry
Spec sheet
Each documented value carries its source. Missing fields stay visible as not sourced or not published, rather than disappearing from the page.
- API model ID
- Not published
- Context window
- 128K
- Maximum output
- Not sourced yet
- Knowledge cutoff
- Not sourced yet
- Input modalities
- Not sourced yet
- Output modalities
- Not sourced yet
- Parameters
- Not sourced yet
- Availability
- Not sourced yet
- Cloud regions
- Not tracked yet
- Lifecycle
- Established
- API capabilities
- Tool calling, structured outputs, and batch support are not tracked yet
- Prompt caching
- Not documented in the pricing record
- Self-host
- Open weights available; hardware estimate not sourced
- Rate limits
- Not tracked yet
How to read this profile
The visual layer above carries the decisions. These notes preserve the model, ranking, coverage, and family context behind the numbers.
Llama 3.1 405B ranks #197 of 224 on the public leaderboard with a score of 28.21/100. It does not yet have enough sourced coverage for a verified position.
Llama 3.1 405B is a open weight model with a 128K context window. No explicit reasoning mode is documented in this profile.
1 of 681 tracked benchmark slots currently have displayable evidence. Missing categories stay blank.
Last updated October 17, 2026. Runtime fields remain blank until a sourced snapshot exists.
Deployment options
Self-host and provider-specific paths stay separate from benchmark evidence so operating constraints are visible before a score becomes the whole decision.
Published weights are available, but BenchLM does not yet have a sourced parameter and VRAM profile for this exact model. Hardware cost estimates stay unavailable until that sizing record is complete.
Estimate VRAM from known parametersQuestions
How does Llama 3.1 405B perform overall in AI benchmarks?
Llama 3.1 405B ranks #197 out of 224 models on the public BenchAlign leaderboard, with a score of 28.21/100. Its evidence status is Estimated, and this profile shows 1 source-displayable benchmark row. The label describes evidence depth, not a provider quality claim; inspect category rows before choosing a workload.
Is Llama 3.1 405B good for instruction following?
Llama 3.1 405B ranks #148 out of 148 eligible models for instruction following, with a public category score of 0/100. Higher-ranked alternatives are available for workloads where this category decides the choice. Check the underlying rows before treating the aggregate as a workload guarantee.
Is Llama 3.1 405B open source?
Llama 3.1 405B is an open-weight model from Meta. Its weights can be downloaded for local or hosted deployment, subject to the published license. Open weight does not automatically mean open source: training data and training code may remain private, and commercial restrictions can still apply.
Does Llama 3.1 405B have full benchmark coverage on BenchLM?
No. Llama 3.1 405B currently has 11 source-displayable rows across 681 tracked benchmark slots. The profile exposes published, non-generated evidence and leaves missing categories blank until an exact evaluation is available. Coverage describes how much was measured; it is not a penalty added to an individual benchmark result.
What is the context window size of Llama 3.1 405B?
Llama 3.1 405B has a reported context window of 128K in the exact-model catalog record. The value stays visible, but the profile marks its source link as unavailable instead of presenting it as directly documented. Maximum output length remains separate because providers often publish a different limit.