Anthropic · Model release
Data as of October 14, 2026 · How the score is built
Released Oct 22, 2024200K contextClaude 3.5 Haiku model card addendum
Claude 3.5 Haiku
Decision readingClaude 3.5 Haiku is tracked, but not publicly ranked yet. The profile exposes 10 sourced benchmark rows and leaves unsupported fields blank until a published record exists.
Released Oct 22, 2024 — see all recent releases
Claude 3.5 Haiku will be repriced, updated or retired. Get each notice with its source and date. Follow model changes
Decision snapshot
Each value carries a field reference instead of floating alone. Markers compare this model with the current ranked and priced catalog; they are not absolute quality thresholds.
Capability
Unranked
field median 49.8Not eligible for a public rank
Price
Not listed
input median $0.80No comparable first-party hosted token rate
Speed
Not measured
field median 90 tok/sTime to first token not measured
Context
200Ktokens
field median 256,000Reported for this model; direct source link not stored
Strongest published evidence
Published rows are visible, but no category has enough eligible evidence for a comparative rank.
Validate before choosing
10 published rows leave some tracked benchmark slots empty. No comparable first-party API token rate is published.
Source-linked · 10 displayable benchmark rows
Follow model changesCategory score record
Scores and ranks appear only where published evidence can be displayed. The table keeps the score, weight, cohort, and evidence state together.
| Category | Score | Rank | Percentile | Weight | Benchmarks | Evidence |
|---|---|---|---|---|---|---|
| AgenticWeight 22%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| CodingRank Not rankedWeight 20%2 benchmarksVerified | 24.5 | 2 benchmarks | Verified | |||
| ReasoningWeight 17%2 benchmarksVerified | Score pending | 2 benchmarks | Verified | |||
| MultimodalWeight 12%0 benchmarksNot measured | Not measured | 0 benchmarks | Not measured | |||
| KnowledgeRank Not rankedWeight 12%3 benchmarksVerified | 54.3 | 3 benchmarks | Verified | |||
| MultilingualWeight 7%1 benchmarkVerified | Score pending | 1 benchmark | Verified | |||
| Inst. FollowingWeight 5%1 benchmarkVerified | Score pending | 1 benchmark | Verified | |||
| MathWeight 5%1 benchmarkVerified | Score pending | 1 benchmark | Verified |
10 of 670 tracked benchmark slots have displayable evidence · bars run 0–100
Coverage detailsHow much of this is verified
Coverage is split by category so a strong number never hides a thin evidence base. Verified means the row is tied to a published source; provisional rows remain visible but separate.
- AgenticNot measured
- Coding2/2 verified
- Reasoning2/2 verified
- MultimodalNot measured
- Knowledge3/3 verified
- Multilingual1/1 verified
- Inst. Following1/1 verified
- Math1/1 verified
Benchmark ledger
Coding opens by default. The marker compares each value with the best source-verified result in the catalog; provisional leaders do not set the reference. Expand the remaining categories for every published row.
Coding2 rows
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| HumanEvalEvaluating Large Language Models Trained on Code | Score88.1% | Versus best verified row Best verified: o4-mini · 97.3% | Gap9.2 behind | WeightDisplay only | |
| SWE-bench VerifiedSoftware Engineering Benchmark Verified | Score40.6% | Versus best verified row Best verified: Claude Opus 5 · 96% | Gap55.4 behind | WeightDisplay only |
Reasoning2 rows
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| BBHBIG-Bench Hard | Score86.6% | Versus best verified row Best verified: Claude 3.5 Haiku · 86.6% | GapBest verified | WeightDisplay only | |
| DROPDiscrete Reasoning Over Paragraphs | Score83.1% | Versus best verified row Best verified: o1-mini · 83.9% | Gap0.8 behind | WeightDisplay only |
Knowledge3 rows
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| MMLU-ProMassive Multitask Language Understanding Professional | Score65% | Versus best verified row Best verified: Qwen3.7 Max · 89.6% | Gap24.6 behind | Weight6% ref. weight | |
| GPQAGraduate-Level Google-Proof Q&A | Score41.6% | Versus best verified row Best verified: GPT-6 Astra · 96% | Gap54.4 behind | Weight3% ref. weight | |
| MMLUMassive Multitask Language Understanding | Score80.3% | Versus best verified row Best verified: o1 · 91.8% | Gap11.5 behind | WeightDisplay only |
Multilingual1 row
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| MGSMMultilingual Grade School Math | Score85.6% | Versus best verified row Best verified: o4-mini · 93.7% | Gap8.1 behind | WeightDisplay only |
Inst. Following1 row
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| IFEvalInstruction-Following Eval | Score85.9% | Versus best verified row Best verified: LFM2.5-24B-A2B · 96.8% | Gap10.8 behind | WeightDisplay only |
Math1 row
| Benchmark | Score | Versus best verified row | Gap | Weight | Evidence |
|---|---|---|---|---|---|
| AIME 2024American Invitational Mathematics Examination 2024 | Score5.3% | Versus best verified row Best verified: GPT-OSS 120B · 95.8% | Gap90.5 behind | WeightDisplay only |
Bars run 0–100; the dark tick marks the best source-verified value
All 10 rowsLineage
The sequence follows explicit supersedes links. Each score is estimated for that model; a relative can inform a sparse estimate but never sets a floor, so a newer release can score below an earlier one. Scores and prices remain blank when the corresponding public row or first-party rate is unavailable.
Base entry
Claude 3.5 Haiku release history
Radar confirmed these at the source. Use Claude 3.5 Haiku in your work? Explore Radar to follow supported changes and choose your alerts.
Radar
Spec sheet
Each documented value carries its source. Missing fields stay visible as not sourced or not published, rather than disappearing from the page.
- API model ID
- Not published
- Context window
- 200K
- Maximum output
- Not sourced yet
- Knowledge cutoff
- Not sourced yet
- Input modalities
- Not sourced yet
- Output modalities
- Not sourced yet
- Parameters
- Not disclosed by the provider
- Availability
- Not sourced yet
- Cloud regions
- Not tracked yet
- Lifecycle
- Established
- API capabilities
- Tool calling, structured outputs, and batch support are not tracked yet
- Prompt caching
- Not documented in the pricing record
- Self-host
- Weights are not published
- Rate limits
- Not tracked yet
How to read this profile
The visual layer above carries the decisions. These notes preserve the model, ranking, coverage, and family context behind the numbers.
We track Claude 3.5 Haiku, but the public leaderboard excludes this profile until enough non-generated benchmark coverage is available. Only published rows appear above.
Claude 3.5 Haiku is a proprietary model with a 200K context window. No explicit reasoning mode is documented in this profile.
Official exact values from Anthropic model card addendum, October 2024. BenchLM stores gpqa, mmluPro, mmlu, humaneval, sweVerified, aime2024, ifeval, mgsm, bbh, drop. Each value's source and evaluation setting is recorded in row-level source verification. Kept out of rankings because the published results cover few of the benchmarks BenchLM ranks on.
10 of 670 tracked benchmark slots currently have displayable evidence. Missing categories stay blank.
Last updated October 14, 2026. Runtime fields remain blank until a sourced snapshot exists.
Questions
How does Claude 3.5 Haiku perform overall in AI benchmarks?
Claude 3.5 Haiku has 10 source-displayable benchmark rows, but it does not qualify for a public overall rank. The available rows remain visible by category without being converted into a site-wide score. Missing evidence stays blank instead of being estimated from an earlier model.
Is Claude 3.5 Haiku good for knowledge and understanding?
Claude 3.5 Haiku has source-displayable benchmark coverage for knowledge and understanding, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Claude 3.5 Haiku good for coding and programming?
Claude 3.5 Haiku has source-displayable benchmark coverage for coding and programming, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Claude 3.5 Haiku good for mathematics?
Claude 3.5 Haiku has source-displayable benchmark coverage for mathematics, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Claude 3.5 Haiku good for reasoning and logic?
Claude 3.5 Haiku has source-displayable benchmark coverage for reasoning and logic, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Claude 3.5 Haiku good for instruction following?
Claude 3.5 Haiku has source-displayable benchmark coverage for instruction following, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Is Claude 3.5 Haiku good for multilingual tasks?
Claude 3.5 Haiku has source-displayable benchmark coverage for multilingual tasks, but the public category table does not assign it a rank there. The individual rows remain available for inspection. A missing category position means the evidence threshold was not met; it does not convert the model's unmeasured work into a zero.
Does Claude 3.5 Haiku have full benchmark coverage on BenchLM?
No. Claude 3.5 Haiku currently has 10 source-displayable rows across 670 tracked benchmark slots. The profile exposes published, non-generated evidence and leaves missing categories blank until an exact evaluation is available. Coverage describes how much was measured; it is not a penalty added to an individual benchmark result.
What is the context window size of Claude 3.5 Haiku?
Claude 3.5 Haiku has a reported context window of 200K in the exact-model catalog record. The value stays visible, but the profile marks its source link as unavailable instead of presenting it as directly documented. Maximum output length remains separate because providers often publish a different limit.