DocVQA
We show this table for reference; we do not rank on it.
DocVQA asks questions about a single document image. This page shows four historical Anthropic reports on the test split, with ANLS, zero-shot setup, report dates, and source cohorts kept visible.
Evaluation results published by Anthropic. Historical DocVQA test results reported by Anthropic in March and June 2024.
Reported ANLS on DocVQA — March and June 2024 provider reports; checked October 1, 2026
We mirror the published reported anls view for DocVQA. Claude 3.5 Sonnet (initial June 2024) has the highest reported value at 95.2%. These March and June 2024 provider cohorts do not establish a shared run or harness; the highest reported ANLS applies only to the four rows shown. We do not use these results to rank models overall.
Claude 3.5 Sonnet (initial June 2024)
Anthropic
Initial June 2024 edition
Claude 3 Sonnet
Anthropic
March 2024 Claude 3 report
Claude 3 Opus
Anthropic
March 2024 Claude 3 report
4 own-provider reportsMultimodal & GroundedHistorical provider cohortsDisplay onlyUpdated March and June 2024 provider reports; checked October 1, 2026
| Source model edition | ANLS (%) | Report date | Setup and source | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| Claude 3.5 Sonnet (initial June 2024) | 95.2% | 0-shot; test; ANLS; temperature undisclosed Initial Claude 3.5 Sonnet, June 2024 report Original provider report · Claude 3.5 Sonnet addendum, Table 2, PDF page 44 | Not reported | Unknown | |
| Claude 3 Sonnet | 89.5% | 0-shot; T=0; test; ANLS Claude 3 family, March 2024 report Original provider report · Claude 3 model card, Table 3, PDF page 8 | Not reported | Unknown | |
| Claude 3 OpusNamed generation reference | 89.3% | 0-shot; T=0; test; ANLS Claude 3 family, March 2024 report Original provider report · Claude 3 model card, Table 3, PDF page 8 | Not reported | Closed | |
| Claude 3 HaikuNamed generation reference | 88.8% | 0-shot; T=0; test; ANLS Claude 3 family, March 2024 report Original provider report · Claude 3 model card, Table 3, PDF page 8 | Not reported | Closed |
ANLS measures answer similarity and can give partial credit. Report dates are publication labels; evaluation dates and API snapshots are undisclosed. These two provider cohorts do not establish a shared run or harness.
How we show these DocVQA reports
Initial Claude 3.5 Sonnet has the highest reported ANLS here at 95.2%, in the June 2024 report. The other three results come from the March Claude 3 report. The model card identifies all four as zero-shot DocVQA test results. These are provider reports with separate cohorts; matching metric labels do not establish a shared run or harness.
ANLS scores an answer by its normalized edit similarity to the reference text. It can give partial credit for close answers, so these percentages should not be read as exact-match accuracy. The benchmark asks questions about a single document image.
Each row keeps its source, report date, and setup. March results use temperature 0; the June report does not disclose temperature. API snapshots, DocVQA image processing, scorer versions, evaluation dates, and evaluated question counts are unreported. Profile links refer to named generations only; the June Sonnet 3.5 row preserves its initial edition.
Snapshot
These March and June 2024 provider cohorts do not establish a shared run or harness; the highest reported ANLS applies only to the four rows shown. The table retains each source configuration and its reported value.
4 own-provider reports are shown for DocVQA. The benchmark falls in the Multimodal & Grounded category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. DocVQA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About DocVQA
Year
2021
Tasks
Questions about single document images; provider run counts undisclosed
Format
Text answers drawn from document images
ANLS uses normalized edit similarity rather than exact-match accuracy. The March Claude 3 cohort and initial June Claude 3.5 Sonnet cohort are separate provider reports. They share a task, split, metric, and zero-shot label; the sources do not establish identical API snapshots, image processing, or scorer versions. Results stay display only.
Freshness and provenance
Version
March and June 2024 own-provider reports
Refresh cadence
Pinned historical reports
Staleness state
Historical provider cohorts
Question availability
Public upstream; no task contents mirrored
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does DocVQA measure?
DocVQA asks questions about a single document image. This page shows four historical Anthropic reports on the test split, with ANLS, zero-shot setup, report dates, and source cohorts kept visible.
Which source row has the highest reported DocVQA value?
Claude 3.5 Sonnet (initial June 2024) has the highest reported DocVQA value at 95.2%. These March and June 2024 provider cohorts do not establish a shared run or harness; the highest reported ANLS applies only to the four rows shown. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on DocVQA?
The March and June 2024 provider reports; checked October 1, 2026 snapshot contains 4 AI models.
Does 95.2% ANLS mean 95.2% of answers were exactly correct?
ANLS can give partial credit when an answer is close to the reference text. It is a similarity metric, so a 95.2% ANLS score does not establish 95.2% exact-match accuracy.
Do these rows represent current Claude models?
These are March and June 2024 reports. The 95.2% row belongs to the initial Claude 3.5 Sonnet edition. Profile links identify the named generation for navigation; the sources do not disclose evaluated API snapshots.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 5,600+ readers.
One email each week. Unsubscribe anytime.