# DocVQA

> DocVQA asks questions about a single document image. This page shows four historical Anthropic reports on the test split, with ANLS, zero-shot setup, report dates, and source cohorts kept visible.

Canonical page: https://benchlm.ai/benchmarks/docvqa

- Category: [Multimodal & Grounded](/multimodal-grounded)
- Last updated: March and June 2024 provider reports; checked October 1, 2026

Evaluation results published by [Anthropic](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf). Historical DocVQA test results reported by Anthropic in March and June 2024.

Initial Claude 3.5 Sonnet has the highest reported ANLS here at 95.2%, in the June 2024 report. The other three results come from the March Claude 3 report. The model card identifies all four as zero-shot DocVQA test results. These are provider reports with separate cohorts; matching metric labels do not establish a shared run or harness.

ANLS scores an answer by its normalized edit similarity to the reference text. It can give partial credit for close answers, so these percentages should not be read as exact-match accuracy. The benchmark asks questions about a single document image.

Each row keeps its source, report date, and setup. March results use temperature 0; the June report does not disclose temperature. API snapshots, DocVQA image processing, scorer versions, evaluation dates, and evaluated question counts are unreported. Profile links refer to named generations only; the June Sonnet 3.5 row preserves its initial edition.

These March and June 2024 provider cohorts do not establish a shared run or harness; the highest reported ANLS applies only to the four rows shown.

- [March Claude 3 report](https://www.anthropic.com/news/claude-3-family)
- [Initial Sonnet 3.5 report](https://www.anthropic.com/news/claude-3-5-sonnet)
- [Official model card: Table 3 and June Table 2](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf)
- [Original DocVQA task](https://www.docvqa.org/datasets/docvqa)

## About DocVQA

- Year: 2021
- Tasks: Questions about single document images; provider run counts undisclosed
- Format: Text answers drawn from document images
- Paper: [DocVQA: A Dataset for VQA on Document Images](https://www.docvqa.org/datasets/docvqa)

ANLS uses normalized edit similarity rather than exact-match accuracy. The March Claude 3 cohort and initial June Claude 3.5 Sonnet cohort are separate provider reports. They share a task, split, metric, and zero-shot label; the sources do not establish identical API snapshots, image processing, or scorer versions. Results stay display only.

DocVQA is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (4 own-provider reports)

| Source model edition | ANLS (%) | Report date | Setup | Original source |
|---|---|---|---|---|
| [Claude 3.5 Sonnet (initial June 2024)](https://www.anthropic.com/news/claude-3-5-sonnet) | 95.2% | 2024-06-21 | 0-shot; test; ANLS; temperature undisclosed; Initial Claude 3.5 Sonnet, June 2024 report | [Provider report](https://www.anthropic.com/news/claude-3-5-sonnet); [Claude 3.5 Sonnet addendum, Table 2, PDF page 44](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf) |
| [Claude 3 Sonnet](https://www.anthropic.com/news/claude-3-family) | 89.5% | 2024-03-04 | 0-shot; T=0; test; ANLS; Claude 3 family, March 2024 report | [Provider report](https://www.anthropic.com/news/claude-3-family); [Claude 3 model card, Table 3, PDF page 8](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf) |
| [Claude 3 Opus](/models/claude-3-opus) | 89.3% | 2024-03-04 | 0-shot; T=0; test; ANLS; Claude 3 family, March 2024 report | [Provider report](https://www.anthropic.com/news/claude-3-family); [Claude 3 model card, Table 3, PDF page 8](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf) |
| [Claude 3 Haiku](/models/claude-3-haiku) | 88.8% | 2024-03-04 | 0-shot; T=0; test; ANLS; Claude 3 family, March 2024 report | [Provider report](https://www.anthropic.com/news/claude-3-family); [Claude 3 model card, Table 3, PDF page 8](https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404/Claude%203%20Model%20Card.pdf) |

ANLS measures answer similarity and can give partial credit. Report dates are publication labels; evaluation dates and API snapshots are undisclosed. These two provider cohorts do not establish a shared run or harness.

## FAQ

### What does DocVQA measure?

DocVQA asks questions about a single document image. This page shows four historical Anthropic reports on the test split, with ANLS, zero-shot setup, report dates, and source cohorts kept visible.

### Which source row has the highest reported DocVQA value?

Claude 3.5 Sonnet (initial June 2024) has the highest reported DocVQA value at 95.2%. These March and June 2024 provider cohorts do not establish a shared run or harness; the highest reported ANLS applies only to the four rows shown.

### How many models are evaluated on DocVQA?

The March and June 2024 provider reports; checked October 1, 2026 contains 4 AI models.

### Does DocVQA affect BenchLM's overall score?

Not directly. DocVQA is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.