# ExtractBench: schema-guided enterprise document extraction (LlamaIndex leaderboard) (ExtractBench (LlamaIndex))

> Scores schema-guided extraction from 370 enterprise documents and 4,869 pages on value F1, record completeness, word and page grounding and cost, split by short, medium and long documents.

Canonical page: https://benchlm.ai/benchmarks/extractbench

- Category: [Multimodal & Grounded](/multimodal-grounded)
- Last updated: October 25, 2026 snapshot

## About ExtractBench (LlamaIndex)

- Year: 2026
- Tasks: 370 documents, 4,869 pages, 8 business domains, 67 document types, each with a JSON Schema
- Format: Order-insensitive value F1 overall and by document length; word and page grounding F1; cost and latency
- Difficulty: Short documents are near solved; long record lists and grounding separate systems
- Paper: [ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction](https://arxiv.org/abs/2607.29677)

Each ExtractBench document comes with a JSON Schema; the system returns structured values with source evidence. Value accuracy is an order-insensitive value F1 that counts every record in long lists, so truncating a list costs recall. Grounding is scored at word level and page level against the evidence the system cites. Documents are grouped as short, medium and long, and each row reports cost per page and latency per document. The board covers LlamaExtract, specialized extraction APIs, coding agents such as Codex and Claude Code running the extraction as a task, commercial vision-language models and open-weight models. We mirror the leaderboard CSV from the ExtractBench repository, which the public site loads at runtime, and store each catalog model's best overall value F1 on its row, including runs made through a coding-agent harness. The ExtractBench name is also used by Contextual AI for an unrelated benchmark; this page is LlamaIndex's.

ExtractBench (LlamaIndex) is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (49 configurations)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [LlamaExtract Agentic Plus](https://www.extractbench.ai/#leaderboard) | LlamaExtract · Default · 7.50¢ per page | LlamaIndex | 96.17% |
| 2 | [Reducto Deep Extract](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Default · 6.35¢ per page | Reducto | 96.08% |
| 3 | [Pulse (Effort)](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Effort · 10.50¢ per page | Pulse | 96.07% |
| 4 | [LlamaExtract Agentic](https://www.extractbench.ai/#leaderboard) | LlamaExtract · Default · 3.12¢ per page | LlamaIndex | 95.65% |
| 5 | [Claude Sonnet 5.5](/models/claude-sonnet-5-5) | Coding agent · Sonnet 5.5 Evidence · 5.69¢ per page | Anthropic | 94.68% |
| 6 | [GPT-6 Sol](/models/gpt-6-sol) | Coding agent · GPT-6 Sol Evidence · 9.80¢ per page | OpenAI | 94.57% |
| 7 | [Pulse (Non-effort)](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Non-effort · 3.00¢ per page | Pulse | 94.51% |
| 8 | [LlamaExtract Cost-Effective](https://www.extractbench.ai/#leaderboard) | LlamaExtract · Default · 1.00¢ per page | LlamaIndex | 94.21% |
| 9 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | Coding agent · GPT-5.6 Sol Evidence · 21.65¢ per page | OpenAI | 93.77% |
| 10 | [GPT-5.5](/models/gpt-5-5) | Coding agent · GPT-5.5 · 27.83¢ per page | OpenAI | 93.57% |
| 11 | [GPT-5.5](/models/gpt-5-5) | Coding agent · GPT-5.5 Evidence · 33.63¢ per page | OpenAI | 93.35% |
| 12 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | Coding agent · GPT-5.6 Terra Evidence · 9.45¢ per page | OpenAI | 92.26% |
| 13 | [GPT-6 Astra](/models/gpt-6-astra) | Commercial VLM · Default · 11.09¢ per page | OpenAI | 91.91% |
| 14 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | Coding agent · GPT-5.6 Luna Evidence · 1.20¢ per page | OpenAI | 91.16% |
| 15 | [GPT-6 Luna](/models/gpt-6-luna) | Coding agent · GPT-6 Luna Evidence · 0.97¢ per page | OpenAI | 91.01% |
| 16 | [Qwen3.8-Flash-Next](/models/qwen3-8-flash-next) | Open-weight model · Default · no listed cost | Alibaba | 89.88% |
| 17 | [Qwen3.8-27B](/models/qwen3-8-27b) | Open-weight model · Default · no listed cost | Alibaba | 89.75% |
| 18 | [Extend (Max Context)](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Max Context · 10.00¢ per page | Extend | 88.62% |
| 19 | [Qwen3.6-35B-A3B](/models/qwen3-6-35b-a3b) | Open-weight model · Default · no listed cost | Alibaba | 88.11% |
| 20 | [Qwen3.5-35B-A3B](/models/qwen3-5-35b-a3b) | Open-weight model · Default · no listed cost | Alibaba | 88.00% |
| 21 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | Commercial VLM · Default · 0.32¢ per page | DeepSeek | 87.11% |
| 22 | [Claude Opus 4.8](/models/claude-opus-4-8) | Coding agent · Opus 4.8 · 16.17¢ per page | Anthropic | 87.09% |
| 23 | [Claude Opus 4.8](/models/claude-opus-4-8) | Coding agent · Opus 4.8 Evidence · 63.24¢ per page | Anthropic | 86.84% |
| 24 | [Qwen3.5 9B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Alibaba | 86.10% |
| 25 | [Claude Opus 5.5](/models/claude-opus-5-5) | Coding agent · Opus 5.5 Evidence · 12.14¢ per page | Anthropic | 86.06% |
| 26 | [Extend Extract](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Default · 6.80¢ per page | Extend | 85.72% |
| 27 | [Datalab (Accurate + Balanced)](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Accurate + Balanced · 3.50¢ per page | Datalab | 85.70% |
| 28 | [Reducto Extract](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · Default · 4.57¢ per page | Reducto | 83.79% |
| 29 | [Kimi K3](/models/kimi-k3) | Commercial VLM · Default · 6.75¢ per page | Moonshot AI | 83.17% |
| 30 | [Qwen3.5 4B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Alibaba | 82.43% |
| 31 | [NuExtract3](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | NuMind | 81.02% |
| 32 | [GLM-5.3-Flash](/models/glm-5-3-flash) | Commercial VLM · Default · 0.34¢ per page | Z.AI | 80.75% |
| 33 | [Gemini 3.8 Flash](/models/gemini-3-8-flash) | Commercial VLM · Default · 0.43¢ per page | Google | 80.71% |
| 34 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Commercial VLM · Default · 1.00¢ per page | Google | 79.84% |
| 35 | [Reducto Extract (v4)](https://www.extractbench.ai/#leaderboard) | Specialized extraction API · v4 · 4.59¢ per page | Reducto | 79.58% |
| 36 | [Lift Datalab 9B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Datalab | 78.43% |
| 37 | [GPT-5.4 nano](/models/gpt-5-4-nano) | Commercial VLM · Default · 0.21¢ per page | OpenAI | 74.90% |
| 38 | [Gemma 4 26B A4B](/models/gemma-4-26b-a4b) | Open-weight model · Default · no listed cost | Google | 70.27% |
| 39 | [Gemma 4 E4B](/models/gemma-4-e4b) | Open-weight model · Default · no listed cost | Google | 69.64% |
| 40 | [InternVL3.5 14B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | OpenGVLab | 66.35% |
| 41 | [InternVL3.5 8B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | OpenGVLab | 64.63% |
| 42 | [Qwen3.5 2B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Alibaba | 63.40% |
| 43 | [Kimi-VL A3B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Moonshot AI | 57.77% |
| 44 | [Gemma 4 E2B](/models/gemma-4-e2b) | Open-weight model · Default · no listed cost | Google | 51.85% |
| 45 | [NuExtract 2.0 8B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | NuMind | 49.42% |
| 46 | [Qwen3.5 0.8B](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Alibaba | 46.63% |
| 47 | [MiniCPM-V 4.5](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | OpenBMB | 45.85% |
| 48 | [Gemma 4 12B](/models/gemma-4-12b) | Open-weight model · Default · no listed cost | Google | 45.39% |
| 49 | [GLM-4.6V Flash](https://www.extractbench.ai/#leaderboard) | Open-weight model · Default · no listed cost | Z.AI | 45.19% |

## ExtractBench (Kaggle)

Evaluation results published by [Kaggle Benchmarks](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard). 

Source snapshot: 2026-10-20 source capture.

Published Kaggle aggregates only. Model configurations and evaluation dates stay attached to each result. These results do not enter overall or category rankings. Task data, prompts, answers, and per-case outputs are not included.

- [Kaggle leaderboard and methodology](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard)
- [Published leaderboard data](https://www.kaggle.com/api/v1/benchmarks/llamaindex-org/extractbench-leaderboard/leaderboard)
- [ExtractBench project](https://www.extractbench.ai/)
- [ExtractBench evaluation code](https://github.com/run-llama/ExtractBench)
- [ExtractBench paper](https://arxiv.org/abs/2607.29677)
- [Apache 2.0 license shown by Kaggle](https://www.apache.org/licenses/LICENSE-2.0)

| Source model | Configuration | ExtractBench overall score |
|---|---|---|
| [GPT-5.6 Sol](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 91.0% |
| [GPT-6 Astra](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-09-23 | 90.3% |
| [GPT-5.6 Terra](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 90.0% |
| [Gemini 4 Argon](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-10-09 | 89.8% |
| [GPT-5.5](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 89.1% |
| [Claude Sonnet 5.5](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-09-30 | 89.0% |
| [Gemini 3 Flash Preview](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 89.0% |
| [GPT-5.6 Luna](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 89.0% |
| [Claude Opus 5](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 88.8% |
| [Claude Sonnet 5](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-21 | 88.6% |
| [Gemini 3.1 Pro Preview](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-21 | 87.6% |
| [Gemini 3.8 Flash](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-09-02 | 87.1% |
| [GPT-5.4 mini](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 86.4% |
| [Gemini 3.5 Flash](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 85.6% |
| [Gemini 3.7 Flash](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 85.5% |
| [Claude Haiku 4.5](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 81.6% |
| [Gemma 4 31B](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 79.9% |
| [Gemini 3.5 Flash-Lite](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 79.4% |
| [Gemma 4 26B A4B](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 77.8% |
| [GPT-5.4 nano](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-17 | 68.9% |
| [Gemini 3.6 Flash](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) | Task version 6; Evaluated 2026-08-18 | 66.3% |

## FAQ

### What does ExtractBench measure?

ExtractBench gives a system a document and a JSON Schema and scores the values it returns. Value F1 counts every record, so a truncated list loses recall. Grounding checks whether the cited evidence points at the right words and pages. Cost per page and latency per document are reported with every row.

### Why do coding agents appear on a document benchmark?

LlamaIndex ran Codex and Claude Code as extraction systems: the agent reads the document, writes code where it helps and returns the schema. Those rows are accurate but cost far more per page than an extraction API. We map each run to the model it used and name the harness in the row note.

### Is this the same ExtractBench as Contextual AI's?

No. Contextual AI publishes an unrelated benchmark under the same name. This page mirrors LlamaIndex's ExtractBench, with its own dataset, schema-guided task and leaderboard. Kaggle Benchmarks also reruns the LlamaIndex task through its own harness; that board is shown as a separate table on this page because its numbers differ.