# FACTS Multimodal

> FACTS Multimodal checks whether a model answers a question about an image without leaving out essential facts or contradicting them. It is the image part of the FACTS suite.

Canonical page: https://benchlm.ai/benchmarks/facts-multimodal

- Category: [Multimodal & Grounded](/multimodal-grounded)
- Last updated: Latest evaluation November 12, 2026

Evaluation results published by [Google DeepMind and Google Research](https://www.kaggle.com/benchmarks/google/facts-multimodal). undefined

Each example pairs an image with a written request. A judge model compares the answer with a human-written rubric of essential and optional facts. An answer counts as factual only when it covers the essential facts and contradicts none of the rubric. Kaggle publishes the overall score with separate public-set and private-set scores, plus Coverage and Factuality columns. The benchmark has about 1,500 examples: 711 public and 811 held out.

The judge is an automated model, validated against human raters but not equal to them. This is Kaggle benchmark version 2 at task revision 8; scores from the earlier version are not interchangeable. It is separate from the provider-reported Facts-VLM figures.

These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference.

- [Kaggle leaderboard and Apache 2.0 declaration](https://www.kaggle.com/benchmarks/google/facts-multimodal)
- [FACTS Benchmark Suite paper](https://arxiv.org/abs/2512.10791)
- [Published leaderboard data](https://www.kaggle.com/api/v1/benchmarks/google/facts-multimodal/leaderboard)

## About FACTS Multimodal

- Year: 2025
- Tasks: About 1,500 image questions: 711 public and 811 held out
- Format: Rubric-judged answers about images
- Difficulty: Image-grounded factual answering

Each example pairs an image with a written request. A judge model compares the answer with a human-written rubric of essential and optional facts. An answer counts as factual only when it covers the essential facts and contradicts none of the rubric. Kaggle publishes the overall score with separate public-set and private-set scores, plus Coverage and Factuality columns. The benchmark has about 1,500 examples: 711 public and 811 held out. These results are display-only and do not enter overall or category rankings.

FACTS Multimodal is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (22 source rows)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [Gemini 4 Carbon](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 57.5%; Private Score 62.5%; Coverage 75.3%; Factuality 72.7%; Evaluated 2026-11-05 | — | 60.2% |
| 2 | [Gemini 3.8 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 57.5%; Private Score 57.7%; Coverage 73.7%; Factuality 71.7%; Evaluated 2026-11-05 | — | 57.6% |
| 3 | [Claude Opus 5.5](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 55.2%; Private Score 57.5%; Coverage 72.9%; Factuality 71.1%; Evaluated 2026-11-05 | — | 56.4% |
| 4 | [GPT-6 Astra](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 54.3%; Private Score 56.0%; Coverage 63.9%; Factuality 80.6%; Evaluated 2026-11-05 | — | 55.2% |
| 5 | [Gemini 4 Argon](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 51.4%; Private Score 56.5%; Coverage 71.9%; Factuality 67.7%; Evaluated 2026-11-05 | — | 54.2% |
| 6 | [Gemini 3.6 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 53.1%; Private Score 54.1%; Coverage 71.4%; Factuality 67.4%; Evaluated 2026-11-12 | — | 53.7% |
| 7 | [GPT-6.1 Sol](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 51.7%; Private Score 53.3%; Coverage 62.0%; Factuality 78.6%; Evaluated 2026-11-05 | — | 52.6% |
| 8 | [Claude Sonnet 5.5](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 50.3%; Private Score 50.6%; Coverage 70.4%; Factuality 65.3%; Evaluated 2026-11-05 | — | 50.4% |
| 9 | [Gemini 3.1 Pro Preview](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 49.4%; Private Score 51.3%; Coverage 69.1%; Factuality 65.5%; Evaluated 2026-11-05 | — | 50.4% |
| 10 | [Gemini 3 Flash Preview](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 47.9%; Private Score 51.2%; Coverage 68.5%; Factuality 63.6%; Evaluated 2026-11-12 | — | 49.7% |
| 11 | [GPT-6.1 Terra](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 46.2%; Private Score 51.0%; Coverage 58.2%; Factuality 82.9%; Evaluated 2026-11-12 | — | 48.8% |
| 12 | [GPT-5.6 Sol](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 45.2%; Private Score 47.4%; Coverage 59.5%; Factuality 69.5%; Evaluated 2026-11-11 | — | 46.4% |
| 13 | [GPT-6 Sol](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 41.7%; Private Score 45.9%; Coverage 54.2%; Factuality 78.3%; Evaluated 2026-11-11 | — | 43.9% |
| 14 | [Gemma 4 31B](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 41.2%; Private Score 43.3%; Coverage 59.9%; Factuality 57.6%; Evaluated 2026-11-05 | — | 42.3% |
| 15 | [Grok 4.7](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 38.9%; Private Score 42.6%; Coverage 54.9%; Factuality 65.4%; Evaluated 2026-11-05 | — | 40.9% |
| 16 | [GPT-5.6 Terra](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 37.9%; Private Score 41.6%; Coverage 54.3%; Factuality 61.6%; Evaluated 2026-11-11 | — | 39.9% |
| 17 | [Gemma 4 26B A4B](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 37.1%; Private Score 41.6%; Coverage 59.1%; Factuality 55.7%; Evaluated 2026-11-05 | — | 39.5% |
| 18 | [Grok 4.6](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 37.5%; Private Score 40.8%; Coverage 60.7%; Factuality 53.7%; Evaluated 2026-11-12 | — | 39.3% |
| 19 | [Claude Haiku 5.5](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 37.6%; Private Score 40.1%; Coverage 63.2%; Factuality 54.4%; Evaluated 2026-11-05 | — | 38.9% |
| 20 | [GPT-6 Luna](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 35.8%; Private Score 41.1%; Coverage 51.7%; Factuality 68.7%; Evaluated 2026-11-05 | — | 38.7% |
| 21 | [GPT-5.6 Luna](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 33.4%; Private Score 40.7%; Coverage 53.1%; Factuality 57.2%; Evaluated 2026-11-11 | — | 37.4% |
| 22 | [Gemini 3.5 Flash-Lite](https://www.kaggle.com/benchmarks/google/facts-multimodal) | Public Score 33.8%; Private Score 38.1%; Coverage 58.8%; Factuality 51.4%; Evaluated 2026-11-05 | — | 36.1% |

## FACTS Multimodal v1 (archived)

Evaluation results published by [Google DeepMind and Google Research](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1). 

Source snapshot: Latest evaluation October 5, 2026.

Scores are shown as published percentages. Reasoning settings, secondary metrics and evaluation dates stay on each row. Missing results are not zero.

- [Kaggle version 1 leaderboard and Apache 2.0 declaration](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1)
- [FACTS Benchmark Suite paper](https://arxiv.org/abs/2512.10791)
- [Published leaderboard data](https://www.kaggle.com/api/v1/benchmarks/google/facts-multimodal/versions/1/leaderboard)

| Source model | Configuration | Factuality score (%) |
|---|---|---|
| [Gemini 3.8 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 72.5%; Factuality 60.6%; Evaluated 2026-09-04 | 50.7% |
| [Gemini 3.7 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 71.0%; Factuality 61.5%; Evaluated 2026-08-15 | 49.8% |
| [GPT-6.1 Sol](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 62.6%; Factuality 68.3%; Evaluated 2026-10-05 | 48.5% |
| [Claude Opus 5.5](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 73.2%; Factuality 57.6%; Evaluated 2026-09-25 | 48.4% |
| [GPT-6 Astra](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 63.0%; Factuality 69.6%; Evaluated 2026-09-12 | 47.7% |
| [Gemini 3.6 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 70.9%; Factuality 57.5%; Evaluated 2026-07-23 | 47.2% |
| [Gemini 2.5 Pro](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 67.7%; Factuality 58.8%; Evaluated 2025-12-07 | 46.9% |
| [GPT-5.5](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 61.3%; Factuality 64.8% | 45.8% |
| [GPT-5](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 59.9%; Factuality 64.7%; Evaluated 2025-12-07 | 44.1% |
| [GPT-6 Sol](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 54.8%; Factuality 70.8%; Evaluated 2026-09-25 | 43.1% |
| [GPT-5.1](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 58.3%; Factuality 65.0%; Evaluated 2025-12-07 | 41.8% |
| [GPT-5 mini](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 58.8%; Factuality 65.1%; Evaluated 2025-12-07 | 41.5% |
| [Gemini 3.5 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 70.2%; Factuality 50.3% | 41.4% |
| [Gemini 3.1 Pro Preview](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 67.6%; Factuality 52.5% | 41.3% |
| [Gemini 3 Flash Preview](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 68.0%; Factuality 51.6% | 41.3% |
| [Gemini 2.5 Flash](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 64.8%; Factuality 53.9%; Evaluated 2025-12-07 | 41.0% |
| [GPT-5.4](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 58.3%; Factuality 59.9% | 41.0% |
| [GPT-4.1](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 57.3%; Factuality 62.2%; Evaluated 2025-12-07 | 40.1% |
| [o3](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 62.2%; Factuality 55.7%; Evaluated 2025-12-07 | 39.9% |
| [Claude Opus 4.8](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 64.0%; Factuality 54.5% | 39.8% |
| [GPT-5.2](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 56.8%; Factuality 62.1% | 39.7% |
| [Gemini 3.1 Flash-Lite Preview](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 62.3%; Factuality 51.2% | 39.4% |
| [Claude Opus 4.5](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 62.1%; Factuality 51.1%; Evaluated 2025-12-07 | 39.2% |
| [Claude Opus 4.6](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 64.7%; Factuality 46.3% | 36.8% |
| [Claude Opus 4.7](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 64.4%; Factuality 48.6% | 35.8% |
| [Gemini 3.5 Flash-Lite](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 59.5%; Factuality 47.8%; Evaluated 2026-07-23 | 35.8% |
| [o4 mini](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 59.1%; Factuality 47.6%; Evaluated 2025-12-07 | 34.5% |
| [Claude Opus 4.1](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 54.6%; Factuality 49.2%; Evaluated 2025-12-07 | 33.1% |
| [Claude Sonnet 4.6](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 62.4%; Factuality 42.8% | 32.6% |
| [Claude Sonnet 4.5 (thinking)](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 56.0%; Factuality 42.6%; Evaluated 2025-12-07 | 31.1% |
| [Claude Sonnet 4](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 50.2%; Factuality 46.2%; Evaluated 2025-12-07 | 28.6% |
| [Grok 4](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 58.2%; Factuality 34.4%; Evaluated 2025-12-07 | 25.7% |
| [Gemma 4 31B](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 50.0%; Factuality 34.7% | 24.7% |
| [Gemma 3 27B](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 50.5%; Factuality 32.6%; Evaluated 2025-12-07 | 24.6% |
| [Gemma 4 26B A4B](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 44.7%; Factuality 35.8% | 23.9% |
| [Mistral Small 3.1](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 42.8%; Factuality 34.3%; Evaluated 2025-12-07 | 21.2% |
| [Gemma 3 12B](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 47.7%; Factuality 28.2%; Evaluated 2025-12-07 | 20.8% |
| [grok-4-fast-reasoning](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 51.0%; Factuality 25.2%; Evaluated 2025-12-07 | 17.7% |
| [Gemma 3 4B](https://www.kaggle.com/benchmarks/google/facts-multimodal/versions/1) | Coverage 40.9%; Factuality 18.2%; Evaluated 2025-12-07 | 13.6% |

## FAQ

### What does FACTS Multimodal measure?

FACTS Multimodal checks whether a model answers a question about an image without leaving out essential facts or contradicting them. It is the image part of the FACTS suite.

### Which source row has the highest reported FACTS Multimodal value?

Gemini 4 Carbon has the highest reported FACTS Multimodal value at 60.2%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference.

### How many models are evaluated on FACTS Multimodal?

The Latest evaluation November 12, 2026 contains 22 AI models.

### Does FACTS Multimodal affect BenchLM's overall score?

Not directly. FACTS Multimodal is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.