FACTS Multimodal
We show this table for reference; we do not rank on it.
FACTS Multimodal checks whether a model answers a question about an image without leaving out essential facts or contradicting them. It is the image part of the FACTS suite.
Evaluation results published by Google DeepMind and Google Research. How Kaggle Benchmarks reruns tests.
Factuality score (%) on FACTS Multimodal v2 — Latest evaluation November 12, 2026
We mirror the published factuality score (%) view for FACTS Multimodal v2. Gemini 4 Carbon has the highest reported value at 60.2%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. We do not use these results to rank models overall.
Gemini 4 Carbon
Public Score 57.5%; Private Score 62.5%; Coverage 75.3%; Factuality 72.7%; Evaluated 2026-11-05
Gemini 3.8 Flash
Public Score 57.5%; Private Score 57.7%; Coverage 73.7%; Factuality 71.7%; Evaluated 2026-11-05
Claude Opus 5.5
Public Score 55.2%; Private Score 57.5%; Coverage 72.9%; Factuality 71.1%; Evaluated 2026-11-05
22 source rowsMultimodal & GroundedCurrentDisplay onlyUpdated Latest evaluation November 12, 2026
Factuality score (%) table (22 source rows)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Gemini 4 CarbonCreator not reportedPublic Score 57.5%; Private Score 62.5%; Coverage 75.3%; Factuality 72.7%; Evaluated 2026-11-05 | 60.2% | Not reported | Unknown |
| 2 | Gemini 3.8 FlashCreator not reportedPublic Score 57.5%; Private Score 57.7%; Coverage 73.7%; Factuality 71.7%; Evaluated 2026-11-05 | 57.6% | Not reported | Unknown |
| 3 | Claude Opus 5.5Creator not reportedPublic Score 55.2%; Private Score 57.5%; Coverage 72.9%; Factuality 71.1%; Evaluated 2026-11-05 | 56.4% | Not reported | Unknown |
| 4 | GPT-6 AstraCreator not reportedPublic Score 54.3%; Private Score 56.0%; Coverage 63.9%; Factuality 80.6%; Evaluated 2026-11-05 | 55.2% | Not reported | Unknown |
| 5 | Gemini 4 ArgonCreator not reportedPublic Score 51.4%; Private Score 56.5%; Coverage 71.9%; Factuality 67.7%; Evaluated 2026-11-05 | 54.2% | Not reported | Unknown |
| 6 | Gemini 3.6 FlashCreator not reportedPublic Score 53.1%; Private Score 54.1%; Coverage 71.4%; Factuality 67.4%; Evaluated 2026-11-12 | 53.7% | Not reported | Unknown |
| 7 | GPT-6.1 SolCreator not reportedPublic Score 51.7%; Private Score 53.3%; Coverage 62.0%; Factuality 78.6%; Evaluated 2026-11-05 | 52.6% | Not reported | Unknown |
| 8 | Claude Sonnet 5.5Creator not reportedPublic Score 50.3%; Private Score 50.6%; Coverage 70.4%; Factuality 65.3%; Evaluated 2026-11-05 | 50.4% | Not reported | Unknown |
| 9 | Gemini 3.1 Pro PreviewCreator not reportedPublic Score 49.4%; Private Score 51.3%; Coverage 69.1%; Factuality 65.5%; Evaluated 2026-11-05 | 50.4% | Not reported | Unknown |
| 10 | Gemini 3 Flash PreviewCreator not reportedPublic Score 47.9%; Private Score 51.2%; Coverage 68.5%; Factuality 63.6%; Evaluated 2026-11-12 | 49.7% | Not reported | Unknown |
| 11 | GPT-6.1 TerraCreator not reportedPublic Score 46.2%; Private Score 51.0%; Coverage 58.2%; Factuality 82.9%; Evaluated 2026-11-12 | 48.8% | Not reported | Unknown |
| 12 | GPT-5.6 SolCreator not reportedPublic Score 45.2%; Private Score 47.4%; Coverage 59.5%; Factuality 69.5%; Evaluated 2026-11-11 | 46.4% | Not reported | Unknown |
| 13 | GPT-6 SolCreator not reportedPublic Score 41.7%; Private Score 45.9%; Coverage 54.2%; Factuality 78.3%; Evaluated 2026-11-11 | 43.9% | Not reported | Unknown |
| 14 | Gemma 4 31BCreator not reportedPublic Score 41.2%; Private Score 43.3%; Coverage 59.9%; Factuality 57.6%; Evaluated 2026-11-05 | 42.3% | Not reported | Unknown |
| 15 | Grok 4.7Creator not reportedPublic Score 38.9%; Private Score 42.6%; Coverage 54.9%; Factuality 65.4%; Evaluated 2026-11-05 | 40.9% | Not reported | Unknown |
| 16 | GPT-5.6 TerraCreator not reportedPublic Score 37.9%; Private Score 41.6%; Coverage 54.3%; Factuality 61.6%; Evaluated 2026-11-11 | 39.9% | Not reported | Unknown |
| 17 | Gemma 4 26B A4BCreator not reportedPublic Score 37.1%; Private Score 41.6%; Coverage 59.1%; Factuality 55.7%; Evaluated 2026-11-05 | 39.5% | Not reported | Unknown |
| 18 | Grok 4.6Creator not reportedPublic Score 37.5%; Private Score 40.8%; Coverage 60.7%; Factuality 53.7%; Evaluated 2026-11-12 | 39.3% | Not reported | Unknown |
| 19 | Claude Haiku 5.5Creator not reportedPublic Score 37.6%; Private Score 40.1%; Coverage 63.2%; Factuality 54.4%; Evaluated 2026-11-05 | 38.9% | Not reported | Unknown |
| 20 | GPT-6 LunaCreator not reportedPublic Score 35.8%; Private Score 41.1%; Coverage 51.7%; Factuality 68.7%; Evaluated 2026-11-05 | 38.7% | Not reported | Unknown |
| 21 | GPT-5.6 LunaCreator not reportedPublic Score 33.4%; Private Score 40.7%; Coverage 53.1%; Factuality 57.2%; Evaluated 2026-11-11 | 37.4% | Not reported | Unknown |
| 22 | Gemini 3.5 Flash-LiteCreator not reportedPublic Score 33.8%; Private Score 38.1%; Coverage 58.8%; Factuality 51.4%; Evaluated 2026-11-05 | 36.1% | Not reported | Unknown |
FACTS Multimodal v1 (archived)
Scores are shown as published percentages. Reasoning settings, secondary metrics and evaluation dates stay on each row. Missing results are not zero. How Kaggle Benchmarks reruns tests.
| Source rank | Source model or configuration | Factuality score (%) | Source setup | Parameters (B) | Open / closed |
|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | 50.7% | Coverage 72.5%; Factuality 60.6%; Evaluated 2026-09-04Original result | Not reported | Unknown |
| 2 | Gemini 3.7 Flash | 49.8% | Coverage 71.0%; Factuality 61.5%; Evaluated 2026-08-15Original result | Not reported | Unknown |
| 3 | GPT-6.1 Sol | 48.5% | Coverage 62.6%; Factuality 68.3%; Evaluated 2026-10-05Original result | Not reported | Unknown |
| 4 | Claude Opus 5.5 | 48.4% | Coverage 73.2%; Factuality 57.6%; Evaluated 2026-09-25Original result | Not reported | Unknown |
| 5 | GPT-6 Astra | 47.7% | Coverage 63.0%; Factuality 69.6%; Evaluated 2026-09-12Original result | Not reported | Unknown |
| 6 | Gemini 3.6 Flash | 47.2% | Coverage 70.9%; Factuality 57.5%; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 7 | Gemini 2.5 Pro | 46.9% | Coverage 67.7%; Factuality 58.8%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 8 | GPT-5.5 | 45.8% | Coverage 61.3%; Factuality 64.8%Original result | Not reported | Unknown |
| 9 | GPT-5 | 44.1% | Coverage 59.9%; Factuality 64.7%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 10 | GPT-6 Sol | 43.1% | Coverage 54.8%; Factuality 70.8%; Evaluated 2026-09-25Original result | Not reported | Unknown |
| 11 | GPT-5.1 | 41.8% | Coverage 58.3%; Factuality 65.0%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 12 | GPT-5 mini | 41.5% | Coverage 58.8%; Factuality 65.1%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 13 | Gemini 3.5 Flash | 41.4% | Coverage 70.2%; Factuality 50.3%Original result | Not reported | Unknown |
| 14 | Gemini 3.1 Pro Preview | 41.3% | Coverage 67.6%; Factuality 52.5%Original result | Not reported | Unknown |
| 15 | Gemini 3 Flash Preview | 41.3% | Coverage 68.0%; Factuality 51.6%Original result | Not reported | Unknown |
| 16 | Gemini 2.5 Flash | 41.0% | Coverage 64.8%; Factuality 53.9%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 17 | GPT-5.4 | 41.0% | Coverage 58.3%; Factuality 59.9%Original result | Not reported | Unknown |
| 18 | GPT-4.1 | 40.1% | Coverage 57.3%; Factuality 62.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 19 | o3 | 39.9% | Coverage 62.2%; Factuality 55.7%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 20 | Claude Opus 4.8 | 39.8% | Coverage 64.0%; Factuality 54.5%Original result | Not reported | Unknown |
| 21 | GPT-5.2 | 39.7% | Coverage 56.8%; Factuality 62.1%Original result | Not reported | Unknown |
| 22 | Gemini 3.1 Flash-Lite Preview | 39.4% | Coverage 62.3%; Factuality 51.2%Original result | Not reported | Unknown |
| 23 | Claude Opus 4.5 | 39.2% | Coverage 62.1%; Factuality 51.1%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 24 | Claude Opus 4.6 | 36.8% | Coverage 64.7%; Factuality 46.3%Original result | Not reported | Unknown |
| 25 | Claude Opus 4.7 | 35.8% | Coverage 64.4%; Factuality 48.6%Original result | Not reported | Unknown |
| 26 | Gemini 3.5 Flash-Lite | 35.8% | Coverage 59.5%; Factuality 47.8%; Evaluated 2026-07-23Original result | Not reported | Unknown |
| 27 | o4 mini | 34.5% | Coverage 59.1%; Factuality 47.6%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 28 | Claude Opus 4.1 | 33.1% | Coverage 54.6%; Factuality 49.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 29 | Claude Sonnet 4.6 | 32.6% | Coverage 62.4%; Factuality 42.8%Original result | Not reported | Unknown |
| 30 | Claude Sonnet 4.5 (thinking) | 31.1% | Coverage 56.0%; Factuality 42.6%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 31 | Claude Sonnet 4 | 28.6% | Coverage 50.2%; Factuality 46.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 32 | Grok 4 | 25.7% | Coverage 58.2%; Factuality 34.4%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 33 | Gemma 4 31B | 24.7% | Coverage 50.0%; Factuality 34.7%Original result | Not reported | Unknown |
| 34 | Gemma 3 27B | 24.6% | Coverage 50.5%; Factuality 32.6%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 35 | Gemma 4 26B A4B | 23.9% | Coverage 44.7%; Factuality 35.8%Original result | Not reported | Unknown |
| 36 | Mistral Small 3.1 | 21.2% | Coverage 42.8%; Factuality 34.3%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 37 | Gemma 3 12B | 20.8% | Coverage 47.7%; Factuality 28.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 38 | grok-4-fast-reasoning | 17.7% | Coverage 51.0%; Factuality 25.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
| 39 | Gemma 3 4B | 13.6% | Coverage 40.9%; Factuality 18.2%; Evaluated 2025-12-07Original result | Not reported | Unknown |
How to read this leaderboard
Use it to compare how reliably models describe and reason about images without inventing details. The Coverage and Factuality columns show whether a model loses points by leaving facts out or by stating wrong ones.
Operator receipt: 22 sourced rows are currently displayable on this page; the highest published score among these 22 source rows is Gemini 4 Carbon at 60.2%, which does not establish a market leader.
Honest limit: The judge is an automated model, validated against human raters but not equal to them. This is Kaggle benchmark version 2 at task revision 8; scores from the earlier version are not interchangeable. It is separate from the provider-reported Facts-VLM figures.
How to read FACTS Multimodal
Each example pairs an image with a written request. A judge model compares the answer with a human-written rubric of essential and optional facts. An answer counts as factual only when it covers the essential facts and contradicts none of the rubric. Kaggle publishes the overall score with separate public-set and private-set scores, plus Coverage and Factuality columns. The benchmark has about 1,500 examples: 711 public and 811 held out.
The judge is an automated model, validated against human raters but not equal to them. This is Kaggle benchmark version 2 at task revision 8; scores from the earlier version are not interchangeable. It is separate from the provider-reported Facts-VLM figures.
Snapshot
These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. The table retains each source configuration and its reported value.
22 source rows are shown for FACTS Multimodal. The benchmark falls in the Multimodal & Grounded category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. FACTS Multimodal is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About FACTS Multimodal
Year
2025
Tasks
About 1,500 image questions: 711 public and 811 held out
Format
Rubric-judged answers about images
Difficulty
Image-grounded factual answering
Each example pairs an image with a written request. A judge model compares the answer with a human-written rubric of essential and optional facts. An answer counts as factual only when it covers the essential facts and contradicts none of the rubric. Kaggle publishes the overall score with separate public-set and private-set scores, plus Coverage and Factuality columns. The benchmark has about 1,500 examples: 711 public and 811 held out. These results are display-only and do not enter overall or category rankings.
Freshness and provenance
Version
FACTS Multimodal 2025
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does FACTS Multimodal measure?
FACTS Multimodal checks whether a model answers a question about an image without leaving out essential facts or contradicting them. It is the image part of the FACTS suite.
Which source row has the highest reported FACTS Multimodal value?
Gemini 4 Carbon has the highest reported FACTS Multimodal value at 60.2%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on FACTS Multimodal?
The Latest evaluation November 12, 2026 snapshot contains 22 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.