# Kaggle Benchmarks: Verified AI Leaderboards Explained

> What Kaggle Benchmarks is, who runs it, how Kaggle reruns labs' benchmarks on its own infrastructure, and the 27 Kaggle leaderboards mirrored on BenchLM as display-only evidence.

Canonical page: https://benchlm.ai/benchmarks/kaggle
- **Source captured:** 2026-10-23
- **Boards mirrored:** 27 (25 current protocols, 2 archived versions)
- **Source rows stored:** 1162 model configurations from the complete public downloads
- **Kaggle organizations:** Cohere Labs, Gert Labs, Google DeepMind, Google DeepMind and Google Research, Kaggle, LlamaIndex, Open Benchmarks (Kaggle reproductions), OpenAI, Rehearsals

Kaggle Benchmarks is Kaggle's leaderboard platform for AI models, launched July 30, 2025 at https://www.kaggle.com/benchmarks. Kaggle does not write the benchmarks. It reruns tests from labs such as Google DeepMind, Meta, IBM Research, and LlamaIndex on its own infrastructure, using the authors' code and settings, and publishes the results. Game Arena followed on August 4, 2025, and Community Benchmarks, which anyone can build with the kaggle-benchmarks Python library, arrived in January 2026.

## What a verified Kaggle result means

A verified row means the number came from a run Kaggle executed, every model on the board ran under the same code and settings, and the row can be downloaded and checked from the public JSON endpoint. It does not mean the benchmark is a good test, that its data is free of contamination, or that Kaggle's settings match a vendor's. Kaggle board pages also hide some rows behind default filters, so the top of the Kaggle page is not always the top row of the full download. SimpleQA Verified and SWE-bench Verified are benchmark names where "verified" describes human-checked questions, a different sense.

## How we treat Kaggle rows

- Display only: no Kaggle board enters the overall or category rankings.
- Source names kept: a row reads as Kaggle prints it and is never joined to a model profile by name alone.
- Aggregates, not tasks: scores, configurations, dates, and intervals are stored; prompts, questions, answers, documents, and videos stay with the creators.
- Refreshed from the download: a refresh refuses to overwrite a board whose metrics or task versions changed.

## Kaggle leaderboards on BenchLM

### Factuality and search

Google DeepMind and Google Research publish the FACTS boards through Kaggle. The suite average, document grounding, and search-assisted answering are separate tests.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [FACTS Benchmark Suite](/benchmarks/facts) | Google DeepMind and Google Research | Published average (%) | 40 | October 13, 2026 | [google/facts](https://www.kaggle.com/benchmarks/google/facts) |
| [FACTS Grounding v2](/benchmarks/facts-grounding) | Google DeepMind and Google Research | Grounding score (%) | 55 | October 13, 2026 | [google/facts-grounding](https://www.kaggle.com/benchmarks/google/facts-grounding) |
| [FACTS Search V2](/benchmarks/facts-search) | Google DeepMind and Google Research | Published score (%) | 22 | October 9, 2026 | [google/facts-search](https://www.kaggle.com/benchmarks/google/facts-search) |

### Documents and video

Enterprise document parsing, schema-guided extraction, and hour-long video questions.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [ParseBench (Kaggle)](/benchmarks/parsebench) | LlamaIndex | ParseBench score | 34 | 2026-10-23 source capture | [llamaindex-org/parsebench](https://www.kaggle.com/benchmarks/llamaindex-org/parsebench) |
| [ExtractBench (Kaggle)](/benchmarks/extractbench) | LlamaIndex | ExtractBench overall score | 21 | 2026-10-20 source capture | [llamaindex-org/extractbench-leaderboard](https://www.kaggle.com/benchmarks/llamaindex-org/extractbench-leaderboard) |
| [VideoQA](/benchmarks/video-qa) | Google DeepMind | Top-1 accuracy (%) | 20 | October 9, 2026 | [deepmind/video-qa](https://www.kaggle.com/benchmarks/deepmind/video-qa) |

### Scientific coding

Kaggle runs SciCode as four tracks. Main problems and decomposed subproblems, each with and without the scientists’ background notes, are separate protocols and are not averaged here.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [SciCode Main Standard (Kaggle)](/benchmarks/scicode-main-standard-kaggle) | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | [open-benchmarks/scicode](https://www.kaggle.com/benchmarks/open-benchmarks/scicode) |
| [SciCode Main With Background (Kaggle)](/benchmarks/scicode-main-with-background-kaggle) | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | [open-benchmarks/scicode](https://www.kaggle.com/benchmarks/open-benchmarks/scicode) |
| [SciCode Subproblem Standard (Kaggle)](/benchmarks/scicode-subproblem-standard-kaggle) | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | [open-benchmarks/scicode](https://www.kaggle.com/benchmarks/open-benchmarks/scicode) |
| [SciCode Subproblem With Background (Kaggle)](/benchmarks/scicode-subproblem-with-background-kaggle) | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | [open-benchmarks/scicode](https://www.kaggle.com/benchmarks/open-benchmarks/scicode) |

### Multilingual and math

A 16-language MMLU average from Cohere Labs and the Bengali split of MGSM.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [Global MMLU Lite (Kaggle)](/benchmarks/global-mmlu-lite-kaggle) | Cohere Labs | Kaggle average | 45 | Source retrieved 2026-10-20T15:19:04.746Z | [cohere-labs/global-mmlu-lite](https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite) |
| [MGSM Bengali (Kaggle)](/benchmarks/mgsm-bengali) | Open Benchmarks (Kaggle reproductions) | Accuracy | 46 | 2026-10-20 source capture | [open-benchmarks/mgsm-bengali](https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-bengali) |

### Games, behavior, and agents

Game Arena is a pooled rating across strategic games, not a percentage. Human Behavior Bench and Adversarial Customer Service are simulations with their own scoring.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [Game Arena](/benchmarks/game-arena) | Kaggle | Pooled Bradley-Terry rating | 39 | October 19, 2026 | [kaggle/game-arena](https://www.kaggle.com/benchmarks/kaggle/game-arena) |
| [Human Behavior Bench](/benchmarks/human-behavior-bench) | Rehearsals | Distribution agreement | 11 | 2026-10-20 source capture | [rehearsals/human-behavior-bench](https://www.kaggle.com/benchmarks/rehearsals/human-behavior-bench) |
| [Adversarial Customer Service](/benchmarks/adversarial-customer-service) | Gert Labs | Simulation score | 11 | Source retrieved 2026-10-20T15:19:04.675Z | [gert-labs/adversarial-customer-service](https://www.kaggle.com/benchmarks/gert-labs/adversarial-customer-service) |

### Reproductions of benchmarks we already track

Kaggle reruns of GPQA, AIME 2025, MMLU, MMLU-Pro, SimpleQA, SimpleQA Verified, and MATH-500. Each sits on the existing benchmark page as its own source table beside the provider, Vals, Artificial Analysis, and Mercor rows, so there is no second page for the same test.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [GPQA Diamond Zero-Shot (Kaggle)](/benchmarks/gpqa-diamond) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 18 | 2026-10-20 source capture | [open-benchmarks/gpqa](https://www.kaggle.com/benchmarks/open-benchmarks/gpqa) |
| [GPQA Diamond Few-Shot (Kaggle)](/benchmarks/gpqa-diamond) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | [open-benchmarks/gpqa](https://www.kaggle.com/benchmarks/open-benchmarks/gpqa) |
| [GPQA Main Zero-Shot (Kaggle)](/benchmarks/gpqa) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | [open-benchmarks/gpqa](https://www.kaggle.com/benchmarks/open-benchmarks/gpqa) |
| [GPQA Main Few-Shot (Kaggle)](/benchmarks/gpqa) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | [open-benchmarks/gpqa](https://www.kaggle.com/benchmarks/open-benchmarks/gpqa) |
| [AIME 2025 (Kaggle)](/benchmarks/aime2025) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 68 | 2026-10-23 source capture | [open-benchmarks/aime-2025](https://www.kaggle.com/benchmarks/open-benchmarks/aime-2025) |
| [MMLU-Pro (Kaggle)](/benchmarks/mmlu-pro) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 64 | 2026-10-23 source capture | [open-benchmarks/mmlu-pro](https://www.kaggle.com/benchmarks/open-benchmarks/mmlu-pro) |
| [MMLU (Kaggle)](/benchmarks/mmlu) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 61 | 2026-10-23 source capture | [open-benchmarks/mmlu](https://www.kaggle.com/benchmarks/open-benchmarks/mmlu) |
| [SimpleQA Verified (Kaggle)](/benchmarks/simpleqa-verified) | Google DeepMind | Task score (%) | 57 | 2026-10-23 source capture | [deepmind/simpleqa-verified](https://www.kaggle.com/benchmarks/deepmind/simpleqa-verified) |
| [SimpleQA (Kaggle)](/benchmarks/simpleqa) | OpenAI | Task score (%) | 49 | 2026-10-23 source capture | [openai/simpleqa](https://www.kaggle.com/benchmarks/openai/simpleqa) |
| [MATH-500 (Kaggle)](/benchmarks/math-500) | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 22 | 2026-10-20 source capture | [open-benchmarks/math-500](https://www.kaggle.com/benchmarks/open-benchmarks/math-500) |

### Archived protocols

Kaggle keeps earlier versions at their own addresses. We mirror them so a version 1 score is never read as a version 2 score.

| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---:|---|---|
| [FACTS Search V1](/benchmarks/facts-search-v1) | Google DeepMind and Google Research | Published score (%) | 31 | Archived version 1 | [google/facts-search/versions/1](https://www.kaggle.com/benchmarks/google/facts-search/versions/1) |
| [FACTS Grounding V1](/benchmarks/facts-grounding-v1) | Google DeepMind and Google Research | Published score (%) | 35 | Archived version 1 | [google/facts-grounding/versions/1](https://www.kaggle.com/benchmarks/google/facts-grounding/versions/1) |

## Questions

### Is Kaggle Benchmarks free to use?

Browsing a leaderboard, following it, and downloading its rows as JSON or CSV are free and need no account. Creating a task or a Community Benchmark needs a phone-verified Kaggle account and draws on model-call quotas Kaggle provides. Researchers who need more compute and model access can apply for a Benchmarks Resource Grant.

### What is the difference between Research and Community Benchmarks?

Research Benchmarks are evaluations built by researchers at AI labs, such as the FACTS suite from Google DeepMind or MultiLoKo from Meta, and Kaggle reproduces their results. Community Benchmarks are tasks anyone writes with the kaggle-benchmarks Python library. Kaggle says the two are technically identical; research boards simply need far more compute.

### What is the kaggle-benchmarks Python library?

kaggle-benchmarks is Kaggle's open-source Python library for writing benchmark tasks, published on GitHub at Kaggle/kaggle-benchmarks under the Apache 2.0 license. A task is a Python function marked with the @kbench.task decorator that sends prompts to a model and asserts whether the output is correct. Community Benchmarks, launched in January 2026, are built from these tasks and run on Kaggle against the models the author picks.

### How is Kaggle Benchmarks different from Kaggle competitions?

A Kaggle competition ranks people: participants submit predictions or notebooks, and a fixed metric scores them against a hidden test set. Kaggle Benchmarks ranks AI models: Kaggle runs each model through the same benchmark tasks and publishes one leaderboard per benchmark. Both reuse the same leaderboard machinery Kaggle has run since 2010, but a benchmark score describes a model, not a competitor.

### How do I download a Kaggle leaderboard?

Every public board has a JSON endpoint at kaggle.com/api/v1/benchmarks/<owner>/<slug>/leaderboard, which the Download menu on the board page also shows as a curl command. The same popup offers a CSV. The JSON carries each model configuration, the task version, the native score, an evaluation date, and any confidence interval the run produced.

### Why do Kaggle results not count toward BenchLM rankings?

We import Kaggle boards as published aggregates and keep the model names as Kaggle prints them, so a row is never silently joined to a model profile that might be a different checkpoint or effort setting. Rows that are not joined cannot enter a score, so every Kaggle board is display only and sits outside the overall and category rankings.

### What is Kaggle Game Arena?

Game Arena is the part of Kaggle Benchmarks where models play each other in chess, poker, Werewolf, Dark Hex, and other games on Kaggle infrastructure. Each game has its own leaderboard, and a unified board fits one pooled Bradley-Terry rating to every win, loss, and draw, with each game weighted equally. The rating is relative, not a percentage.

### Does Kaggle run the evaluations itself?

Kaggle says it does not develop benchmarks. For research boards it works with the creators to use their evaluation code, data, and settings, then reproduces the results on its own infrastructure and keeps adding models as they are released. Community tasks also execute on Kaggle, against the models the task author selects.

## Related

- [Benchmark directory](/benchmarks)
- [How benchmark confidence works](/benchmark-confidence)
- [Methodology](/methodology)