Kaggle Benchmarks
Kaggle Benchmarks is Kaggle's leaderboard platform for AI models, launched July 30, 2025. Kaggle doesn't write the benchmarks. It reruns tests from labs such as Google DeepMind, Meta, IBM Research, and LlamaIndex on its own infrastructure, using the authors' code and settings, and publishes the results. We mirror 27 of those boards as display-only evidence.
Every Kaggle board we store is listed below, grouped by what it tests, followed by what a Kaggle score does and doesn't establish. Open a board for its source rows, run settings, and evaluation dates.
Boards mirrored
27
25 current protocols and 2 archived versions kept at their own addresses.
Source rows stored
1162
Complete public downloads, including rows Kaggle's default page filters hide. Each row is one model configuration, not one model.
Benchmark creators
9
Kaggle organizations publishing the boards we store: Cohere Labs, Gert Labs, Google DeepMind, Google DeepMind and Google Research, Kaggle, LlamaIndex, Open Benchmarks (Kaggle reproductions), OpenAI, Rehearsals.
Kaggle leaderboards on BenchLM
Source rows count model configurations in the complete public download. Source update is the date Kaggle shows on the board, or our capture date where the board shows none. Every row links to its benchmark page and to the Kaggle board it came from.
Factuality and search
Google DeepMind and Google Research publish the FACTS boards through Kaggle. The suite average, document grounding, and search-assisted answering are separate tests.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| FACTS Benchmark Suite Four benchmark dimensions; public and held-out examples | Google DeepMind and Google Research | Published average (%) | 40 | October 13, 2026 | google/facts |
| FACTS Grounding v2 1,719 document-grounded examples: 860 public and 859 held out | Google DeepMind and Google Research | Grounding score (%) | 55 | October 13, 2026 | google/facts-grounding |
| FACTS Search V2 Search-assisted factual questions; public and held-out sets | Google DeepMind and Google Research | Published score (%) | 22 | October 9, 2026 | google/facts-search |
Documents and video
Enterprise document parsing, schema-guided extraction, and hour-long video questions.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| ParseBench (Kaggle) 2,078 source pages across five dimensions | LlamaIndex | ParseBench score | 34 | 2026-10-23 source capture | llamaindex-org/parsebench |
| ExtractBench (Kaggle) 370 documents; 4,869 pages; 67 document types | LlamaIndex | ExtractBench overall score | 21 | 2026-10-20 source capture | llamaindex-org/extractbench-leaderboard |
| VideoQA 101 five-choice questions about videos lasting 40 to 90 minutes | Google DeepMind | Top-1 accuracy (%) | 20 | October 9, 2026 | deepmind/video-qa |
Scientific coding
Kaggle runs SciCode as four tracks. Main problems and decomposed subproblems, each with and without the scientists’ background notes, are separate protocols and are not averaged here.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| SciCode Main Standard (Kaggle) Scientific research main problems | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | open-benchmarks/scicode |
| SciCode Main With Background (Kaggle) Scientific research main problems | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | open-benchmarks/scicode |
| SciCode Subproblem Standard (Kaggle) Decomposed scientific coding problems | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | open-benchmarks/scicode |
| SciCode Subproblem With Background (Kaggle) Decomposed scientific coding problems | Open Benchmarks (Kaggle reproductions) | Problem success rate | 62 | 2026-10-20 source capture | open-benchmarks/scicode |
Multilingual and math
A 16-language MMLU average from Cohere Labs and the Bengali split of MGSM.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| Global MMLU Lite (Kaggle) 400 test questions per language; source average of 16 language dimensions | Cohere Labs | Kaggle average | 45 | Source retrieved 2026-10-20T15:19:04.746Z | cohere-labs/global-mmlu-lite |
| MGSM Bengali (Kaggle) 250 Bengali math problems | Open Benchmarks (Kaggle reproductions) | Accuracy | 46 | 2026-10-20 source capture | open-benchmarks/mgsm-bengali |
Games, behavior, and agents
Game Arena is a pooled rating across strategic games, not a percentage. Human Behavior Bench and Adversarial Customer Service are simulations with their own scoring.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| Game Arena Strategic games; per-model coverage retained | Kaggle | Pooled Bradley-Terry rating | 39 | October 19, 2026 | kaggle/game-arena |
| Human Behavior Bench 1,199 tasks; 239 public and 960 private | Rehearsals | Distribution agreement | 11 | 2026-10-20 source capture | rehearsals/human-behavior-bench |
| Adversarial Customer Service Multi-agent customer-authentication simulation; task version 21 | Gert Labs | Simulation score | 11 | Source retrieved 2026-10-20T15:19:04.675Z | gert-labs/adversarial-customer-service |
Reproductions of benchmarks we already track
Kaggle reruns of GPQA, AIME 2025, MMLU, MMLU-Pro, SimpleQA, SimpleQA Verified, and MATH-500. Each sits on the existing benchmark page as its own source table beside the provider, Vals, Artificial Analysis, and Mercor rows, so there is no second page for the same test.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| GPQA Diamond Zero-Shot (Kaggle) 198 Diamond-set questions, zero-shot; task version 5 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 18 | 2026-10-20 source capture | open-benchmarks/gpqa |
| GPQA Diamond Few-Shot (Kaggle) 198 Diamond-set questions, few-shot; task version 4 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | open-benchmarks/gpqa |
| GPQA Main Zero-Shot (Kaggle) 448 main-set questions, zero-shot; task version 4 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | open-benchmarks/gpqa |
| GPQA Main Few-Shot (Kaggle) 448 main-set questions, few-shot; task version 4 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 55 | 2026-10-20 source capture | open-benchmarks/gpqa |
| AIME 2025 (Kaggle) AIME 2025 integer-answer problems; task version 6 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 68 | 2026-10-23 source capture | open-benchmarks/aime-2025 |
| MMLU-Pro (Kaggle) 10-option multiple-choice questions across 14 subjects; task version 12 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 64 | 2026-10-23 source capture | open-benchmarks/mmlu-pro |
| MMLU (Kaggle) Four-option multiple-choice questions across 57 subjects; task version 7 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 61 | 2026-10-23 source capture | open-benchmarks/mmlu |
| SimpleQA Verified (Kaggle) 1,000 verified short-form factual prompts; task version 7 | Google DeepMind | Task score (%) | 57 | 2026-10-23 source capture | deepmind/simpleqa-verified |
| SimpleQA (Kaggle) 4,326 short-form factual questions; task version 7 | OpenAI | Task score (%) | 49 | 2026-10-23 source capture | openai/simpleqa |
| MATH-500 (Kaggle) 500 competition math problems; task version 8 | Open Benchmarks (Kaggle reproductions) | Accuracy (%) | 22 | 2026-10-20 source capture | open-benchmarks/math-500 |
Archived protocols
Kaggle keeps earlier versions at their own addresses. We mirror them so a version 1 score is never read as a version 2 score.
| Benchmark | Published by | Score shown | Source rows | Source update | Kaggle board |
|---|---|---|---|---|---|
| FACTS Search V1 Search-assisted factual questions; public and held-out sets | Google DeepMind and Google Research | Published score (%) | 31 | Archived version 1 | google/facts-search/versions/1 |
| FACTS Grounding V1 1,719 examples: 860 public and 859 private | Google DeepMind and Google Research | Published score (%) | 35 | Archived version 1 | google/facts-grounding/versions/1 |
Not mirrored: Kaggle's BrowseComp, LiveCodeBench, MathVista, and MultiLoKo boards last evaluated a model between September and November 2025, and Kaggle publishes no public HLE, MLE-bench, or MMMU board.
What a verified Kaggle result means
Coverage of the launch described these as verified leaderboards, and Kaggle's own wording is that it independently reproduces and publicly releases results. In practice a verified row means three things: the number came from a run Kaggle executed, every model on the board ran under the same code and settings, and you can download the row and check it. That is a stronger claim than a vendor's self-reported score, where the harness, prompt, and sampling are often undisclosed.
A verified Kaggle row does not mean the benchmark is a good test, that its data is free of contamination, or that Kaggle's settings match the ones a vendor used. Kaggle's SciCode main-problem tracks top out below 20%, while providers and Artificial Analysis report SciCode results in the 60s under their own protocols; Kaggle counts a research problem as solved only when every step is right, and its subproblem tracks sit in the 50s. Nor does it mean the page shows everything: FACTS Grounding displays 45 of its 55 rows by default, and VideoQA 16 of 18, so the model at the top of the Kaggle page is not always the top row in the full download.
One naming trap: SimpleQA Verified and SWE-bench Verified are benchmark names, where verified means the questions were human-checked. That is a property of the dataset, not of who ran the evaluation.
How we treat Kaggle rows
- Display only
- No Kaggle board enters the overall or category rankings. The rows appear on benchmark pages with their source names and settings, and the score ignores them.
- Source names kept
- A row reads as Kaggle prints it. We don't guess which API snapshot or reasoning effort it was, so a Kaggle row is never attached to a model profile by name alone.
- Aggregates, not tasks
- We store scores, configurations, dates, and intervals. Prompts, questions, answers, documents, and videos stay with the benchmark creators. Kaggle labels these boards Apache 2.0; that declaration covers the leaderboard, not third-party task material.
- Refreshed from the download
- A refresh re-reads the public JSON and refuses to overwrite a board whose metrics or task versions changed, so a protocol change gets reviewed instead of silently replacing the old rows. How confidence works.
How Kaggle benchmarks a model
For a research board, Kaggle takes the creator's evaluation code, data, and parameter settings and runs them itself. The result on the leaderboard is Kaggle's run, not a number copied from a model card. When the creator keeps a held-out set, as Meta does for MultiLoKo and Google does for FACTS Grounding and FACTS Search, Kaggle reports the public and private splits separately, which is why several of our FACTS boards carry three scores per row.
Each row in the download names the model version, any configuration, harness, or agent used, the task version, the native score, and an evaluation date. Some rows carry a confidence interval or standard deviation. We keep all of that attached to the row rather than flattening it to one number, because two rows for the same model name can be different checkpoints or different reasoning settings.
Game Arena works differently. Instead of a fixed question set it runs matches, and an adaptive scheduler fits a Bradley-Terry model after each batch and spends the next games on the pairs whose order is still uncertain. Kaggle reports that this uses about 65% fewer games than a full round robin while reaching the same final ranking. The unified Game Arena board we mirror is that pooled rating; the chess and poker boards keep their own Elo and big-blinds-per-100 metrics.
Versions matter. When Kaggle revises judges or data, the board gets a new version and the old one stays at its own address. FACTS Grounding version 2 changed the judge ensemble and version 1 is still published, so we store both and never show a version 1 score next to a version 2 score as if they were the same test.
What Kaggle Benchmarks is
Kaggle Benchmarks lives at kaggle.com/benchmarks. Kaggle is the Google-owned data science community that has run prediction competitions since 2010, and the benchmarks platform reuses that competition machinery: a fixed task, a hidden or public test set, and a public leaderboard that updates as new entries arrive. The entries here are AI models rather than competitors.
At launch on July 30, 2025 the platform carried more than 70 leaderboards reproduced from published research benchmarks, including SciCode, GPQA, SimpleQA, LiveCodeBench, BrowseComp, FACTS Grounding, MATH-500, MMLU, MathVista, and MGSM. Kaggle calls these Research Benchmarks. Game Arena followed on August 4, 2025 with a chess tournament, and models now play poker, Werewolf, Dark Hex, and other games there too. In January 2026 Kaggle added Community Benchmarks, which anyone can build from Python tasks with the kaggle-benchmarks library.
Every public board has a JSON download at a predictable address, and that download is what we store. It includes rows the board page hides behind its default filters, so our row counts can exceed what the Kaggle page shows at first glance.
Who runs it
The Kaggle team at Google operates the platform; Meg Risdal, Kaggle's product lead, announced it. The benchmarks themselves belong to their creators. Google DeepMind and Google Research publish the FACTS suite, Meta publishes MultiLoKo, IBM Research publishes ITBench and Enterprise Operations Bench, LlamaIndex publishes ParseBench and ExtractBench, and Cohere Labs publishes Global MMLU Lite.
Kaggle's part is infrastructure: compute, access to OpenAI, Google, Anthropic, Grok, Qwen, and DeepSeek models, and the work of re-running a board when a new model ships. Since April 7, 2026 it has offered Benchmarks Resource Grants that give outside researchers the same hosting, so a benchmark author publishes once and Kaggle keeps the leaderboard current.
Questions
Is Kaggle Benchmarks free to use?
Browsing a leaderboard, following it, and downloading its rows as JSON or CSV are free and need no account. Creating a task or a Community Benchmark needs a phone-verified Kaggle account and draws on model-call quotas Kaggle provides. Researchers who need more compute and model access can apply for a Benchmarks Resource Grant.
What is the difference between Research and Community Benchmarks?
Research Benchmarks are evaluations built by researchers at AI labs, such as the FACTS suite from Google DeepMind or MultiLoKo from Meta, and Kaggle reproduces their results. Community Benchmarks are tasks anyone writes with the kaggle-benchmarks Python library. Kaggle says the two are technically identical; research boards simply need far more compute.
What is the kaggle-benchmarks Python library?
kaggle-benchmarks is Kaggle's open-source Python library for writing benchmark tasks, published on GitHub at Kaggle/kaggle-benchmarks under the Apache 2.0 license. A task is a Python function marked with the @kbench.task decorator that sends prompts to a model and asserts whether the output is correct. Community Benchmarks, launched in January 2026, are built from these tasks and run on Kaggle against the models the author picks.
How is Kaggle Benchmarks different from Kaggle competitions?
A Kaggle competition ranks people: participants submit predictions or notebooks, and a fixed metric scores them against a hidden test set. Kaggle Benchmarks ranks AI models: Kaggle runs each model through the same benchmark tasks and publishes one leaderboard per benchmark. Both reuse the same leaderboard machinery Kaggle has run since 2010, but a benchmark score describes a model, not a competitor.
How do I download a Kaggle leaderboard?
Every public board has a JSON endpoint at kaggle.com/api/v1/benchmarks/<owner>/<slug>/leaderboard, which the Download menu on the board page also shows as a curl command. The same popup offers a CSV. The JSON carries each model configuration, the task version, the native score, an evaluation date, and any confidence interval the run produced.
Why do Kaggle results not count toward BenchLM rankings?
We import Kaggle boards as published aggregates and keep the model names as Kaggle prints them, so a row is never silently joined to a model profile that might be a different checkpoint or effort setting. Rows that are not joined cannot enter a score, so every Kaggle board is display only and sits outside the overall and category rankings.
What is Kaggle Game Arena?
Game Arena is the part of Kaggle Benchmarks where models play each other in chess, poker, Werewolf, Dark Hex, and other games on Kaggle infrastructure. Each game has its own leaderboard, and a unified board fits one pooled Bradley-Terry rating to every win, loss, and draw, with each game weighted equally. The rating is relative, not a percentage.
Does Kaggle run the evaluations itself?
Kaggle says it does not develop benchmarks. For research boards it works with the creators to use their evaluation code, data, and settings, then reproduces the results on its own infrastructure and keeps adding models as they are released. Community tasks also execute on Kaggle, against the models the task author selects.