# Graduate-Level Google-Proof Q&A (GPQA)

> A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Designed to be difficult even for skilled non-experts with access to Google.

Canonical page: https://benchlm.ai/benchmarks/gpqa

- Category: [Knowledge](/knowledge)
- Last updated: October 17, 2026

## About GPQA

- Year: 2023
- Tasks: 448 questions
- Format: Multiple choice questions
- Difficulty: Graduate level
- Paper: [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://arxiv.org/abs/2311.12022)

GPQA questions are crafted by PhD-level domain experts and validated to be answerable by experts but challenging for non-experts even with internet access. This makes it an excellent test of deep scientific knowledge and reasoning.

## Benchmark definition

GPQA is a four-choice biology, physics and chemistry benchmark written and validated by domain experts. The published family contains a 546-question Extended set, a 448-question Main set and the stricter 198-question Diamond subset.

**Variant check:** At the September 22, 2026 editorial check, the GPQA source ledger was mixed: most underlying reports said GPQA Diamond, while some identified only GPQA. The table does not expose a complete per-row subset or protocol label.

## How to read this benchmark

The table does not expose complete per-row variant or evaluation-setup labels. Consult the original provider or benchmark report before comparing results, and compare only runs that establish the same subset, prompting, tool access and scoring protocol.

**Provenance:** At the September 22, 2026 editorial check, the table was assembled from provider and secondary reports. It was not an official benchmark-owner leaderboard, and the source ledger contained no benchmark-owner or independent result rows for this lane.

**Source freshness:** The primary paper was submitted on November 20, 2023 and published at COLM 2024. The official repository and gated dataset define the benchmark; this page's refresh date records changes to the source ledger, not a new GPQA release.

**Honest limit:** The mixed source ledger prevents a controlled whole-table comparison, and the gated questions should not be reproduced. Treat the sorted values as directional until the original provider or benchmark report confirms the subset and setup for the particular results being compared.

### Primary sources

- [GPQA paper](https://arxiv.org/abs/2311.12022)
- [Official GPQA repository](https://github.com/idavidrein/gpqa)
- [Official gated dataset](https://huggingface.co/datasets/Idavidrein/gpqa)

### Related BenchLM pages

- [GPQA Diamond benchmark](/benchmarks/gpqa-diamond)
- [Knowledge ranking](/knowledge)
- [Benchmark confidence](/benchmark-confidence)

GPQA follows the active BenchAlign v5.8 policy. [See the methodology](/methodology#benchlm-defaults-and-caveats) for the current scoring and qualification rules.

## Leaderboard (143 models)

| Rank | Model | Creator | Score |
|------|-------|---------|-------|
| 1 | [GPT-6 Astra](/models/gpt-6-astra) | OpenAI | 96% |
| 2 | [Sakana Fugu-Ultra](/models/sakana-fugu-ultra) | Sakana AI | 95.5% |
| 3 | [Sakana Fugu](/models/sakana-fugu) | Sakana AI | 95.5% |
| 4 | [GPT-5.6 Sol](/models/gpt-5-6-sol) | OpenAI | 94.6% |
| 5 | [Claude Opus 4.7 (Adaptive)](/models/claude-opus-4-7-adaptive) | Anthropic | 94.2% |
| 6 | [Claude Mythos 5](/models/claude-mythos-5) | Anthropic | 94.1% |
| 7 | [Claude Opus 4.8](/models/claude-opus-4-8) | Anthropic | 93.6% |
| 8 | [GPT-5.5](/models/gpt-5-5) | OpenAI | 93.6% |
| 9 | [Kimi K3](/models/kimi-k3) | Moonshot AI | 93.5% |
| 10 | [GPT-5.6 Terra](/models/gpt-5-6-terra) | OpenAI | 92.9% |
| 11 | [GPT-5.4](/models/gpt-5-4) | OpenAI | 92.8% |
| 12 | [Ornith-1.5-397B](/models/ornith-1-5-397b) | Ornith AI | 92.8% |
| 13 | [Qwen3.8 Max](/models/qwen3-8-max) | Alibaba | 92.6% |
| 14 | [Qwen3.7 Max](/models/qwen3-7-max) | Alibaba | 92.4% |
| 15 | [GPT-5.2](/models/gpt-5-2) | OpenAI | 92.4% |
| 16 | [GPT-5.6 Luna](/models/gpt-5-6-luna) | OpenAI | 92.3% |
| 17 | [Hy4 preview](/models/hy4-preview) | Tencent | 92.3% |
| 18 | [Gemini 3.5 Flash](/models/gemini-3-5-flash) | Google | 92.2% |
| 19 | [Qwen3.8-Flash-Next](/models/qwen3-8-flash-next) | Alibaba | 91.7% |
| 20 | [Claude Opus 4.6](/models/claude-opus-4-6) | Anthropic | 91.3% |
| 21 | [GLM-5.2](/models/glm-5-2) | Z.AI | 91.2% |
| 22 | [Qwen3.8-Omni-Flash](/models/qwen3-8-omni-flash) | Alibaba | 91% |
| 23 | [DeepSeek V4.1 Flash 1017](/models/deepseek-v4-1-flash) | DeepSeek | 90.9% |
| 24 | [Kimi K2.6](/models/kimi-2-6) | Moonshot AI | 90.5% |
| 25 | [Qwen3.6 Plus](/models/qwen3-6-plus) | Alibaba | 90.4% |
| 26 | [Qwen3.7 Plus](/models/qwen3-7-plus) | Alibaba | 90.3% |
| 27 | [DeepSeek V4 Pro 0813](/models/deepseek-v4-pro-0813) | DeepSeek | 90.1% |
| 28 | [Grok 4.3](/models/grok-4-3) | xAI | 90.1% |
| 29 | [Claude Sonnet 4.6](/models/claude-sonnet-4-6) | Anthropic | 89.9% |
| 30 | [Interfaze Beta](/models/interfaze-beta) | Interfaze | 89.9% |
| 31 | [Inkling-Small](/models/inkling-small) | Thinking Machines Lab | 89.5% |
| 32 | [Qwen3.8-27B](/models/qwen3-8-27b) | Alibaba | 89.2% |
| 33 | [Ornith-1.5-35B-A3B](/models/ornith-1-5-35b-a3b) | Ornith AI | 89.2% |
| 34 | DeepSeek V4 Pro (High) | DeepSeek | 89.1% |
| 35 | [Qwen3.5 397B](/models/qwen3-5-397b) | Alibaba | 88.4% |
| 36 | [DeepSeek V4 Flash 0731](/models/deepseek-v4-flash-0731) | DeepSeek | 88.1% |
| 37 | [GPT-5.4 mini](/models/gpt-5-4-mini) | OpenAI | 88% |
| 38 | [Inkling](/models/inkling) | Thinking Machines Lab | 87.9% |
| 39 | [Qwen3.6-27B](/models/qwen3-6-27b) | Alibaba | 87.8% |
| 40 | [Kimi K2.5 (Reasoning)](/models/kimi-k2-5-reasoning) | Moonshot AI | 87.6% |
| 41 | [Kimi K2.5](/models/kimi-k2-5) | Moonshot AI | 87.6% |
| 42 | DeepSeek V4 Flash (High) | DeepSeek | 87.4% |
| 43 | [Hy3 Preview](/models/hy3-preview) | Tencent | 87.2% |
| 44 | [Claude Opus 4.5](/models/claude-opus-4-5) | Anthropic | 87% |
| 45 | [Nemotron 3 Ultra](/models/nemotron-3-ultra) | NVIDIA | 87% |
| 46 | [Qwen3.5-122B-A10B](/models/qwen3-5-122b-a10b) | Alibaba | 86.6% |
| 47 | [Ornith-1.5-9B](/models/ornith-1-5-9b) | Ornith AI | 86.4% |
| 48 | [GLM-5](/models/glm-5) | Z.AI | 86% |
| 49 | [Qwen3.6-35B-A3B](/models/qwen3-6-35b-a3b) | Alibaba | 86% |
| 50 | [Ternary Bonsai 2 27B](/models/ternary-bonsai-2-27b) | Prism ML | 85.8% |
| 51 | [GLM-4.7](/models/glm-4-7) | Z.AI | 85.7% |
| 52 | [Qwen3.5-27B](/models/qwen3-5-27b) | Alibaba | 85.5% |
| 53 | [Ling 3.0 Flash](/models/ling-3-0-flash) | InclusionAI | 85.0% |
| 54 | [Gemma 4 31B](/models/gemma-4-31b) | Google | 84.3% |
| 55 | [Qwen3.5-35B-A3B](/models/qwen3-5-35b-a3b) | Alibaba | 84.2% |
| 56 | [MAI-Thinking-1](/models/mai-thinking-1) | Microsoft | 84.2% |
| 57 | [Ling 3.0 Flash FP8](/models/ling-3-0-flash-fp8) | InclusionAI | 84% |
| 58 | [MiMo-V2-Flash](/models/mimo-v2-flash) | Xiaomi | 83.7% |
| 59 | [Claude Sonnet 4.5](/models/claude-sonnet-4-5) | Anthropic | 83.4% |
| 60 | [Gemini 2.5 Pro](/models/gemini-2-5-pro) | Google | 83% |
| 61 | [GPT-5.4 nano](/models/gpt-5-4-nano) | OpenAI | 82.8% |
| 62 | [Gemma 4 26B A4B](/models/gemma-4-26b-a4b) | Google | 82.3% |
| 63 | [Qwen3 235B 2507 (Reasoning)](/models/qwen3-235b-2507-reasoning) | Alibaba | 81.1% |
| 64 | [DeepSeek-R1-0528](/models/deepseek-r1-0528) | DeepSeek | 81% |
| 65 | [Grok 3 Mini](/models/grok-3-mini) | xAI | 80.3% |
| 66 | [DeepSeek V3.1 (Reasoning)](/models/deepseek-v3-1-reasoning) | DeepSeek | 80.1% |
| 67 | [GPT-OSS 120B](/models/gpt-oss-120b) | OpenAI | 80.1% |
| 68 | [Claude 4 Opus Thinking](/models/claude-4-opus-thinking) | Anthropic | 79.6% |
| 69 | [Nemotron 3 Super 100B](/models/nemotron-3-super-100b) | NVIDIA | 79.2% |
| 70 | [o1-pro](/models/o1-pro) | OpenAI | 79% |
| 71 | [Gemma 4 12B](/models/gemma-4-12b) | Google | 78.8% |
| 72 | [Claude 3.7 Sonnet Thinking](/models/claude-3-7-sonnet-thinking) | Anthropic | 78.2% |
| 73 | [o4-mini](/models/o4-mini) | OpenAI | 77.6% |
| 74 | [Qwen3 235B 2507](/models/qwen3-235b-2507) | Alibaba | 77.5% |
| 75 | [o3-mini](/models/o3-mini) | OpenAI | 77.2% |
| 76 | [Nemotron Ultra 253B](/models/nemotron-ultra-253b) | NVIDIA | 76.0% |
| 77 | [o1](/models/o1) | OpenAI | 75.7% |
| 78 | [Nemotron 3.5 Lightning 30B A3B NVFP4](/models/nemotron-3-5-lightning-30b-a3b-nvfp4) | NVIDIA | 75.6% |
| 79 | [Grok 3 [Beta]](/models/grok-3-beta) | xAI | 75.4% |
| 80 | [GLM-4.7-Flash](/models/glm-4-7-flash) | Z.AI | 75.2% |
| 81 | [Kimi K2](/models/kimi-k2) | Moonshot AI | 75.1% |
| 82 | [Claude 4 Opus](/models/claude-4-opus) | Anthropic | 74.9% |
| 83 | [DeepSeek V3.1](/models/deepseek-v3-1) | DeepSeek | 74.9% |
| 84 | [LFM2.5-24B-A2B](/models/lfm2-5-24b-a2b) | LiquidAI | 74.4% |
| 85 | [Mercury 2](/models/mercury-2) | Inception | 73.6% |
| 86 | [Nemotron 3 Nano 30B](/models/nemotron-3-nano-30b) | NVIDIA | 73% |
| 87 | DeepSeek V4 Pro | DeepSeek | 72.9% |
| 88 | [Nemotron 3 Nano Omni 30B A3B](/models/nemotron-3-nano-omni-30b-a3b) | NVIDIA | 72.2% |
| 89 | [DeepSeek-R1](/models/deepseek-r1) | DeepSeek | 71.5% |
| 90 | [GPT-OSS 20B](/models/gpt-oss-20b) | OpenAI | 71.5% |
| 91 | [Ministral 3 14B (Reasoning)](/models/ministral-3-14b-reasoning) | Mistral | 71.2% |
| 92 | DeepSeek V4 Flash | DeepSeek | 71.2% |
| 93 | [GPT-5 nano](/models/gpt-5-nano) | OpenAI | 71.2% |
| 94 | [Qwen3 235B-A22B (Reasoning)](/models/qwen3-235b-a22b-reasoning) | Alibaba | 71.1% |
| 95 | [ZAYA1-8B](/models/zaya1-8b) | Zyphra | 71% |
| 96 | [Magistral Medium](/models/magistral-medium) | Mistral | 70.8% |
| 97 | [GPT-4.5](/models/gpt-4-5) | OpenAI | 69.5% |
| 98 | [Phi-4-reasoning-plus](/models/phi-4-reasoning-plus) | Microsoft | 68.9% |
| 99 | [Qwen3 32B (Reasoning)](/models/qwen3-32b-reasoning) | Alibaba | 68.4% |
| 100 | [Magistral Small](/models/magistral-small) | Mistral | 68.2% |
| 101 | [Claude 3.7 Sonnet](/models/claude-3-7-sonnet) | Anthropic | 68% |
| 102 | [Granite 4.2 30B](/models/granite-4-2-30b) | IBM | 66.4% |
| 103 | [GPT-4.1](/models/gpt-4-1) | OpenAI | 66.3% |
| 104 | [Phi-4-reasoning](/models/phi-4-reasoning) | Microsoft | 65.8% |
| 105 | [QwQ-32B](/models/qwq-32b) | Alibaba | 65.6% |
| 106 | [Gemini 2.0 Flash](/models/gemini-2-0-flash) | Google | 65.2% |
| 107 | [Gemini 2.5 Flash-Lite](/models/gemini-2-5-flash-lite) | Google | 64.6% |
| 108 | [GPT-4.1 mini](/models/gpt-4-1-mini) | OpenAI | 64.2% |
| 109 | [Granite 4.2 8B](/models/granite-4-2-8b) | IBM | 64.1% |
| 110 | [Qwen3 235B-A22B](/models/qwen3-235b-a22b) | Alibaba | 62.9% |
| 111 | [Qwen2.5-Max](/models/qwen2-5-max) | Alibaba | 60.1% |
| 112 | [o1-mini](/models/o1-mini) | OpenAI | 60% |
| 113 | [Claude 3.5 Sonnet](/models/claude-3-5-sonnet) | Anthropic | 59.4% |
| 114 | [DeepSeek V3](/models/deepseek-v3) | DeepSeek | 59.1% |
| 115 | [Ling 2.6 Flash](/models/ling-2-6-flash) | InclusionAI | 59% |
| 116 | [Gemma 4 E4B](/models/gemma-4-e4b) | Google | 58.6% |
| 117 | [Mellum2-12B-A2.5B-Thinking](/models/mellum2-12b-a2-5b-thinking) | JetBrains | 57.6% |
| 118 | [ZAYA1-74B-Preview](/models/zaya1-74b-preview) | Zyphra | 57.3% |
| 119 | [Phi-4](/models/phi-4) | Microsoft | 56.1% |
| 120 | [Grok 2](/models/grok-2) | xAI | 56% |
| 121 | [Granite 4.2 3B](/models/granite-4-2-3b) | IBM | 54.8% |
| 122 | [Qwen3 32B](/models/qwen3-32b) | Alibaba | 54.6% |
| 123 | [Grok 2 mini](/models/grok-2-mini) | xAI | 51% |
| 124 | [Llama 3.3 70B](/models/llama-3-3-70b) | Meta | 50.5% |
| 125 | [Gemini 2.0 Flash-Lite](/models/gemini-2-0-flash-lite) | Google | 50.5% |
| 126 | [GPT-4.1 nano](/models/gpt-4-1-nano) | OpenAI | 50.3% |
| 127 | [Gemini 1.5 Flash](/models/gemini-1-5-flash) | Google | 50% |
| 128 | [Llama 3.1 70B](/models/llama-3-1-70b) | Meta | 46.7% |
| 129 | [Qwen2.5-VL-32B](/models/qwen2-5-vl-32b) | Alibaba | 46% |
| 130 | [Mistral Small 3.1](/models/mistral-small-3-1) | Mistral | 46.0% |
| 131 | [Gemma 4 E2B](/models/gemma-4-e2b) | Google | 43.4% |
| 132 | [Soofi S 30B-A3B](/models/soofi-s-30b-a3b) | Soofi Project | 43.4% |
| 133 | [Claude 3.5 Haiku](/models/claude-3-5-haiku) | Anthropic | 41.6% |
| 134 | [Mellum2-12B-A2.5B-Instruct](/models/mellum2-12b-a2-5b-instruct) | JetBrains | 40.9% |
| 135 | [LFM2.5-350M](/models/lfm2-5-350m) | LiquidAI | 30.6% |
| 136 | [Llama 3.1 8B](/models/llama-3-1-8b) | Meta | 30.4% |
| 137 | [Phi-3.5-mini](/models/phi-3-5-mini) | Microsoft | 30.4% |
| 138 | [Granite-4.0-1B](/models/granite-4-0-1b) | IBM | 29.9% |
| 139 | [Granite-4.0-H-1B](/models/granite-4-0-h-1b) | IBM | 29.7% |
| 140 | [Granite-4.0-H-350M](/models/granite-4-0-h-350m) | IBM | 26.1% |
| 141 | [LFM2.5-VL-450M](/models/lfm2-5-vl-450m) | LiquidAI | 25.7% |
| 142 | [LFM2.5-230M](/models/lfm2-5-230m) | LiquidAI | 25.4% |
| 143 | [Granite-4.0-350M](/models/granite-4-0-350m) | IBM | 24.1% |

## FAQ

### What does GPQA measure?

GPQA is a four-choice science benchmark written and validated by domain experts. The published family includes a 546-question Extended set, a 448-question Main set, and the stricter 198-question Diamond subset. A result is interpretable only when its source identifies which subset and evaluation setup it used.

### Are all rows on this page GPQA Diamond results?

No. At the September 22, 2026 editorial check, underlying reports were mostly labelled GPQA Diamond, while some identified only GPQA. The table does not expose a variant label for every row. Consult the original provider or benchmark report, and compare results only when they establish the same subset and setup.

### Where should I verify a GPQA result?

Use the paper, official repository, and gated dataset for the benchmark definition. At the September 22, 2026 editorial check, this source ledger was assembled from provider and secondary reports, not an official owner-run leaderboard. The dataset authors use controlled access and contamination markers, so this page does not reproduce benchmark questions.

## Compare Top Models on GPQA

- [GPT-6 Astra vs Sakana Fugu-Ultra](/compare/gpt-6-astra-vs-sakana-fugu-ultra)
- [Sakana Fugu-Ultra vs Sakana Fugu](/compare/sakana-fugu-vs-sakana-fugu-ultra)
- [Sakana Fugu vs GPT-5.6 Sol](/compare/gpt-5-6-sol-vs-sakana-fugu)
- [GPT-5.6 Sol vs Claude Opus 4.7 (Adaptive)](/compare/claude-opus-4-7-adaptive-vs-gpt-5-6-sol)

## Related Reading

- [GPQA benchmark explainer](/blog/posts/gpqa-diamond-science-benchmark)