VideoQA
We show this table for reference; we do not rank on it.
DeepMind's 1H-VideoQA tests recall and reasoning over videos lasting 40 to 90 minutes. The current 101-question, five-choice task remains separate from the original 126-question set.
Evaluation results published by Google DeepMind, Google Research, and Kaggle. How Kaggle Benchmarks reruns tests.
Kaggle labels this benchmark Apache 2.0. We retain aggregate measurements with attribution and source links. That declaration does not establish rights to third-party documents, videos, or other underlying task material; no task contents are included. Original article terms.
Top-1 accuracy (%) on VideoQA — October 9, 2026
We mirror the published top-1 accuracy (%) view for VideoQA. Gemini 3.7 Flash has the highest reported value at 89.1%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. We do not use these results to rank models overall.
Gemini 3.7 Flash
Kaggle benchmark v1; task revision 21; Evaluated 2026-08-27
Gemini 3.8 Flash
Kaggle benchmark v1; task revision 21; Evaluated 2026-09-10
Gemini 4 Argon
Kaggle benchmark v1; task revision 21; Evaluated 2026-10-09
20 source rowsMultimodal & GroundedRefreshingDisplay onlyUpdated October 9, 2026
Top-1 accuracy (%) table (20 source rows)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | Gemini 3.7 FlashCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-08-27 | 89.1% | Not reported | Unknown |
| 2 | Gemini 3.8 FlashCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-09-10 | 87.1% | Not reported | Unknown |
| 3 | Gemini 4 ArgonCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-10-09 | 87.0% | Not reported | Unknown |
| 4 | Gemini 3.6 FlashCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-07-30 | 82.2% | Not reported | Unknown |
| 5 | Gemini 3.5 FlashCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-06-08 | 80.2% | Not reported | Unknown |
| 6 | Gemini 3 Flash PreviewCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-05-18 | 79.2% | Not reported | Unknown |
| 7 | Gemini 2.5 ProCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-06-08 | 78.9% | Not reported | Unknown |
| 8 | Gemini 3.1 Pro PreviewCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-06-08 | 78.2% | Not reported | Unknown |
| 9 | Gemini 3.1 Flash-Lite PreviewCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-06-08 | 78.2% | Not reported | Unknown |
| 10 | Gemini 3.5 Flash-LiteCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-07-30 | 75.2% | Not reported | Unknown |
| 11 | GPT-6 AstraCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-09-17 | 75.2% | Not reported | Unknown |
| 12 | Claude Opus 4.8Creator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-06-08 | 71.3% | Not reported | Unknown |
| 13 | Claude Haiku 5.5Creator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-10-09 | 68.3% | Not reported | Unknown |
| 14 | Claude Sonnet 4.6Creator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-05-18 | 67.3% | Not reported | Unknown |
| 15 | GPT-5.6 TerraCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-07-30 | 67.3% | Not reported | Unknown |
| 16 | GPT-6 SolCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-09-26 | 66.3% | Not reported | Unknown |
| 17 | GPT-5.6 SolCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-07-30 | 62.4% | Not reported | Unknown |
| 18 | Grok 4.6Creator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-09-17 | 60.0% | Not reported | Unknown |
| 19 | Grok 4.7Creator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-10-22 | 59.4% | Not reported | Unknown |
| 20 | GPT-6 LunaCreator not reportedKaggle benchmark v1; task revision 21; Evaluated 2026-09-26 | 57.4% | Not reported | Unknown |
How to read VideoQA
VideoQA uses 101 five-choice questions about videos lasting 40 to 90 minutes. The source recommends one frame per second, uniform subsampling when context is limited, and zero-shot prompting; audio is optional. Accuracy counts the selected correct answers.
The current set is smaller than the original 126-question release because questions are removed when their YouTube videos disappear. The download does not disclose every model's frame count, audio use, or generation settings, so this table does not establish identical input budgets. Kaggle labels the benchmark Apache 2.0, while the separate dataset card lists its license as Unknown; no videos, prompts, or answers are reproduced.
The complete official download includes models hidden by the source page's default filters. Source names and task versions stay separate from canonical model profiles; these results do not enter overall or category rankings.
Snapshot
These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. The table retains each source configuration and its reported value.
20 source rows are shown for VideoQA. The benchmark falls in the Multimodal & Grounded category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. VideoQA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About VideoQA
Year
2024
Tasks
101 five-choice questions about videos lasting 40 to 90 minutes
Format
Zero-shot long-video multiple-choice answering
VideoQA uses 101 five-choice questions about videos lasting 40 to 90 minutes. The source recommends one frame per second, uniform subsampling when context is limited, and zero-shot prompting; audio is optional. Accuracy counts the selected correct answers. The current set is smaller than the original 126-question release because questions are removed when their YouTube videos disappear. The download does not disclose every model's frame count, audio use, or generation settings, so this table does not establish identical input budgets. Kaggle labels the benchmark Apache 2.0, while the separate dataset card lists its license as Unknown; no videos, prompts, or answers are reproduced. The complete official download includes models hidden by the source page's default filters. Source names and task versions stay separate from canonical model profiles; these results do not enter overall or category rankings.
Freshness and provenance
Version
VideoQA 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does VideoQA measure?
DeepMind's 1H-VideoQA tests recall and reasoning over videos lasting 40 to 90 minutes. The current 101-question, five-choice task remains separate from the original 126-question set.
Which source row has the highest reported VideoQA value?
Gemini 3.7 Flash has the highest reported VideoQA value at 89.1%. These are published source configurations. Compare the task version and disclosed settings before treating a score difference as a model difference. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on VideoQA?
The October 9, 2026 snapshot contains 20 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.