A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.
BenchLM is tracking SimpleQA in the local dataset, but exact-source verification records for these rows are still being attached. To avoid a blank benchmark page, BenchLM shows the current tracked rows below as a display-only reference table.
These tracked rows are useful for inspection and spot-checking, but until exact-source attachments are completed they should not be treated as fully verified public benchmark rows.
BenchLM mirrors the published tracked score view for SimpleQA. GPT-5.4 leads the public snapshot at 97% , followed by GPT-5.2 Pro (97%) and GPT-5.3 Instant (96%). BenchLM does not use these results to rank models overall.
GPT-5.4
OpenAI
gpt-5-4
GPT-5.2 Pro
OpenAI
gpt-5-2-pro
GPT-5.3 Instant
OpenAI
gpt-5-3-instant
The published SimpleQA snapshot is tightly clustered at the top: GPT-5.4 sits at 97%, while the third row is only 1.0 points behind. The broader top-10 spread is 2.0 points, so many of the published scores sit in a relatively narrow band.
111 models have been evaluated on SimpleQA. The benchmark falls in the Knowledge category. This category carries a 12% weight in BenchLM.ai's overall scoring system. Within that category, SimpleQA contributes 13% of the category score, so strong performance here directly affects a model's overall ranking.
Year
2024
Tasks
Factual questions
Format
Short-form Q&A
Difficulty
Factual accuracy focused
SimpleQA prioritizes two key properties: questions should have short, factual answers that can be easily verified, and questions should be diverse and challenging. It serves as a crucial test of factual knowledge and accuracy.
Version
SimpleQA 2024
Refresh cadence
Annual
Staleness state
Refreshing
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
A benchmark that evaluates the ability of language models to answer short, fact-seeking questions accurately. Focuses on factual correctness rather than reasoning complexity.
GPT-5.4 currently leads the published SimpleQA snapshot with a tracked score of 97%. BenchLM shows this benchmark for display only and does not use it in overall rankings.
111 AI models are included in BenchLM's mirrored SimpleQA snapshot, based on the public leaderboard captured on April 20, 2026.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.