Vals MedQA (MedQA)
We show this table for reference; we do not rank on it.
Evaluating language model bias in medical questions.
MedQA score on MedQA — April 16, 2026
We mirror the published medqa score view for MedQA. o1 leads the public snapshot at 96.52%, followed by GPT-5.1 (96.38%) and Gemini 3.1 Pro Preview (96.37%). We do not use these results to rank models overall.
o1
OpenAI
high reasoning
GPT-5.1
OpenAI
high reasoning
Gemini 3.1 Pro Preview
high reasoning
95 modelsKnowledgeCurrentDisplay onlyUpdated April 16, 2026
MedQA score table (95 models)
ScoreHow MedQA is shown here
BenchLM mirrors the public Vals AI MedQA leaderboard captured from https://www.vals.ai/benchmarks/medqa and updated by Vals on April 16, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.
MedQA is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.
Snapshot
The published MedQA snapshot places o1 first at 96.52%. The third row is 0.15 points behind. The broader top-10 range is 0.64 points, so many of the published results sit in a relatively narrow band.
95 models have been evaluated on MedQA. The benchmark falls in the Knowledge category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. MedQA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About MedQA
Year
2026
Tasks
Medical question answering
Format
Accuracy score
Difficulty
Medical knowledge and bias evaluation
BenchLM mirrors the public Vals AI MedQA leaderboard as display-only external evidence. The captured snapshot preserves overall scores, task-level scores where Vals publishes them, uncertainty, latency, and cost-per-test metadata. It is excluded from BenchLM weighted rankings.
Freshness and provenance
Version
MedQA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does MedQA measure?
Evaluating language model bias in medical questions.
Which model leads the published MedQA snapshot?
o1 currently leads the published MedQA snapshot with 96.52% medqa score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on MedQA?
The April 16, 2026 snapshot contains 95 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 2,000+ readers.
One email each week. Unsubscribe anytime.