Vals MedQA (MedQA)
Evaluating language model bias in medical questions.
Data verifiedHow BenchLM shows MedQA
BenchLM mirrors the public Vals AI MedQA leaderboard captured from https://www.vals.ai/benchmarks/medqa and updated by Vals on April 16, 2026. The snapshot preserves overall scores, uncertainty, latency, cost-per-test metadata, and task-level scores where Vals publishes them.
MedQA is display only on BenchLM. Vals proprietary or Vals-hosted aggregate views are useful context, but BenchLM does not use them as weighted ranking inputs or as a replacement for benchmark-native source records.
MedQA score on MedQA — April 16, 2026
BenchLM mirrors the published medqa score view for MedQA. O1 leads the public snapshot at 96.52% , followed by GPT-5.1 (96.38%) and Gemini 3.1 Pro Preview (96.37%). BenchLM does not use these results to rank models overall.
O1
OpenAI
openai/o1-2024-12-17
GPT-5.1
OpenAI
openai/gpt-5.1-2025-11-13
Gemini 3.1 Pro Preview
google/gemini-3.1-pro-preview
The published MedQA snapshot is tightly clustered at the top: O1 sits at 96.52%, while the third row is only 0.15 points behind. The broader top-10 spread is 0.64 points, so many of the published scores sit in a relatively narrow band.
95 models have been evaluated on MedQA. The benchmark falls in the External benchmark mirrors category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. MedQA is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About MedQA
Year
2026
Tasks
Medical question answering
Format
Accuracy score
Difficulty
Medical knowledge and bias evaluation
BenchLM mirrors the public Vals AI MedQA leaderboard as display-only external evidence. The captured snapshot preserves overall scores, task-level scores where Vals publishes them, uncertainty, latency, and cost-per-test metadata. It is excluded from BenchLM weighted rankings.
BenchLM freshness & provenance
Version
MedQA 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
MedQA score table (95 models)
FAQ
What does MedQA measure?
Evaluating language model bias in medical questions.
Which model leads the published MedQA snapshot?
O1 currently leads the published MedQA snapshot with 96.52% medqa score. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on MedQA?
95 AI models are included in BenchLM's mirrored MedQA snapshot, based on the public leaderboard captured on April 16, 2026.
The AI models change fast. We track them for you.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.