A filtered hard subset of KMMLU containing ~5,000 questions that most models get wrong.
BenchLM is tracking KMMLU-Hard in the local dataset, but exact-source verification records for these rows are still being attached. To avoid a blank benchmark page, BenchLM shows the current tracked rows below as a display-only reference table.
These tracked rows are useful for inspection and spot-checking, but until exact-source attachments are completed they should not be treated as fully verified public benchmark rows.
BenchLM mirrors the published tracked score view for KMMLU-Hard. GPT-5.4 leads the public snapshot at 72.8% , followed by GPT-5 mini (60.6%) and GPT-5 nano (51.7%). BenchLM does not use these results to rank models overall.
GPT-5.4
OpenAI
gpt-5-4
GPT-5 mini
OpenAI
gpt-5-mini
GPT-5 nano
OpenAI
gpt-5-nano
The published KMMLU-Hard snapshot is tightly clustered at the top: GPT-5.4 sits at 72.8%, while the third row is only 21.0 points behind. The broader top-10 spread is 48.2 points, so the benchmark still separates strong models even when the leaders cluster.
11 models have been evaluated on KMMLU-Hard. The benchmark falls in the Korean Benchmarks category. BenchLM tracks this category separately from its weighted global scoring system, so these results are best compared on the dedicated Korean benchmark views. KMMLU-Hard is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
Year
2025
Tasks
~5,000 questions
Format
Multiple choice questions
Difficulty
Advanced Korean reasoning
Provides strong signals for advanced frontier models attempting reasoning in Korean.
Version
KMMLU-Hard 2025
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
A filtered hard subset of KMMLU containing ~5,000 questions that most models get wrong.
GPT-5.4 currently leads the published KMMLU-Hard snapshot with a tracked score of 72.8%. BenchLM shows this benchmark for display only and does not use it in overall rankings.
11 AI models are included in BenchLM's mirrored KMMLU-Hard snapshot, based on the public leaderboard captured on April 10, 2026.
For engineers, researchers, and the plain curious — a weekly brief on new models, ranking shifts, and pricing changes.
Free. No spam. Unsubscribe anytime.