Chinese Massive Multitask Language Understanding (CMMLU)
We show this table for reference; we do not rank on it.
A Chinese multitask academic benchmark reported in DeepSeek-V4 base-model evaluations.
Benchmark score on CMMLU — October 29, 2026
We compile the CMMLU rows from provider self-reports. DeepSeek V4 Pro Base leads the table at 90.8%, followed by DeepSeek V4 Flash Base (90.4%) and LongCat-Flash-Lite-Sparse (84.3%). We do not use these results to rank models overall.
DeepSeek V4 Pro Base
DeepSeek
DeepSeek V4 Flash Base
DeepSeek
LongCat-Flash-Lite-Sparse
Meituan
3 modelsKnowledgeCurrentDisplay onlyUpdated October 29, 2026
Benchmark score table (3 models)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | DeepSeek V4 Pro BaseDeepSeek | 90.8% | Not reported | Open |
| 2 | DeepSeek V4 Flash BaseDeepSeek | 90.4% | Not reported | Open |
| 3 | LongCat-Flash-Lite-SparseMeituan | 84.3% | Not reported | Open |
Among the reported CMMLU rows, DeepSeek V4 Pro Base is first at 90.8%. The third row is 6.5 points behind. The broader top-10 range is 6.5 points, so many of the published results sit in a relatively narrow band.
3 models have been evaluated on CMMLU. The benchmark falls in the Knowledge category. CMMLU is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About CMMLU
Year
2026
Tasks
Chinese academic QA
Format
Exact match
Difficulty
Broad Chinese knowledge
BenchLM stores CMMLU as a display-only provider-table row when exact values are published in DeepSeek-V4 evaluations.
Freshness and provenance
Version
CMMLU 2026
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does CMMLU measure?
A Chinese multitask academic benchmark reported in DeepSeek-V4 base-model evaluations.
Which model scores highest on CMMLU?
DeepSeek V4 Pro Base by DeepSeek currently leads with a score of 90.8% on CMMLU.
How many models are evaluated on CMMLU?
3 AI models have published results on CMMLU in the BenchLM catalog.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 6,100+ subscribers.
One email each week. Unsubscribe anytime.