# PMPP-Hard GPU kernel optimization (AUC, 0-10 hours) (PMPP-Hard)

> An agentic GPU kernel-optimization benchmark of 69 hard tasks, adapted by Thinking Machines Lab for consistent cross-model evaluation. Each model gets four independent attempts with a 10-hour budget, and the score is the area under the mean tasks-solved curve, out of a maximum of 690 task-hours.

Canonical page: https://benchlm.ai/benchmarks/pmpphardauc

- Category: [Coding](/coding)
- Last updated: November 13, 2026

## About PMPP-Hard

- Year: 2026
- Tasks: 69 hard GPU kernel-optimization tasks
- Format: Area under the mean tasks-solved curve over 10 hours (task-hours, max 690)
- Difficulty: Long-horizon GPU kernel optimization
- Paper: [Introducing Inkling 2 Small](https://thinkingmachines.ai/news/inkling-2-small/)

Thinking Machines introduced the adapted PMPP-Hard runs with Inkling 2 Small on November 5, 2026. Each model uses its maximum reasoning setting, the PI harness and a shared compaction policy; each submitted solution gets two independent LLM audits, with a third review on disagreement. Reported AUC (task-hours): GPT-6 Astra 616.39, Kimi K3.1 Pro 583.86, Claude Opus 5.5 499.65, DeepSeek V4.1 Flash 497.36, Inkling 2 Small 494.27, Mistral Large 4 478.39, Kimi K3 453.58, GLM 5.3 430.96 and GLM 5.2 389.12. BenchLM stores the Inkling 2 Small row; the other figures are Thinking Machines runs and are listed here for context.

PMPP-Hard is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (1 models)

The one result on this page was reported by the lab that built the model.

| Rank | Model | Creator | Score | Who ran it |
|------|-------|---------|-------|-----------|
| 1 | [Inkling 2 Small](/models/inkling-2-small) | Thinking Machines Lab | 494.27 task-hours | Reported by Thinking Machines Lab |

## FAQ

### What does PMPP-Hard measure?

An agentic GPU kernel-optimization benchmark of 69 hard tasks, adapted by Thinking Machines Lab for consistent cross-model evaluation. Each model gets four independent attempts with a 10-hour budget, and the score is the area under the mean tasks-solved curve, out of a maximum of 690 task-hours.

### Which model scores highest on PMPP-Hard?

Inkling 2 Small by Thinking Machines Lab currently leads with a score of 494.27 task-hours on PMPP-Hard.

### How many models are evaluated on PMPP-Hard?

1 AI models have published results on PMPP-Hard in the BenchLM catalog.

### Does PMPP-Hard affect BenchLM's overall score?

Not directly. PMPP-Hard is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.