The desk
How we work
A ranking is a shortlist.
Rank orders the evidence; it does not make the decision. Every row here opens to its sources so you can disagree with us.
We do not invent scores to fill a table.
A model without a public, independent row stays unranked, and says so. Vendor-run numbers are shown, labeled, and kept out of the rank.
When we were wrong, the fix is on the page.
Scoring bugs, missing coverage, and re-weightings are dated and linked, because a ranking you cannot audit is a launch post with a table.
BenchLM is an independent benchmark publication for large language models, published at benchlm.ai. It gathers publicly reported evaluation results from provider model cards, benchmark leaderboards and papers into one ranking, and labels every ranked position Supported or Estimated according to the evidence behind it. BenchLM does not run private evaluations, and commercial relationships do not change a score or a rank.
The site exists because AI evaluation is fragmented. Benchmark results are scattered across model cards, launch posts, benchmark leaderboards, papers, and screenshots. BenchLM pulls those rows into one place, applies ranking guardrails, and tries to make the final score understandable instead of mystical.
Where the data comes from
Benchmark scores come from official provider tables, model cards, release announcements, benchmark-native public leaderboards, and other public documentation. We cross-reference multiple sources when possible. If two public rows conflict, BenchLM prefers the more exact or more controlled source rather than whichever number looks better.
Pricing and runtime metadata are tracked separately from benchmark performance. They are useful for decisions, but they are not treated as benchmark rows and do not change any capability score.
How scoring works
The overall score starts from independent external leaderboards, such as benchmark composite indices and human-preference leaderboards, each weighted by how uncertain it is. When several are available, no single family carries more than half of that external part of the score, and preference leaderboards together carry at most a quarter. Direct benchmark results then add a correction of at most 3 points, scaled by how independent they are.
Missing tests stay blank. BenchAlign v5.8 blends a model's available signals with eligible peers, its own benchmark estimate, a reviewed lineage prior, and the previous policy estimate. Peers exclude the model's own family and provider, and their influence is capped. Supported Overall scores keep their existing calculation. The category boards also use a preference-free general estimate where category evidence is thin, so both Supported and Estimated category scores can change.
Harder, less saturated benchmarks carry more weight than legacy rows that mostly act as floor checks, and most saturated tests stay in the score at a lower weight. SWE-bench Verified and SWE-bench (Vals), whose tests are contaminated or saturated, stay visible for display only and do not move any score. The full public methodology lives on the methodology page.
Supported and Estimated
Every position on the overall, agentic, coding, and knowledge boards carries one of two labels. Supported means we have enough independent evidence to stand behind the row. Estimated means the rank is visible and the uncertainty is wider. Both remain visible in one ranking, and generated benchmark rows never enter either.
Retrospective tests reduced average Overall reconstruction error, but three checks of large errors worsened. Estimated Overall ranges are conditional on those tests; sparse source patterns and the changed category priors do not have validated coverage. The ranges do not establish confidence in a rank or future performance. The methodology limits explain what remains unvalidated.
Update frequency
BenchLM updates when new benchmark rows appear, when benchmark protocols materially change, and when new tracked model releases justify a refresh. The current benchmark dataset was last updated on October 20, 2026.
What we don't track
BenchLM is not a private eval lab. We do not run hidden internal model shootouts, and we do not claim to replace hands-on testing in your own workflow. We also do not treat provider contract pricing, regional pricing, or one-off promotional rates as stable source-of-record pricing data. BenchLM is best used for public benchmark performance, relative tradeoffs, and shortlist construction.
Who runs this
BenchLM started in August 2025. It is built and maintained by Glevd (@glevd on X). If you spot an error in the data or want to suggest a model or benchmark to add, email hello@benchlm.ai.