Skip to main content
BenchLM
Data

BenchLM Data

Your AI’s model advice stops at its training cutoff.

BenchLM Data connects Claude, ChatGPT, Cursor, or your own code to today’s rankings, prices, and benchmark results. Ask which model to use, what it will cost a month at your volume, or what changed since you last asked. Every answer carries its source and its date.

Trusted by 800+ users

Rankings published November 16, 2026. Prices checked November 12.Free plan: 1,000 reads a month, no card.

Create an account or sign in

New accounts start on Free: 1,000 reads a month, no card required.

We’ll email you a one-time code.

Free includes 1,000 reads a month. No card.

What your assistant has missed

Pick the model it runs on. The cutoff is the date its maker documents.

Since its cutoff
200 days
Training data ends April 30, 2026, per OpenAI
Model releases
321
from 80 labs
Price changes
15
on list prices we track
Picks it cannot know
5 of 5
today’s answer for coding under $3

Ask today which model to use for coding under $3 per million input tokens, and all five current answers were released after April 30, 2026. GPT-6.1 Sol is one of them: it shipped four months after its own training data ends.

Release ledger through November 16, 2026. Price history through November 12.Model updates

You ask

Which model should we use for coding if we want to stay under $3 per million input tokens?

Your assistant reads

Five coding models with a list price at or under $3 per million input tokens, in ranking order, with coding score, list price, and evidence label
ModelCoding scorePrice per 1M tokensinput / outputEvidence
Gemini 4 CarbonGoogleCoding score 84.4$2.00 / $10.00 per 1M tokensSupported
Claude Sonnet 5.5AnthropicCoding score 81.2$2.00 / $10.00 per 1M tokensSupported
Gemini 4 HydrogenGoogleCoding score 76.1$1.50 / $7.50 per 1M tokensSupported
GPT-6.1 SolOpenAICoding score 73.4$2.00 / $10.00 per 1M tokensSupported
Gemini 4 ArgonGoogleCoding score 72.1$2.00 / $10.00 per 1M tokensSupported
Coding models with a list price at or under $3 per million input tokens, in ranking order, as the recommend query returns them. Ranking published November 16, 2026. Your assistant writes the answer in its own words from rows like these. See the full coding ranking
1,033
models tracked
817
benchmarks
409
models with list prices
November 16, 2026
data refreshed

Counts describe the website catalog. What your AI can read depends on the source; coverage checks are free.

Ask your AI. It answers from current data.

Four things people connect it for. Copy a question, paste it into your assistant, and change the job or the budget to fit yours. Each answer names the ranking it used and how much evidence stands behind the score.

Pick the right model today

Free

A shortlist for a use case or a ranking, inside your price cap, with open weights or a minimum context window if you need them. The answer says which ranking it used and what that ranking doesn’t measure.

Which model should we use for customer support under $1 per million input tokens?

Compare a shortlist

Free

Two to five models side by side: rank and score on every ranking, list price, context window, and release date. Useful the day a release lands and you want it next to the model you run now.

Where does the newest model rank in coding against Claude Sonnet 5.5?

See what changed

30 days on Free · 3 months on Data Pro · all history on Research

Models that entered or left a ranking, the largest rank moves, and price changes since a date, in one answer. Saved ranks keep their methodology version, and Arena rating snapshots go back to May 2023.

What changed in the coding ranking since October 15?

Route work from your own agents

Free

Every query is also a REST path. An orchestrator can ask for a model that fits the task and the budget before it starts the work, and a notebook or dashboard can read the same data.

GET /v1/recommend?surface=coding&maxInputPrice=3Set it up in your own code

Check the dates available before asking for history. Read the scoring methodology.

The same question, with live data

Your assistant calls BenchLM, reads rows like these, and answers in its own words. Change the question to see what it would read.

Paste it into an assistant that has BenchLM connected.

82 models fit this budget today. The answer leaves out 5 that their labs have shut down and 38 that a newer model replaces, and names them, so every pick is one you can still call.

You ask

Which model should we use for coding under $3 per million input tokens? We send about 50M input and 10M output tokens a month.

What your assistant reads: the top five coding models for this question, in ranking order, with score, list price, cost a month and evidence label
RankModelCoding scoreList price per 1MCost a monthEvidence
#2Gemini 4 CarbonGoogle · released Oct 28, 202684.43range 77.8 to 91.0$2.00 / $10.00$200$100 in, $100 outSupported
#4Claude Sonnet 5.5Anthropic · released Sep 28, 202681.15range 75.9 to 86.4$2.00 / $10.00$200$100 in, $100 outSupported
#6Gemini 4 HydrogenGoogle · released Nov 12, 202676.13range 69.4 to 82.8$1.50 / $7.50$150$75 in, $75 outSupported
#8GPT-6.1 SolOpenAI · released Sep 29, 202673.42range 65.8 to 81.0$2.00 / $10.00$200$100 in, $100 outSupported
#9Gemini 4 ArgonGoogle · released Sep 30, 202672.09range 65.5 to 78.7$2.00 / $10.00$200$100 in, $100 outSupported
Ranks are out of 161 models on the coding ranking. Cost is list price at 50M input and 10M output tokens a month, before caching or batch discounts. Prices checked November 12, 2026.Full coding ranking

What you can ask

Every question below works on the Free plan, in plain words, from any connected assistant.

Six kinds of question BenchLM Data answers, with an example of each
You want toWhat comes backTry askingCopy
Pick a modelA shortlist for a use case or a ranking, inside your price cap, with open weights or a context window if you need them. It leaves out models their labs have shut down or replaced, and names them.Which model should we use for customer support replies, under $1 per million input tokens?
Price it at your volumeGive your monthly tokens and each pick comes back in dollars a month at list price, split into input and output.What would the top three coding models cost us at 50M input and 10M output tokens a month?
Compare a shortlistTwo to five models side by side on every ranking, list price, context window, and release date. A typed name is enough: “Sonnet 5.5” finds the model.Compare Sonnet 5.5, Gemini 4 Carbon and GPT-6.1 Sol for coding.
Keep upWhat entered or left a ranking, the largest moves, and every price change since a date, by model name. Ask about a model again and the answer notes what changed since you last asked.What changed in the rankings and prices over the last 7 days?
Check the evidenceThe source scores behind a rank, from the boards that published them, each with its unit, its direction, and its license notice.What are Claude Fable 5.2’s scores on SWE-Bench Pro and Terminal-Bench?
Price it everywhereFull tool listWhat each inference host charges, free API tiers and rate limits, subscription plans, and image, video, and speech prices. Each price says the day it was read.Which hosts serve DeepSeek V4.1 Flash, and what does each charge?
Shipped or refreshed in the week to November 16, 2026.What’s new

What changed this week

November 10 to 16, from the same record your assistant reads. Ask about a model again later and the answer notes what changed since.

Releases and price changes recorded from November 10, 2026 to November 16, 2026
DateChangeWhat happened
ReleaseQwen4 Max from Alibaba. Now #40 overall.
ReleaseHeyGen Decision 1 from HeyGen.
ReleaseHeyGen Decision 1 Turbo from HeyGen.
ReleaseGemini 4 Hydrogen from Google. Now #9 overall.
ReleaseHark Handoff 2 from Hark.
ReleasePerplexity Decider Flash v1.3 4B from Perplexity.
ReleasePerplexity Decider v1.3 27B from Perplexity.
ReleaseSOV-Transcribe-1 from Salesforce.
12 releases and 2 price changes recorded since November 10, 2026.All model updates

Connect your tools

Pick your app. It opens a BenchLM sign-in page, so there is no key to paste, and a new email starts on Free.

Claude

claude.ai, Claude Desktop, and the mobile apps · Sign in

  1. In Claude, go to Customize, then Connectors. Choose Add, then the option to add a custom connector.
  2. Name it BenchLM Data, enter https://data.benchlm.ai/mcp, and keep the detected sign-in settings. You don’t need request headers.
  3. Connect it. A BenchLM page opens: sign in with your email and allow the connection.
  4. On Team and Enterprise plans, an owner adds the connector under Organization settings, then Connectors. Members connect it from Customize, then Connectors, with their own Data account. Free Claude accounts can add one custom connector.
Claude setup docs

Server address

https://data.benchlm.ai/mcp

Lists the 14 tools assistants use most, so less loads into every conversation.

Every tool, same sign-in

https://data.benchlm.ai/mcp/full

All 28, including dated history, host prices, and plans. Use one address per app: connecting both gives the assistant the core tools twice.

From your own code

GET /v1/recommend?surface=coding&maxInputPrice=3Get a key

REST and MCP share one allowance. The OpenAPI file covers every endpoint.

Ask for a recommendation over REST
Recommend coding models under a price cap
curl 'https://data.benchlm.ai/v1/recommend?surface=coding&maxInputPrice=3&limit=5' \
  -H 'Authorization: Bearer YOUR_DATA_KEY'
Read the coding ranking over REST
Read the current coding ranking
curl 'https://data.benchlm.ai/v1/rankings/current?surface=coding&limit=10' \
  -H 'Authorization: Bearer YOUR_DATA_KEY'
View the raw MCP request
Call the current ranking MCP tool
curl 'https://data.benchlm.ai/mcp' \
  -H 'Authorization: Bearer YOUR_DATA_KEY' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -H 'MCP-Protocol-Version: 2025-11-25' \
  --data '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"benchlm_list_current_rankings","arguments":{"surface":"coding","limit":10}}}'

REST and API-key MCP requests send Authorization: Bearer YOUR_DATA_KEY. Supported MCP protocol versions: 2026-07-28, 2025-11-25, 2025-06-18, and 2025-03-26.

Read the OpenAPI document
Usage and coverage checks never use a read.

Plans and pricing

Every plan asks the same questions of the same current data. A paid plan is for running it without you: more reads, a higher rate, and a longer record.

Free

Monthly price: $0

What it is for: Ask an assistantOne person asking by hand, or trying the API. Every question on this page works here. No card.

Monthly reads: 1,000about 30 a day, 30 requests a minute

History: Last 30 daysof ranks, scores, and prices

Start free

Data Pro

Monthly price: $49 USD/month

What it is for: Automate itScripts, agents, routers, and internal dashboards that read on their own every day: sync models and prices, price a job before it runs, follow a whole ranking. A first subscription starts with a 7-day free trial.

Monthly reads: 25,000about 800 a day, 60 requests a minute

History: Last three monthsenough to chart a trend

Choose Data Pro

Research

Monthly price: $249 USD/month

What it is for: Study the recordBacktests and papers that need every saved rank, score, and price, with no age limit.

Monthly reads: 100,000about 3,200 a day, 60 requests a minute

History: Everything we keepwhere a source has dates

Choose Research

Commercial license

Price: By agreement

What it is for: Show it to your customersBenchLM rankings, scores, and prices inside your product, client reports, or proposals, with credit. Passing the data on through an API, files, or resale is agreed separately. Write to hello@benchlm.ai with what you are building and we reply with terms and a price.

Write to us
A read is one successful data response. Free, Pro, and Research cover queries and internal use; showing the data to your customers takes the commercial license.Commercial-use terms

Questions

Check the records you need before you choose a plan. Coverage and usage checks are free, including when your monthly reads are exhausted.

Do I need an API key to connect Claude or ChatGPT?
No. Chat apps and most coding tools sign in through your browser: you enter your email on a BenchLM page and allow the connection, and the app never sees a key. Keys are for REST, scripts, and clients without MCP sign-in. Either way, reads come from the same account allowance.
Is there a free trial?
Yes. Your first Data Pro subscription starts with a 7-day free trial and the full 25,000-read allowance. If checkout saves a card, billing at $49 USD per month starts when the trial ends unless you cancel. If it doesn’t, the trial ends and you go back to Free unless you add a card. Each account gets one trial.
What does Data Pro add to Free?
Room to run it unattended. The tools and the current data are the same on every plan. Data Pro gives you 25,000 reads a month in place of 1,000 and 60 requests a minute in place of 30, which is enough for a script that checks prices every two minutes or a dashboard that re-reads every ranking each day. History reaches back three months in place of 30 days.
How do I disconnect an app or revoke a key?
From your Data account. A signed-in app stays connected as long as it’s used at least once every 30 days, and one account can connect up to 25 apps.
What counts as a read?
One successful data page uses one read. Initialization, discovery, coverage and usage checks, and unsuccessful queries do not use the monthly allowance. All keys and connected apps on an account share one allowance across REST and MCP.
When does my allowance reset?
Free renews on your account’s activation anniversary each month. Pro and Research follow your subscription period. Short months use their last day. Your account and usage responses show the exact monthly and request-rate reset times.
What happens when I reach a limit?
The API and MCP identify the monthly read allowance or request-rate limit, with a reset time and retry delay. Free accounts get a Data Pro upgrade link. Eligible Pro accounts can upgrade to Research with a prorated price difference and a fresh 100,000-read allowance for the remaining subscription period. You can receive usage emails at 75%, 90%, and 100%; manage them in your account. Usage and coverage checks stay available.
Can I check coverage before paying?
Yes. A free account gives you current rankings, source results, model and benchmark catalogs, the last 30 days of history, and coverage checks. The coverage tools identify available sources and date windows without using your monthly reads. Check your model and benchmark first, then choose the plan that includes the history you need.
How far back does the history go?
It depends on the source. Arena’s text leaderboard snapshots go back to May 2023, saved BenchAlign rankings start on March 31, 2026, and many model-card tables have no dates at all. Research removes the history age cap; it does not fill missing records. Check the dates for your exact source and model.
Why choose Research over Data Pro?
Research keeps every record we have, with no age limit, and includes 100,000 reads a month for $249 USD. Data Pro reaches back three months and includes 25,000 reads for $49 USD. Both allow 60 requests a minute. Choose Research when you need the whole record or more reads.
How much history can one query return?
Data Pro reaches back three calendar months, counted by UTC date. Research has no age limit on the records we keep. Coverage varies by source. Historical pages contain up to 500 rows. Saved-state history uses windows of up to 366 days (daily or weekly granularity fits a year in one page) and cursors. Source-run and publisher snapshot history take an exact board and source model ID, with date filters of up to 90 days. Snapshot history supports cursors and preserves publisher dates; run history preserves reported evaluation times. Check the relevant coverage tool first.

Commercial display or redistribution? Ask about commercial use.

Free, Pro, and Research are access plans for queries and internal analysis. Paid plans add reads, a higher request rate, and a longer record; they do not include commercial display or redistribution permission.

Publishing BenchLM-owned outputs or the compiled feed in a customer-facing commercial website, app, or product needs a separate display agreement. Redistributing that material through an API, downloads, or resale needs separately agreed scope. Write to hello@benchlm.ai for a commercial license.

Individual quotations and attributed embeds keep their existing permissions. Publisher measurements keep their own commercial-use and redistribution rights. Retain attribution, source notices, and applicable terms. Read the Data access terms.

Every query, with its fields and plan.

The 28 MCP tools, the REST path for each, and the coverage each one returns. Usage and coverage checks are free. Tools marked /mcp/full are listed at that address only; every tool answers by name at both.

See all available queries
Data coverage, plans, and MCP tools
QueryFields and coveragePlanMCP tool and REST path
Account usageCurrent plan, successful and remaining monthly reads, request-rate usage, and both reset times. Usage checks do not consume a monthly read.Freebenchlm_get_usageGET /v1/usage
Model recommendationUp to ten models in the order of one BenchAlign ranking surface, filtered by use case, price caps, open weights, context window, one or several creators, or tags such as free API, self-hostable, or the lab’s country. A model whose lab has shut down its own API is left out unless you ask for it, and the answer names the ones it left out; so is a model whose replacement is also a candidate in the same answer, unless you ask for it. Each pick carries its url, score, rank, evidence status, list price, context window, and tags; a pick with a provider retirement notice or a successor says so. The response names the surface and its limits, and lists models it could not check because data is missing. Give monthly token volumes to get dollars a month per model.Freebenchlm_recommendGET /v1/recommend
Model comparisonTwo to five models side by side: rank, score, and evidence status on every surface, list price, context window, open-weight flag, license, lab origin, tags, successors, provider deprecation and retirement notices, self-host sizing, the Ollama tag, and release date. Two models also return the comparison page url. Give monthly token volumes to get dollars a month per model.Freebenchlm_compare_modelsGET /v1/compare?modelKeys=a,b
Value boardThe efficient pick, the cheapest model scoring at least 90% of the top score (retention sets the share), then models ranked by score per dollar on one surface: valueIndex is the score divided by the blended, input, or output price, with the formula stated in the response. Each row says whether it is on the price-performance frontier. Filter by minimum score, price cap, open weights, one or several creators, or tags, and leave out tagged models such as retired ones. Models without a price are listed as not evaluated; free models are listed separately and never ranked by division.Freebenchlm_value_boardGET /v1/value
Model feedA paged feed for routers and dashboards: every ranked model with open-weight flag, license, lab country, tags, successors, provider deprecation and retirement notices with their dates and named replacements, context window, overall and per-surface ranks, current input, output, and cached-input prices, and its url. Filter on the server by creator and tags, for example the free models plus the labs you choose, and leave out models the lab has retired or replaced. Up to 250 models per page.Freebenchlm_model_feed/mcp/fullGET /v1/feed/models
Current BenchAlign rankingsFinal scores, published ranks, supported-only ranks, evidence labels and routes, maturity labels, interval bounds, scoring date, and methodology version for Overall, Agentic, Coding, Knowledge, Reasoning, and Multimodal (preliminary). Regional models are listed separately.Freebenchlm_list_current_rankingsGET /v1/rankings/current
Current model rankingsOne model’s final score and rank across every available BenchAlign surface, with explicit gaps where it is unranked.Freebenchlm_get_current_model_rankingsGET /v1/models/{identifier}/rankings/current
BenchAlign scoring basketThe benchmarks behind the current BenchAlign scores: category, benchmark, share of the category reference weight, and any exclusion reason. Results evaluated by external leaderboards appear as one combined line per category.Freebenchlm_get_benchalign_basket/mcp/fullGET /v1/rankings/methodology
Model API pricesCurrent and launch list prices in USD for input, output, and cached input, per 1M tokens or the model’s own unit, with dates and the quoted source page. Prices come from the model’s lab or, for open weights its lab does not sell, one named host.Freebenchlm_list_model_pricingGET /v1/pricing
Inference host pricesWhere a model can be called by API and what each host charges: the per-token list price ten inference hosts and routers show on their own sites, with the host’s model ID, context window, status, the day the list was read, and its source page. One model across hosts, cheapest first, or one host’s whole list. A host flavor of a model (quantized, turbo, fast) is marked as a variant. These are single readings of each price list, with no history.Freebenchlm_list_host_prices/mcp/fullGET /v1/pricing/hosts
Free tiers and subscription plansWhether a provider has a free API tier, a signup credit, or only a paid entry tier, with its published rate limits and what moves an account up; and the monthly plans for Claude, ChatGPT, Gemini, and coding tools, with price, included usage in the provider’s words, caps, and the models each plan names. Every row is copied from the provider’s page with the day it was read. Filter by provider, type, a model, or a monthly price cap.Freebenchlm_list_access_plans/mcp/fullGET /v1/plans
Image, video, speech, and music pricesWhat media models cost in the unit each is billed in: per image, per second of video, per minute of audio, per character of speech, or per song. One row per published variant (tier, resolution, duration), in USD as the provider lists it, with the API model ID, the day the page was read, and its source. Rows are grouped by modality and unit and never converted into tokens; a price kept after a provider shut a model down is marked historical.Freebenchlm_list_media_prices/mcp/fullGET /v1/pricing/media
Model price historyDated price entries for a model or for all models: launch, change, and confirmed. A first-observed date means on or before that date.All plansbenchlm_pricing_history/mcp/fullGET /v1/history/pricing
Model catalogNames, creators, canonical IDs, source links, sourced release dates with their date precision, license, lab origin (country and region), the model each one replaced and the models that replaced it, provider deprecation and retirement notices, self-host sizing for open weights (parameters and the GPU sets the weights fit on), the Ollama tag and pull command where Ollama carries the model, the input and output types the provider documents (text, image, audio, video, PDF) where BenchLM has recorded them, and tags for open weights, license class, self-hosting, Ollama, hosted API, free API, origin, reasoning, input and output types, and lifecycle. Search by name, filter by creator and tags, or leave tagged models out.Freebenchlm_list_modelsbenchlm_get_modelGET /v1/modelsGET /v1/models/{identifier}
Benchmark catalogNames, catalog IDs, source references, and curator categories and tags. Search by name or filter by category, tag, and relationship; discovery labels do not establish result coverage.Freebenchlm_list_benchmarksGET /v1/benchmarks
Current benchmark resultsAll admitted current source results, with native units and exact profiles. Sources include 141 Arena leaderboards, Epoch AI, BullshitBench v2, tables from provider model cards, and results a model’s own lab reported, on separate unranked boards. Search by category or model name. Rows on BenchLM’s recorded boards say who ran the test, and you can filter by it.Freebenchlm_list_current_benchmark_resultsGET /v1/benchmarks/results/current
Benchmark result coverageAvailable source boards, row counts, metrics, units, and source terms, with each BenchLM recorded board’s rows counted by who ran the test. Search boards by word or benchmark key. Coverage checks do not use a read.Freebenchlm_benchmark_result_coverageGET /v1/benchmarks/results/coverage
Published evaluation runsSource-reported benchmark runs for an exact board and model profile. Run dates are distinct from archive commit dates; undated records are identified.All plansbenchlm_benchmark_result_history/mcp/fullGET /v1/history/benchmark_results
Published benchmark snapshotsFollow an exact source model’s published rating, rank, and confidence interval over time. Arena snapshots keep the publisher’s dates, and scoring methods stay separate.All plansbenchlm_benchmark_result_snapshot_history/mcp/fullGET /v1/history/benchmark_result_snapshots
One model across result boardsA model’s published scores and ranks on every result board where its source profile has a reviewed match, from one catalog model ID.All plansbenchlm_model_result_history/mcp/fullGET /v1/history/benchmark_result_snapshots?modelKey=
What changed since a dateModels that entered or left a ranking surface, the largest rank moves, price changes, and rescoring events since a date, in one call. Moves across a rescoring are kept apart from model changes.All plansbenchlm_changesGET /v1/changes?since=
Historical coverageAvailable methods, retained date windows, and delivery coverage. Check this before requesting history.Freebenchlm_history_coverage/mcp/fullGET /v1/coverage
BenchAlign historySaved BenchLM scores, ranks, evidence labels, ranking surfaces, and methodology versions for an exact model.All plansbenchlm_rank_history/mcp/fullGET /v1/history/rank_history
Ranking at a dateStored BenchAlign ranking rows as of a UTC date. The response identifies incomplete coverage and preserves the saved scoring version.All plansbenchlm_ranking_snapshot/mcp/fullGET /v1/history/ranking_snapshot
Benchmark historyCleared measurements for an exact model, benchmark, and protocol. Availability differs by source; this is not every score on the website.All plansbenchlm_benchmark_history/mcp/fullGET /v1/history/benchmark_history
Benchmark changesFirst recorded catalog definitions and results, additions, removals, and reintroductions where retained.All plansbenchlm_benchmark_events/mcp/fullGET /v1/history/benchmark_events
Feedback on missing dataTell BenchLM which model, benchmark or use case the data could not answer, or what looks wrong. Your assistant can send it for you. It does not use a monthly read.Freebenchlm_feedbackPOST /v1/feedback

History queries cover the last 30 days on Free, the latest three calendar months on Data Pro, and all retained history on Research, by UTC date. Current ranking responses contain final BenchAlign outputs. Benchmark result queries contain source scores, including GPQA Diamond, SWE-bench Verified, and FrontierMath evaluated by Epoch AI. BullshitBench v2 includes judge ratings on a 0–2 scale, refusal rates, and published ranks from one pinned snapshot. Source-reported runs retain their original units and dates; artifact publication timestamps do not establish run dates. Arena overall snapshot history contains the publisher’s ratings, ranks, and confidence intervals across retained dates, with scoring methods kept separate. Benchmark measurements and evaluation runs do not establish a leaderboard rank. Missing records stay missing.

What’s new

Changes to the queries, limits, and data, newest first.

  • The value board names an efficient pick and the frontier

    benchlm_value_board now opens with efficientPick: the cheapest model scoring at least 90 percent of the top score (set retention from 0.5 to 1), next to the leader. Each row says whether it is on the price-performance frontier and how many models are as cheap or cheaper and score as high or higher, better on at least one. The score-per-dollar list is unchanged.

  • Large page sizes are lowered, not refused

    A limit above a tool's maximum now returns the largest page it allows, with limitApplied saying what was asked for and what was used, in place of an error. An unknown ranking surface now lists the surfaces that exist, and current rankings also take category as another name for surface.

  • Benchmark changes and score history are current again

    Benchmark changes (benchlm_benchmark_events) and benchmark score history (benchlm_benchmark_history) had stopped at October 1, 2026. Both now run to today and move forward with every data update. The six weeks in between are filled in from the catalog's own record: 862 dated changes to definitions, model coverage and the scoring basket. The freshness field on each answer gives the last recorded date.

  • Lab-reported results on their own boards

    Results a model's own developer reported now have their own boards, benchlm-lab:<benchmark>, apart from the benchmark's board. Each row names the lab, the metric, the setup and a link to the developer's publication, and never counts in a rank. Coverage finds them by benchmark key, and new results arrive with each history update.

  • Who ran each test, on BenchLM's recorded results

    Results on BenchLM's own recorded boards now say who ran each test: the model's own lab, the benchmark's maintainers or an independent evaluator, and whether the value counts toward the model's BenchLM rank. Filter a model's or a board's current results by it with runBy, and coverage shows each board's split in runByCounts.

Earlier changes
  • FrontierPhysics and BenchFlow's SkillsBench results

    Benchmark results now include two BenchFlow leaderboards. FrontierPhysics scores 26 agent setups on 56 end-to-end physics research tasks. BenchFlow's own SkillsBench v1.1 table scores 25 agent setups with and without Skills. SkillsBench history goes back to June 15, 2026, from three dated versions of BenchFlow's Hugging Face dataset.

  • The account page shows your usage

    Your Data account page now shows your reads per day, by kind of question and by app or key, and your recent calls with whether each used a read. It also lists what changed for models you asked about.

  • A core tool list at /mcp, every tool at /mcp/full

    https://data.benchlm.ai/mcp now lists 14 core tools at about a third of the old size, so assistants load less into every conversation. https://data.benchlm.ai/mcp/full lists all 28 with the same sign-in or API key. Every tool still answers by name at both addresses, so existing scripts keep working.

  • Tool descriptions are shorter

    The MCP tool list is about 40 percent smaller, so assistants load less into every conversation. Each description says what the tool answers. The tag and use-case ID lists appear once, on benchlm_recommend, and an unknown ID is refused with the valid ones. No tool or argument was removed.

  • Direction and unit on BenchLM's recorded boards

    BenchLM's own recorded boards now state whether higher or lower is better, and a unit where BenchLM states one, so a score such as 10 can be read. Model-card, capture and Arena boards can still be empty. The basis field says benchlm-catalog.

  • Multilingual speech recognition results

    Benchmark results now include David AI’s DAI-ASR-I18N leaderboard: transcription error rates for 20 systems and speaker diarization error rates for 17 systems on real two-person conversations in 21 languages, lower is better. History goes back to the board’s first version on October 2, 2026, one dated state per published version.

  • Compact benchmark results

    Current results and result coverage take compact: true. Each result row points to its source notice by index and each distinct license is stated once, so a page of scores is about a third of its usual size and every row still reaches its own license and attribution. Compact coverage lists boards and counts without notices.

  • Find the boards of one benchmark in coverage

    Result coverage now takes query, words to find in board ids, names, benchmark keys and source ids, and benchmarkKey, a benchmark key or catalog id such as agentic:hlab. Every word must match, so one call finds the boards for SWE-Bench Pro or Terminal-Bench instead of paging through every board. A search with no match says so in matchNote.

  • Answers say what changed since you last asked

    When you ask a connected assistant about a model you asked about on an earlier day in the last 30 days, the answer now notes what changed since then: a new retirement notice from its lab, a different list price, or a large move in its overall rank. There is nothing to set up. Answers over REST are unchanged.

  • Fifteen more Scale Labs boards

    Benchmark results now include 15 more Scale Labs boards, 324 rows: the retired SEAL boards (coding, math, instruction following, adversarial robustness, tool use, five languages), the first SWE-Bench Pro board and the early Humanity’s Last Exam previews. Each keeps its own metric; adversarial robustness counts violations, so lower is better.

  • Host prices and plans say how old they are

    Host prices and free-tier and plan answers now carry a freshness object: the day the returned rows were read, how many days ago that was, and a stale flag once the oldest is over 10 days old. Quote the date with the price.

  • Dollars a month on recommendations and comparisons

    Give monthlyInputTokens and monthlyOutputTokens to a recommendation or a comparison and each model carries its monthly cost in USD at list price, split into input and output. Caching and batch discounts are not applied, so a real bill can be lower. A model with no usable price says why instead of showing a number.

  • Model lookups accept names

    Model lookups and comparisons now take a typed name such as "Sonnet 5.5" or a router-style id such as "google/gemini-4-carbon", not only the exact key. The answer says what each name matched. A name that matches nothing, or more than one model, returns the three closest keys instead of a bare not-found.

  • The OpenAPI file covers every endpoint

    https://data.benchlm.ai/openapi.json now describes current benchmark results, result coverage, and every history endpoint: ranks, scores, benchmark events, saved snapshots, source runs, and price history. Each lists its parameters, limits, and the plan window, so a client generated from the file reaches all of them.

  • Recommendations skip a model once its replacement is listed

    The recommend answer now leaves out a model when a model that replaces it is also a candidate for the same answer, and names what it left out in supersededSkipped. An older model stays when its replacement does not fit your filters, for example a price cap. Set includeSuperseded to true to rank them all.

  • The change feed names every model

    Models that entered or left a ranking, and the rank moves, now carry the model name and its page link, the way price changes already did. One call to the change feed answers what changed, with no second lookup to turn a model key into a name.

  • Recommendations leave out retired models

    The recommend answer now leaves out a model once its lab has shut down its own API, so every pick is one you can still call there. Each answer names the models it left out, with the retirement date and the replacement the lab names. Set includeRetired to true to rank them again. Models that are only deprecated or replaced stay in.

  • Image, video, speech, and music prices

    Ask what it costs to generate an image, a second of video, a minute of transcription, or a song. The new query lists each published variant of a media model in the unit it is billed in, in USD as the provider lists it, with the day the page was read and its source. Rows are grouped by modality and unit and never converted into tokens, and a price kept after a model was shut down is marked historical.

    benchlm_list_media_pricesGET /v1/pricing/media
  • Input types for 146 more ranked models

    Another 146 ranked models now list the input and output types their provider documents, with the page and the day it was read, so 189 of the 214 ranked models carry them. Filtering by image-input, audio-input, video-input, or document-input now covers most of the ranked list. The other 25 stay unknown: their provider page is gone or does not state the inputs.

  • What a model takes in: images, audio, video, documents

    Models now list the input and output types their provider documents, with the page and the day it was read. Five tags follow: image-input, audio-input, video-input, document-input, and audio-output. This is recorded for part of the catalog so far; where it is not, the field is empty and the model is reported as unknown, never as text-only.

  • Free tiers, rate limits, and subscription plans

    Ask whether a provider has a free API tier, what its rate limits are, and what its monthly plans cost. One query returns free tiers, signup credits, and paid entry tiers for eleven API providers, and the plans for Claude, ChatGPT, Gemini, and five coding tools, each copied from the provider’s page with the day it was read. Filter by provider, by a model, or by a monthly price cap.

    benchlm_list_access_plansGET /v1/plans
  • What you need to run an open model yourself

    Open-weight models now say what it takes to run them. Each one carries its parameter count and the GPU sets its weights fit on at each quantization, with the recommended setup marked, and the Ollama tag, pull command, and download size when Ollama’s library has it. tags=ollama lists the models you can pull today, on the model list, recommend, the value board, and the feed.

  • Which hosts serve a model, and what each charges

    Ask where a model can be called by API and what each host charges. The new query lists the per-token price ten inference hosts and routers show on their own sites, cheapest first for a model, with the host’s model ID, the day the list was read, and its source page. A new hosted-api tag marks every model a host serves, so tags=open-weight,hosted-api finds open models you can use without running them yourself.

    benchlm_list_host_pricesGET /v1/pricing/hosts
  • Retired and replaced models are flagged, and you can leave them out

    Each model now lists the deprecation and retirement notices its provider has published, with the date, the replacement the provider names, and a link, plus the newer models that replaced it. Four tags follow: api-deprecated, api-retired, superseded, and coming-soon. Add excludeTags=api-retired,superseded to recommend, the value board, the feed, or the model list to keep only models still served and not replaced.

  • Free models plus the labs you pick, in one request

    Recommend, the value board, the model feed, and the model list filter on the server by tags and by several creators at once. With match=any, tags=free-api and creator=Anthropic,OpenAI return the free models plus every model from those two labs, so you no longer fetch the whole list and filter it yourself.

  • Every model is tagged: open, licensed, free, and from where

    Each model now carries tags for open weights or proprietary, license class, self-hosting, a free API, the country and region of its lab, and reasoning, along with its license and lab origin as fields. The tags come from the catalog and the price ledger, so a new model is tagged the day it is listed. Add taxonomy=true to the model list for the full set.

  • Recommend shows the benchmarks behind a use case

    Ask for a use case, such as legal or software development, and each pick now names the benchmarks tagged with that use case that have a published score for the model, with a link to each. Add scores=true to get the score BenchLM serves for each one in the same answer, with its source notice.

  • Seven new benchmark boards

    Current results now include BabaIsBench, BinaryAudit, OTelBench, and CompileBench from Quesma, StealthMark, and Epoch AI’s SpireClimb and Epoch Automation Reports. Find them in the benchmark catalog by name, then query their scores like any other board.

  • Free now includes 1,000 reads a month

    The Free allowance went from 250 to 1,000 successful reads a month. Usage and coverage checks still don’t use a read.

  • Which model to use, answered in one call

    Three new queries cover the questions people ask most. Recommend returns up to ten models for a use case or ranking, within a price cap. Compare puts two to five models side by side. The value board ranks models by score per dollar. All three work over MCP and REST, and every row keeps its evidence label.

    benchlm_recommend · benchlm_compare_models · benchlm_value_boardGET /v1/recommend · GET /v1/compare · GET /v1/value
  • What changed since a date

    One query returns the models that entered or left a ranking, the largest rank moves, and price changes since the date you give. When scores were recalculated in that window, those moves are reported apart from changes in the models.

    benchlm_changesGET /v1/changes
  • A model feed for routers and dashboards

    A paged feed of every ranked model with its ranks, list prices, context window, and open-weight flag, up to 250 models per page. The REST paths are now described in a public OpenAPI document.

    benchlm_model_feedGET /v1/feed/models
  • A year of history in one request

    Saved rank and score history accepts windows of up to 366 days, where your plan covers them. Daily or weekly granularity keeps a year on one page.

  • Report what the data couldn’t answer

    Your assistant can send a short note when a model, benchmark, or use case is missing, or when a number looks wrong. It doesn’t use a read.

    benchlm_feedbackPOST /v1/feedback
  • Connect buttons and more forgiving search

    The connect section has a button for each client that opens its install flow or copies its command. Search ignores hyphens, dots, and spaces, so “terminal bench” and “Terminal-Bench” find the same rows.

Start free.
Automate it when you are ready.

Free includes 1,000 reads a month, with no card. Data Pro gives scripts, agents, and dashboards 25,000 reads a month and 60 requests a minute for $49 USD/month, and a first subscription starts with a 7-day free trial.

Looking for the website’s existing downloads and their terms? Open the Data hub and downloads.