Skip to main content
BenchLM
Data
Sent every week

The model changes worth your attention

Rank moves, price changes, launches, and one clear decision note. One email every week.

Get the next weekly brief

Read the sample below first. Subscribe when the format earns the space in your inbox.

Join 6,100+ subscribers.

What arrives

Useful change, not a news firehose

Each issue is built for people choosing, buying, or operating language models.

Rank moves with the evidence attached

See who moved, what changed, and whether the public benchmark coverage supports the move.

Material API price changes

Track pricing that could change a production shortlist, without filling your inbox with every minor update.

Release and availability context

Separate a launch headline from the access limits, evidence gaps, and reasons to wait.

One decision note

Leave with a model to consider, a lower-cost alternative, or a clear reason not to switch yet.

Real sample issue

See the whole format

This is the editorial content of a sent issue, adapted from email to the web.

Weekly LLM benchmark digest

DevDay: Dots, cheaper Sol, and faster Astra

OpenAI's DevDay keynote on September 29, 2026, introduced always-on agents, a cheaper model for complex work, and a faster way to run Astra. These are the releases we'd look at first.

The DevDay releases we'd look at first

Dots and Space bring ongoing work into ChatGPTDots keep working between conversations, using a cloud computer, a browser, and connected apps. Space puts shared pages and files beside that work, with teammates and Dots collaborating in the same place. Dots launch on Pro and Business Premium in eligible markets, with an admin-enabled Enterprise beta and gradual rollout. Space is available on all Pro, Business, and Enterprise plans. SMS access and presentations in Space are coming later.GPT-6.1 Sol lowers the price of complex workOpenAI positions Sol near Astra in capability, at a fifth of the price. Its Standard API rates are $2 input and $10 output per million tokens. Cache reads cost $0.10, a 95% input discount, while cache writes cost $2.50. Those base rates cover prompts up to 272,000 input tokens. The saving depends on reuse; we haven't independently measured the new model's quality.Ultrafast trades a higher bill for speedOpenAI claims up to eight times Standard token generation for Astra in Codex, reaching up to 300 tokens per second. Its official recap puts API speed at up to six times Standard. The keynote priced the tier at six times Standard. It announced Ultrafast across the API, ChatGPT, and Codex, with GPT-6.1 Sol support to follow. Pro 500 costs $500 a month and includes Astra Ultrafast access. Pro 200 reopens at $200. Faster generation doesn't make tests or other tool calls finish eight times sooner.Codex cloud and Agents API keep tasks runningCodex can continue work when your laptop closes, with handoff across phone, browser, and desktop. Codex Security Cloud adds cloud vulnerability scans and proposed fixes. The Agents API public beta exposes the managed Codex harness to developers, including durable sessions and orchestration. Decisions API, for quick choices from predefined options, and Private Intelligence were announced as previews.Your subscription can travel to supported appsSign in with ChatGPT can let eligible subscribers use their plan in participating applications. Sign-in and plan usage are separate permissions. Plugin extensions add fuller interfaces inside ChatGPT and Codex, and OpenAI Marketplace lets enterprise customers spend part of an existing commitment on partner software.

Analysis worth opening

Archive copy reflects the rankings, prices, and availability stated when this issue was sent. Current pages may show newer evidence.

Past issues

Read before you subscribe

Every archived issue is dated. Its claims stay tied to the evidence available on that send date.

DevDay: Dots, cheaper Sol, and faster AstraThe important releases, what they cost, and which features are still previews.Jev cannot write. We gave it a job.Nineteen questions in one call: why TypeSafe’s decision model is getting attention, and where its claims stop.DeepSeek scheduled a reroute, then cancelled it in a footnoteV4 Pro was due to become V4.1 Flash on September 14. The pricing page now says it stays, and the two are 4.4× apart on input.I paid seven platforms to beat OpenRouter. OpenRouter won.5,400 paid requests, $13.07 of invoices, and a scoreboard no page selling a gateway could publish.Three days was the whole migration budgetThe 2023 Codex shutdown, dated and sourced — plus the free Radar stack that counts your runway.OpenAI is keeping ZDR. The safety system is moving.Private Safety Processing is a September preview, not a finished contract.Daybreak Red does not replace BlueGPT-5.6 Cyber vs Sol: the right Daybreak tier, plus persistent Codex Cloud scans.Open weight is eating the frontierDeepSeek shipped 0731 weights; Qwen3.8-Max remained a hosted preview.The frontier is outrunning the scorecardsKimi K3 followed GPT-5.6 Sol by seven days. Comparable evidence is the scarce input.GPT-5.6 arrives, locked behind a previewGPT-5.6 lands in a government-gated preview, Anthropic holds the top two spots, and GLM-5.2 enters the top five.The frontier moved again, twiceOpus 4.7 reaches general availability, Kimi 2.7 enters preview, and the gap between major releases keeps shrinking.

Sponsored editions

Paid partner briefs are labeled before the first partner link. Provider-run claims stay attributed to the sponsor.

Sponsor a newsletter issue
Subscribe to the weekly brief