Every change to the models you run, with its source and its date. Releases, price changes, retirements, API changes, and incidents.Every change to the models you run, with its source.
Follow model changesEvidence for choosing and evaluating AI models, grounded in benchmark explainers, model comparisons, and the BenchLM dataset.
Featured research
Structured output guarantees the shape of a response, not the truth in it. Four synthetic pairs from one support-ticket fixture show a schema pass hiding a wrong field, then two filed runs of Claude Sonnet 5 and GPT-5.6 Terra show the same failure class with denominators.
Glevd · September 14, 2026 · 13 min read
Read the articleA chronological record of benchmark analysis, model decisions, and evaluation practice.
Showing 87 of 88 articles, plus the featured article above
Deprecated means flagged, not brokenDeprecated means still working, stop building on it. Five AI providers spell that state five different ways, and on Microsoft Foundry the docs and the API use the word to mean opposite things.Research notes13 min read
The same AI model retires on three different datesClaude 3 Haiku has three retirement dates across three routes, 143 days apart. Opus 4.1 has two and a pricing tier in between. The receipts, the policy language that causes it, and the ten-model table.Research notes15 min read
OpenAI Codex deprecation: the date, the reason, and what replaced itOpenAI shut down the original Codex API in March 2023 with three days' notice. The dates, the stated reason, what replaced it, and why the name now means something else.Research notes13 min read
Fast on a wafer is not a shortlistCerebras is quicker than Groq on the models both host. The decision is which weights sit on the wafer, and whether your agent is actually waiting on tokens.Pricing and cost11 min read
Groq is not a modelAfter Nvidia licensed the chip and hired the team, what is left is a fast, narrow catalog. Most people searching “groq” still think they are buying a brain.Pricing and cost11 min read
K3 is open. You still cannot run it.Moonshot’s 2.8T flagship sits next to the closed frontier. Open here means downloadable shards and a license, not a machine you own.Benchmark explainers12 min read
Opus 5 ate the default. Fable is waiting.Claude Opus 5 is Fable-adjacent at $5/$25. That is the default now. Fable 5 still has a job, and 5.1 has to reopen a gap or admit the specialty is gone.Model comparisons13 min read
We measure the session. They sell the week.Grok Bot launched August 11 as an always-on cloud computer. Hermes and OpenClaw ship the same architecture. Our agentic scores still time a session, not a week.Benchmark explainers10 min read
How to Monitor OpenAI API ChangesOpenAI now offers an official release-notes RSS feed, but API changes still span changelog, deprecation, pricing, status, and model-ID surfaces. Here is how to monitor the gaps.Research notes8 min read
Claude Pro vs Max: Which Plan Is Worth It?Claude Pro is $20 monthly. Max costs $100 or $200 for 5x or 20x the usage, with one important Fable 5 difference. Here is when each plan pays.Pricing and cost9 min read
Claude Opus 5 changes the Claude defaultClaude Opus 5 hits 96% on SWE-bench Verified at Opus 4.8's $5/$25 price. Fable 5 still wins harder repo tests, so the upgrade case is not universal.Benchmark explainers8 min read
AI Voice Agents for Customer ServiceCustomer-service voice agents work best on bounded, tool-backed workflows. This guide covers architecture, model choice, rollout gates, and a transparent cost example.Pricing and cost11 min read
Best Enterprise AI Search ToolsGlean leads for a managed company-wide rollout, but Google, Perplexity, Microsoft, Elastic, Onyx, Coveo, Guru, and SearchBlox fit different search estates.Model comparisons13 min read
Diffusion LLMs Ranked by EvidenceMercury 2 and Celeris-1 have comparable hosted runtime rows. Gemini Diffusion, DiffusionGemma, LLaDA, and Dream remain provider or research evidence.Model comparisons9 min read
Diffusion vs Autoregressive Language ModelsAutoregressive LLMs generate left to right. Diffusion LLMs revise multiple token positions over denoising steps. Architecture changes the speed trade, not the need to test.Research notes10 min read
Nine Glean Alternatives for Enterprise SearchThe right Glean alternative depends on the estate: Microsoft, Google Cloud, Elastic, customer service, verified knowledge, web research, open source, or private deployment.Research notes13 min read
How LLM Enterprise Search WorksLLM enterprise search retrieves permission-safe evidence before generating an answer. Connectors, identity, ranking, citations, and abstention matter more than the chat box.Research notes10 min read
Mercury 2 vs Gemini LiveMercury 2 is a cheap text reasoning layer for a cascaded voice stack. Gemini 3.1 Flash Live Preview is native audio with preview-model risk.Model comparisons11 min read
Mercury 2 vs GPT-Realtime 2.1Mercury 2 is the cheaper text model for a cascaded voice stack. GPT-Realtime 2.1 is the native speech-to-speech choice. The latency numbers are not interchangeable.Model comparisons11 min read
Time to First Token ExplainedTTFT measures the wait before a model begins answering. It is not output speed or full application latency, and it should be measured as a distribution.Research notes11 min read
What Is an AI Voice Agent?An AI voice agent listens, decides, uses tools, and speaks inside a live conversation. Its quality comes from the full loop, not one model.Research notes10 min read
Best LLM Observability ToolsAs of July 28, 2026, Langfuse is our pick for teams that need traces, evaluations, datasets, and an open-source self-hosting route in one product. LangSmith, Phoenix, Braintrust, Helicone, and Datadog fit different operating constraints.Engineering guides13 min read
Twelve Prompt Fixes, Including One RegressionWe ran twelve before-and-after AI prompt examples three times each. Revised prompts improved bounded work overall, but one detailed rewrite made the result worse.Research notes9 min read
Marketing Drafts That Stay Inside the FactsWe tested 24 marketing drafts against one product source ledger. The useful outcome was copy that followed the facts and requested format before editorial review.Research notes8 min read
Write Prompts Around the Finished WorkA practical six-part method for writing AI prompts that produce testable work: outcome, context, sources, constraints, output contract, and acceptance check.Research notes9 min read
Four Prompt Optimizers, One Test SetWe attempted 48 public prompt-optimizer rewrites and ran every usable artifact three times. Shared cases tied on accepted outputs; availability was the observed difference.Research notes8 min read
Coding Changes That Survive ReviewWe ran 24 AI coding changes against a frozen repository. Better prompts did not improve the test-pass rate; they reduced unrelated code and review surface.Research notes9 min read
The Latency Tax in Agentic WorkflowsAgentic workflows compound latency across model calls, tools, retries, and handoffs. Learn the core patterns, see practical examples, trace the critical path, and build a faster agentic AI workflow without sacrificing reliability.Research notes18 min read
How to Optimize an AI PromptOptimize AI prompts with a test loop: preserve facts and constraints, define the output contract, test representative inputs, and fix one failure at a time.Research notes10 min read
Is ChatGPT Plus Worth It?As of July 17, 2026, the $8 Go tier carries the same 160-messages-per-3-hours chat allowance as Plus, and OpenAI's own rate card budgets $100–$200 a month for a developer using Codex. We read every current OpenAI pricing page. Plus is worth $20 for exactly two kinds of user.Pricing and cost9 min read
Kimi K3: The Open Model Closing the GapMoonshot released Kimi K3's full 2.8T-parameter weights on July 27, 2026. The open release is real; serving it still demands infrastructure on the scale of the model.Benchmark explainers5 min read
Thinking Machines Chose Open Weights FirstWhy Thinking Machines made its first foundation-model release open weight, what Inkling changes for the lab, and where it falls short of the closed frontier.Benchmark explainers12 min read
Best LLM for Math 2026: AIME, HMMT & MATH-500 RankingsBest LLM for math 2026: GPT-5.4 leads AIME 2025, MATH-500, and BRUMO. Compare Claude, Gemini, DeepSeek-R1, GPT-5.5, and value picks by use case.Model comparisons10 min read
Test the replacement on what already failsA replacement model can raise the benchmark score and still break the field your application needs. Build a small workload evaluation from your own failing cases, run both configurations against the same checks, and keep the incumbent when the numbers say so. With a downloadable kit and two filed runs.Benchmark explainers12 min read
The Complete Guide to LLM Benchmarking: Everything You Need to KnowEverything you need to know about LLM benchmarking — what benchmarks measure, how to choose the right ones, common pitfalls, and how to interpret results for real-world model selection.Benchmark explainers15 min read
How to Interpret LLM Benchmark Results: A Practical GuideHow to read LLM benchmark scores correctly — what differences are meaningful, what to ignore, common misinterpretations, and how to translate benchmark data into model selection decisions.Benchmark explainers10 min readNew analysis, material benchmark and price changes, and one model decision worth revisiting.
Read a sample issueJoin 2,000+ readers.