Diffusion LLMs Ranked by Evidence

Only two diffusion language models have current, comparable hosted runtime rows: Celeris-1 and Mercury 2.

Celeris-1 leads those rows on both output rate and first-answer delay.

We leave the other entries unranked because their evidence uses different workloads and timing boundaries.

That is a smaller leaderboard than the marketing category suggests.

Only two hosted runtime rows are comparable

Table 1 Download CSV

Rank	Model	First answer	Output rate	Evidence status	Production access
1	Celeris-1	0.64 s	2,086 tok/s	Cross-provider runtime row	Hosted API, US access
2	Mercury 2	3.53 s	802 tok/s	Cross-provider runtime row	Hosted API
—	Gemini Diffusion	Not comparable	1,479 tok/s sampling claim	Google-published experimental test	Demo, no general API
—	DiffusionGemma	Not comparable	More than 1,000 tok/s claim	Google-published H100 test	Open experimental model
—	LLaDA	Not comparable	No hosted row	Peer-reviewed research preprint	Open research implementation
—	Dream 7B	Not comparable	No hosted row	Research paper and code	Open research implementation

The first two rows share the current runtime feed and can be ordered within it. Celeris-1 is not yet eligible for the overall quality leaderboard because its sourced task coverage is narrow. Mercury 2 also remains unranked on overall quality. Speed evidence can identify a runtime leader without proving the better model.

The remaining speed figures do not share the same test. Google's Gemini Diffusion page labels 1,479 tokens per second as sampling speed and lists 0.84 seconds of separate overhead. DiffusionGemma's page says more than 1,000 tokens per second on one H100. Neither result is a hosted cross-provider measurement, so neither receives a synthetic rank.

We use an evidence ladder to decide what belongs in the numbered rows. A current hosted runtime measurement can rank against another row from the same feed. A provider test can describe its own model when hardware, workload, and timing boundary are visible. A paper can establish an evaluation for the exact checkpoint and protocol. None can silently move into another tier.

Table 2 Download CSV

Evidence tier	What it supports	What it does not support
Shared hosted runtime	Ordering inside the same snapshot	A permanent speed guarantee
Provider performance report	A dated result on the stated setup	Cross-provider rank without a shared method
Research paper	Reproducible checkpoint and protocol claims	Managed-service latency or availability
Architecture description	How the provider says generation works	Task quality or production throughput

This ladder also prevents missing rows from becoming zeros. A research model with no managed endpoint is unranked, not slower than every commercial service.

Diffusion is a decoding method, not a quality rank

An autoregressive model commits to the next token, appends it to the sequence, and repeats. A diffusion language model starts from a noisy or masked representation and revises several token positions over a series of denoising steps. That parallelism can reduce the number of sequential model invocations needed to produce a response.

Parallel decoding does not eliminate order from language. Implementations still need to preserve dependencies, decide which positions are ready, and spend additional work correcting uncertain tokens. Serving software, hardware utilization, prompt processing, batching, and the requested reasoning budget can dominate the user-visible result.

This is why architecture alone cannot predict time to first answer.

Mercury 2 has fast output, not the fastest start

Inception's Mercury 2 release reports 1,009 tokens per second on NVIDIA Blackwell, a 128K context window, tool use, schema-aligned JSON, and OpenAI-compatible chat completions. The official rate is a provider result. The hosted row here measures 802 tokens per second, which is still high, alongside a 3.53-second first-answer delay.

Those two facts can coexist. Output rate measures how quickly tokens arrive after generation begins. First-answer latency measures the wait before usable output appears. A workload that needs a long answer may value Mercury's throughput, a router that returns one short label may care almost entirely about the initial delay.

Inception prices Mercury 2 at $0.25 per million input tokens and $0.75 per million output tokens. That makes it inexpensive to replay against a real task set. It does not make provider quality tables independent evidence, so the model page keeps its stored benchmark rows out of the public ranking until exact verification is attached.

Celeris-1 leads the comparable speed rows

Celeris-1 records 2,086 output tokens per second and 0.64 seconds to first answer in the current hosted runtime feed. Celeris separately reports 1,664 tokens per second at p50 on a 1,000-token provider workload. We keep the two results separate because the prompts and timing method differ.

The model accepts text and images, returns text, exposes a 131,072-token window, and uses an OpenAI-compatible endpoint. Official pricing is $2 per million input tokens and $6 per million output tokens. Public API access is currently limited to the United States.

The evidence limit is task coverage. Celeris has one source-attached MMLU-Pro result and does not yet have enough verified categories for an overall rank. A fast knowledge-test response does not establish coding, multilingual, tool-use, or visual performance.

Gemini Diffusion is still an experiment

Google DeepMind's Gemini Diffusion page presents an experimental text model and an interactive demo. Its table reports 1,479 tokens per second of sampling speed, 0.84 seconds of overhead, and provider-run task results. Google explicitly separates sampling from overhead, which is helpful, but the deployment is not a generally available API that can enter the hosted test unchanged.

The evidence supports a narrower conclusion: Google has demonstrated a high-throughput diffusion model in a controlled product preview. It does not support placing Gemini Diffusion above or below Celeris-1 on a shared production leaderboard.

DiffusionGemma is the more accessible Google research path. It is an open, experimental mixture-of-experts model with 26 billion total parameters and 3.8 billion active parameters. Google reports more than 1,000 tokens per second on one H100. Teams can inspect and serve it, but must supply their own production measurements.

LLaDA and Dream belong in a research table

The LLaDA paper established a masked-diffusion approach for large language models and released training and evaluation details. Later work extends the family, but paper results should remain attached to their exact checkpoint and protocol. There is no current managed endpoint in the hosted runtime feed.

Dream 7B publishes model code and checkpoints for diffusion-based reasoning. Its value in this tracker is reproducibility: researchers can inspect the implementation and run controlled comparisons. That is different from a service-level claim about tail latency, concurrency, or regional availability.

Calling either model slow because it lacks a hosted row would be wrong. Calling it production-fast without one would be wrong too.

How to choose a diffusion model

Start with the path the application needs:

1. Short interactive responses: weight first-answer latency heavily and replay short prompts. Celeris-1 is the current measured leader, subject to its US-only availability and limited quality coverage.
2. Long generated responses: measure both first answer and sustained output. Mercury 2's throughput and price make it a practical hosted candidate.
3. Architecture research: use DiffusionGemma, LLaDA, or Dream when weights, code, and controlled experiments matter more than a managed endpoint.
4. Google product evaluation: treat Gemini Diffusion as an experiment until the API, model identifier, quotas, and production terms are public.

For every path, hold prompts, output length, region, concurrency, and hardware constant. Report median and tail behavior. Then run the same task-success gates used for autoregressive models.

Use at least three output lengths. A 20-token routing response emphasizes first-answer delay, a 500-token explanation mixes the opening wait with generation, a 2,000-token result makes sustained output dominant. Record time to the first usable chunk and to the valid completed response, especially for JSON and tool calls that cannot be consumed halfway through.

Concurrency can reverse a single-request result. Send isolated requests first, then repeat at the expected parallel load. Keep rate-limit errors, timeouts, and retries in the report. A system that serves one request at 2,000 tokens per second but queues the tenth request for several seconds may still be the wrong production choice.

Deployment terms are part of the ranking decision. Celeris-1 currently restricts public API access to the United States. Gemini Diffusion has no general production endpoint. Open models require hardware, an inference engine, monitoring, and an operator. Mercury 2 is managed, but the team should confirm regional routing, quotas, and support before treating a successful development request as production availability.

Recheck the model identifier and evidence date when repeating the test. “Mercury,” “Mercury 2,” a research checkpoint, and a provider demo may share an architecture family while representing different systems. The row must name the exact artifact that produced it.

The useful leaderboard is the one that admits when a row does not belong.