DiffusionGemma

DiffusionGemma is an experimental open-weight large language model developed by Google DeepMind for multimodal text generation through discrete diffusion. Released on June 10, 2026, it differs from an autoregressive model that finalizes output one token at a time. DiffusionGemma iteratively denoises a 256-token canvas: it evaluates every position with bidirectional attention, accepts a subset of confident predictions, and revises the rest over multiple passes.^[1][2][3]

The public checkpoint is google/diffusiongemma-26B-A4B-it. It is based on the Gemma 4 26B A4B backbone and contains 25.2 billion total parameters, with about 3.8 billion active for each token. Google released BF16 weights under the Apache License 2.0 and classifies the model as experimental rather than its preferred production model for maximum output quality.^[1][3][6]

Release and specifications

Google released one instruction-tuned DiffusionGemma checkpoint in its official collection. The repository name rounds 25.2B total parameters to 26B and 3.8B active parameters to A4B. The active figure describes routed computation, not checkpoint size or memory footprint.^[3][6]

Scroll sideways for more →

DiffusionGemma accepts interleaved text and images. It processes video by sampling frames and supports up to 60 seconds at one frame per second. The release does not support audio input, even though the model card lists audio among the pretraining data types. The model card claims out-of-the-box support for more than 35 languages after pretraining exposure to more than 140.^[2][3]

Block-diffusion architecture

The model uses one fine-tuned Gemma 4 backbone in two modes. A causal encoder reads the prompt and earlier completed output into a KV cache. A bidirectional decoder then denoises the current 256-token canvas while cross-attending to that cached prefix. Once the canvas is complete, the encoder adds it to the cache and the decoder starts a new canvas. Generation is therefore diffusion-based within a block and autoregressive across blocks.^[3][4]

This arrangement resembles the block-diffusion family described in prior research. Block diffusion combines parallel sampling within fixed blocks with causal dependencies between blocks, which permits variable-length generation and reuse of cached prefix computation.^[8] Earlier large diffusion language models such as LLaDA commonly used absorbing-mask corruption, where a special mask token stands for unknown text.^[9] DiffusionGemma instead uses uniform-state discrete diffusion: it initializes the canvas with random tokens drawn from the vocabulary and can replace uncertain positions with fresh random tokens during denoising.^[4]

Each denoising pass predicts distributions for every canvas position. Self-conditioning converts the previous pass's distributions into vector representations and feeds them into the next pass. The recommended entropy-bound sampler selects the lowest-entropy token predictions while keeping their mutual-information bound below 0.1, then fully re-noises unselected positions.^[2][3][4]

The standard sampler allows at most 48 denoising passes and lowers its temperature from 0.8 to 0.4. It stops early only when the canvas's average model entropy is below 0.005 and the most probable predictions remain unchanged across two consecutive passes. Google says this usually takes 12-16 passes, depending on the task. The model card describes 15-20 finalized tokens per forward pass.^[2][3]

A 256-token canvas consequently does not mean that one forward pass produces 256 finished tokens. All positions receive parallel computation, but commitment is iterative. The documented algorithm also confines revision to the active canvas. After a completed block enters the causal prefix, later canvases condition on it rather than denoising it again.^[3][4]

Capabilities and intended uses

Google positions DiffusionGemma for low-concurrency, single-user workflows where response latency matters: inline editing, rapid local iteration, code completion and infilling, structured text, mathematical notation, and other outputs whose positions can constrain one another. It also supports multi-turn chat, configurable reasoning output, system prompts, and structured function calling.^[1][2][3]

The model's image capabilities include optical character recognition, document parsing, charts, screen understanding, object detection, handwriting, and pointing. A visual token budget from 70 to 1,120 tokens trades image detail for compute. Lower budgets target classification or video frames, while higher budgets preserve small text for OCR and document work.^[3]

Google describes DiffusionGemma as a research model for developers studying fast natural language processing and vision-language systems. The launch post explicitly recommends standard Gemma 4 instead when maximum output quality is the priority.^[1][3]

Training and data

Google says the Gemma 4 26B A4B backbone was fine-tuned to perform both causal encoding and bidirectional denoising. The public explanation identifies uniform-state corruption and self-conditioning but does not give the adaptation dataset size, diffusion training schedule, token count, optimizer, training hardware, duration, energy use, or compute budget.^[4]

The model card describes a pretraining mixture of web documents, code, mathematical text, images, and audio with a January 2025 cutoff. It says the corpus covers more than 140 languages. Google applied multiple rounds of child sexual abuse material filtering, automated removal of some personal and sensitive information, and additional quality and safety filters.^[3]

No full corpus, source-domain proportions, image provenance list, or dataset-level license inventory is published. Google also does not separate the original Gemma 4 pretraining mixture from any data used specifically to adapt the backbone for diffusion or instruction following. The released implementation supports inference and fine-tuning, but it is not a reproduction package for the original model training.^[3][4][7]

Weights, software, and hardware

The official BF16 checkpoint is available from Hugging Face, with additional access through Kaggle and Google Cloud Model Garden. Google's google-deepmind/gemma repository contains the JAX diffusion transformer, sampler, early-stopping logic, tests, example notebook, and an adapter for the Hackable Diffusion fine-tuning toolkit. Transformers, vLLM, SGLang, and MLX are documented inference routes; Google also names Nvidia NIM for deployment.^[2][5][6][7]

The weights and official implementation use the Apache License 2.0. Community quantizations and ports are separate artifacts. At launch, Google said official llama.cpp support was still forthcoming, so the existence of a community GGUF file did not by itself establish full upstream support.^[1][5]

Google says a quantized model can fit within 18 GB of video memory. That claim does not describe the BF16 checkpoint or guarantee that every context length, vision-token budget, batch size, and cache configuration fits in 18 GB.^[1][2]

For speed, Google reports more than 1,100 tokens per second on one H100 using FP8 at low batch, more than 1,000 tokens per second in the launch summary, and more than 700 tokens per second on an RTX 5090. It characterizes the best result as up to four times faster than autoregressive generation on dedicated GPUs.^[1][3]

These figures describe per-user output under favorable accelerator conditions. The launch post does not provide enough prompts, output lengths, repeated-run statistics, or complete baseline configurations to reproduce the four-times figure from the article alone. Google also says the advantage diminishes under high request volume, where autoregressive servers can batch many sequences and DiffusionGemma can cost more to serve. Its footnote warns that unified-memory systems such as Apple Silicon may not receive the same acceleration.^[1]

Evaluation

Google compares the instruction-tuned DiffusionGemma checkpoint with its autoregressive Gemma 4 26B A4B sibling. Both use the recommended entropy-bound sampler where applicable. Selected model-card results are:^[3]

Scroll sideways for more →

DiffusionGemma trails its autoregressive sibling on every listed row except Humanity's Last Exam without tools. The 32.0% MRCR result also shows that a 256K capacity limit does not guarantee reliable retrieval across that span. Google's conclusion that the model trades quality for speed is consistent with its own benchmark table.^[1][3]

MMLU-Pro expands multiple-choice questions to ten options and removes noisy or trivial items to emphasize reasoning.^[11] LiveCodeBench continuously collects recent coding problems to reduce contamination.^[12] Google identifies benchmark versions and some tool conditions, but the model card does not publish complete prompts, seeds, sample counts, variance, or a runnable evaluation harness. The results are author-reported, not independent replications.

An independent June 2026 preprint instrumented the public checkpoint on 686 prompts across six regimes. It found large simultaneous token-commit batches, together with a partial left-to-right bias whose measured strength changed with analysis granularity. Structured JSON showed nearly arbitrary commit order. Confidence correlated with correctness on the paper's mathematics prompts but not on factual recall.^[10] This study supports parallel commitment while qualifying the simpler description that a whole paragraph appears at once. It did not reproduce Google's GPU speed comparison and had not undergone peer review.

Safety and limitations

Google says DiffusionGemma underwent automated and human safety testing for child sexual abuse and exploitation content, dangerous instructions, sexually explicit output, hate speech, and harassment. The tests were run without external safety filters to examine the raw model. The model card reports fewer policy violations than Gemma 3 and Gemma 3n while keeping unjustified refusals low.^[3]

Those results are qualitative. Google does not publish per-category violation rates, sample counts, thresholds, language coverage, adversarial prompts, or a numerical refusal table for DiffusionGemma. The card advises deployers to add safeguards suited to their application and identifies misinformation, privacy violations, malicious use, and inherited social bias as risks.^[3]

The model can produce false or outdated claims, miss language nuance, and struggle with common sense or open-ended tasks. Multimodal input creates further dependence on resolution and visual-token settings. Function calling requires an external runtime to validate and execute requests; model support for a tool-call format does not make the resulting action safe.^[3]

Diffusion introduces its own operational tradeoffs. Quality and latency depend on the entropy sampler and the number of denoising passes. Revision stops at the current canvas boundary, long-context retrieval is weak in the disclosed MRCR comparison, and the speed advantage is specialized for compute-heavy, low-batch accelerators. DiffusionGemma is therefore an experimental latency-focused vision-language model, not a general replacement for Gemma 4 or evidence that diffusion generation is faster on every workload.^[1][3][10]

References

1. O'Donoghue, Brendan; Flennerhag, Sebastian. DiffusionGemma: 4x faster text generation. Google, June 10, 2026. ↩
2. Google AI for Developers. DiffusionGemma model overview. Last updated June 10, 2026. ↩
3. Google AI for Developers. DiffusionGemma model card. Last updated June 10, 2026. ↩
4. Google AI for Developers. Diffusion in Text Generation Explained. Last updated June 10, 2026. ↩
5. Ballantyne, Ian; Sanseviero, Omar. DiffusionGemma: The Developer Guide. Google Developers Blog, June 10, 2026. ↩
6. Google DeepMind. diffusiongemma-26B-A4B-it model repository. Hugging Face, 2026. ↩
7. Google DeepMind. Gemma diffusion implementation. GitHub, 2026. ↩
8. Arriola, Marianne; Gokaslan, Aaron; Chiu, Justin T.; et al. Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models. ICLR 2025. ↩
9. Nie, Shen; Zhu, Fengqi; You, Zebin; et al. Large Language Diffusion Models. arXiv:2502.09992, 2025. ↩
10. Asaria, Ali; Salomone, Tony; Gandhi, Deep. Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens. arXiv:2606.14620v1, June 12, 2026. ↩
11. Wang, Yubo; Ma, Xueguang; Zhang, Ge; et al. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. arXiv:2406.01574, 2024. ↩
12. Jain, Naman; Han, King; Gu, Alex; et al. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv:2403.07974, 2024. ↩