Intrinsic Distribution Matching

Intrinsic Distribution Matching (IDM) is a method for training image generators that use one or two generator evaluations. Introduced in an October 29, 2026 preprint by Deyuan Liu and collaborators, including researchers at InclusionAI, it combines distribution matching distillation with statistical constraints derived from the generator's own features. Experiments use Z-Image-Turbo 6B and Qwen-Image 20B, with full-parameter and LoRA tuning.^[1]

Background

A diffusion model normally produces an image through repeated denoising evaluations. Knowledge distillation can train a faster generator to approximate a slower model. Distribution Matching Distillation (DMD), introduced by Yin and colleagues, instead compares distributions using their scores: gradients of log probability density. Its training signal depends on the difference between scores for reference images and generated images. Conventional DMD uses a reference diffusion model and a separately trained model of the generator's changing output distribution.^[2]

How IDM works

Following APEX, IDM obtains the two velocity estimates from branches of the same network, selected with opposite conditioning vectors. The generated-distribution branch remains learned. IDM retains the complete DMD correction and adds a feature-moment loss; moment matching does not replace the score-based objective.^[3]

An exponential moving average (EMA) of generator parameters supplies the feature extractor. It mean-pools the final-layer activations at high noise levels. Generated and reference samples use the same representation. The added loss compares feature means and per-coordinate standard deviations:

$L_{\mathrm{pop}}=\|\mu_g-\mu_r\|_2^2+\sum_j(\sigma_{g,j}-\sigma_{r,j})^2.$

Here, $g$ denotes generated samples and $r$ reference samples. This matches diagonal Gaussian feature proxies, rather than full feature distributions. Statistics pool across prompts, so their agreement does not establish agreement for every individual prompt.^[3]

During an update, EMA parameters stay fixed but gradients pass through the feature extractor to current generated images. Detached queues hold historical features. These training operations require memory and computation even though no external feature encoder is needed. Inference uses the generator, text encoder and variational autoencoder, without feature queues or moment calculations.^[3]

The paper motivates the added constraint by decomposing a reverse-KL score field into a Gaussian moment component and a non-Gaussian residual. IDM does not separately estimate those sample-space components. Its objective combines a recursive consistent generation loss, DMD regression and feature-moment matching, weighted 1 and 0.5 for the latter two terms. Velocity and endpoint regressions use the same detached DMD correction.^[1]

Reported results

The authors report the following Z-Image results. Scores are percentages; higher is better.^[4]

IDM leads the compared two-evaluation methods on both LongText benchmarks. APEX scores higher on DPG-Bench, and the eight-evaluation reference scores higher on English long-text rendering. Published comparison methods use different training settings and data.^[4]

Controlled ablations use the same full-parameter Z-Image setup at two evaluations. Adding EMA moment matching raises GenEval from 77.46% to 77.81%, LongText-CN from 93.87% to 94.25%, and LongText-EN from 89.79% to 90.20%. A DINOv2 feature control reaches 77.75%, 94.13% and 90.09%, respectively.^[1]

Inference speed

The published timing protocol uses one A100 GPU in BF16: batch size one for latency and ten for throughput. LoRA adapters are merged before timing. These are paper measurements, not measurements produced by running the released repository.^[4]

In the official inference interface, NFE counts conditional transformer evaluations without classifier-free guidance.^[4]

Code, checkpoints and community implementations

As of November 4, 2026, the official repository provides Apache-2.0 inference code, while its official full-model and LoRA checkpoint entries remain marked as forthcoming. Base-model or APEX weights are not IDM checkpoints and cannot establish reproduction of IDM's results.^[4]

Separately, akhaliq/MOSS-Image-IDM-2-Steps provides downloadable community adapters with Apache-2.0 license metadata. Its model card describes a different implementation using a data-space generator, rank-128 LoRA and a separately trained fake-score network. It reports no LongText benchmark score. These adapters are distinct from the official Z-Image and Qwen experiments.^[5]

The community card recommends export 750 of its first version for text-to-image generation. A later editing continuation degraded general text-to-image quality; the card also reports failures on structural editing tasks. Its visual examples do not independently reproduce the paper's benchmark results.^[5]

Continue reading

• LLoRA (Low-Rank Adaptation)Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.Mentioned 4 times above
• Qwen-ImageQwen-Image is an open-weight image-generation foundation model released by Alibaba's Qwen team in August 2025.Mentioned 3 times above
• InclusionAIinclusionAI is an open-source artificial general intelligence (AGI) research initiative established by Ant Group, the financial-technology affiliate of the Alibaba ecosystem.Linked in the introduction
• Stable DiffusionStable Diffusion is a family of generative image models that can synthesize and edit images from text and other conditions.Also in Diffusion Models
• DALL-EDALL-E is a family of text-to-image systems developed by OpenAI that generates images from natural-language descriptions.Also in Diffusion Models
• MidjourneyMidjourney is an artificial intelligence image generation service and independent research lab headquartered in San Francisco, CaliforniaAlso in Diffusion Models

More to read

• Diffusion Transformer (DiT)
• Black Forest Labs
• Imagen (text-to-image model)
• Flux (text-to-image model)
• FLUX.1
• Stable Diffusion 3

Topics:AI ResearchDiffusion ModelsImage GenerationTraining & Optimization

References

1. ^1 ^2 ^3Deyuan Liu et al. Intrinsic Distribution Matching. arXiv:2610.39633, submitted October 29, 2026.
2. ^Tianwei Yin et al. One-step Diffusion with Distribution Matching Distillation. CVPR 2024.
3. ^1 ^2 ^3InclusionAI. Intrinsic Distribution Matching: official project explanation.
4. ^1 ^2 ^3 ^4 ^5InclusionAI. IDM: official code, results and checkpoint availability.
5. ^1 ^2akhaliq. MOSS-Image-IDM-2-Steps model card.