Intrinsic Distribution Matching

Deyuan Liu; Yansen Han; Kaiwen Zheng; Jun Xie; Jiacheng Liu; Zhenglin Cheng; Peng Sun; Jintao Zhang

cs.CVOct 29, 2026

Intrinsic Distribution Matching

Authors: Deyuan Liu, Yansen Han, Kaiwen Zheng, Jun Xie, Jiacheng Liu, Zhenglin Cheng, Peng Sun, Jintao Zhang,

Organizations: Inclusion AI · Westlake University · Zhejiang University · Tsinghua University ·

Abstract

One-step and few-step text-to-image generation must accurately render the details specified in the prompt despite limited opportunities for refinement. Distribution matching distillation (DMD) uses score differences to bring the generated distribution closer to a reference distribution. Following prior work, we estimate this difference with internal velocity branches, avoiding a separate fake-score network. We ask whether an explicit moment constraint improves this learned score supervision. A decomposition of the reverse-KL field into a Gaussian moment component and a non-Gaussian residual identifies statistics that can be constrained explicitly. We introduce Intrinsic Distribution Matching (IDM), which retains the full DMD correction and matches feature means and per-coordinate standard deviations using the generator's exponential-moving-average (EMA) copy, requiring no external feature encoder. Experiments use full-parameter and low-rank adaptation (LoRA) tuning on Z-Image-Turbo 6B and Qwen-Image 20B. At two network evaluations, IDM reaches 94.25% and 90.20% on Chinese and English LongText-Bench, the highest scores among the compared methods at this budget. Controlled ablations show gains from the added moment loss, while generator EMA features perform competitively with an external DINOv2 encoder across the five reported metrics. Code is available at https://github.com/inclusionAI/IDM.

Figures & tables

Figure 1: Overview of IDM for one-step generation. Multi-step sampling refines samples iteratively, whereas one-step generation learns a direct noise-to-data map. IDM combines an internal DMD loss with a feature-moment loss in the generator’s EMA feature space. The right panel illustrates idealized alignment with single-evaluation inference.

Figure 2: IDM (Z-Image), 2 NFE. Text rendering and portrait generation.

Figure 3: Reverse-KL Field Decomposition. The full field

\mathbf{g}_{\mathrm{KL}}=\mathbf{s}_{\theta}-\mathbf{s}_{r}

( left ) is the sum of a Gaussian moment component ( middle ) and a non-Gaussian residual component ( right ). Solid contours show

p_{i}

and dashed contours its moment-matched Gaussian

q_{i}

, for

i\in\{r,\theta\}

. Equal means and covariances eliminate the Gaussian component but can leave a nonzero residual. All fields use the same evaluation points and arrow scale.

Figure 4: Intrinsic losses of IDM . Top, DMD loss: re-noise the predicted clean sample and use the internal velocity difference to construct detached velocity and endpoint targets for

\mathcal{L}_{\text{score}}

in Eq. 12 . Bottom, feature-moment loss: match means and per-coordinate standard deviations of EMA features

\mathbf{h}_{i}:=\phi_{\bar{\theta}}(\mathbf{x}_{i},\mathbf{c})

using current model features and detached model/reference queues

\mathcal{Q}_{\theta},\mathcal{Q}_{r}

in Eq. 13 . Base objective omitted.

Methods		NFEs	Throughput	Latency	Params	FID $\downarrow$	CLIP $\uparrow$	GenEval $\uparrow$
Methods		NFEs	(samples/s)	(s)	(B)	FID $\downarrow$	CLIP $\uparrow$	GenEval $\uparrow$
Few-Step Generative Models	SDXL-LCM Luo et al. (2023)	2	2.89	0.40	0.9	18.11	27.51	0.44
	PixArt-LCM Chen et al. (2024c)	2	3.52	0.31	0.6	10.33	27.24	0.42
	SD3.5-Turbo ( Esser et al., 2024 ) ‡	2	1.61	0.68	8.0	51.47	25.59	0.53
	PCM Wang et al. (2024a) †	2	2.62	0.56	0.9	14.70	27.66	0.55
	SDXL-DMD2 Yin et al. (2024a) †	2	2.89	0.40	0.9	7.61	28.87	0.58

Table 1: System-level comparison of efficiency and quality. Speeds are on a single A100 ( BF16 ). Throughput is samples/s (batch=10); latency is seconds (batch=1). GenEval measures compositional prompt following; FID/CLIP complement the efficiency comparison. LongText results appear in Table 5 . † indicates that distinct models are required for different NFEs. ‡ marks a noncanonical operating point or a secondary comparison source. § indicates that the LoRA weights are merged into the base weights before timing. Params: backbone size in billions; LoRA entries additionally give rank

r

and trainable parameters in parentheses. Blue/red backbone counts indicate full/LoRA tuning, respectively.

Model	Single	Two	Counting	Colors	Position	Attribute	Overall $\uparrow$
Model	Object	Object	Counting	Colors	Position	Binding	Overall $\uparrow$
Show-o ( Xie et al., 2024 )	0.95	0.52	0.49	0.82	0.11	0.28	0.53
Emu3-Gen ( Wang et al., 2024b )	0.98	0.71	0.34	0.81	0.17	0.21	0.54
PixArt- $\alpha$ ( Chen et al., 2024d )	0.98	0.50	0.44	0.80	0.08	0.07	0.48
SD3 Medium ( Esser et al., 2024 )	0.98	0.74	0.63	0.67	0.34	0.36	0.62
FLUX.1 [Dev] ( BlackForest, 2024 )	0.98	0.81	0.74	0.79	0.22	0.45	0.66

Table 2: Quantitative evaluation on GenEval. Per-category and overall scores at the NFE shown in each model label; unlabeled published baselines use their reported default sampling budget. Z-Image is the base model (

\sim

100 NFE), whereas Z-Image-Turbo is the 8-NFE distilled model.

Model	Cultural	Time	Space	Biology	Physics	Chemistry	Overall $\uparrow$
SD v1.5 ( Rombach et al., 2022 )	0.34	0.35	0.32	0.28	0.29	0.21	0.32
SDXL ( Podell et al., 2023 )	0.43	0.48	0.47	0.44	0.45	0.27	0.43
SD3.5-Large ( Esser et al., 2024 ) ‡	0.44	0.50	0.58	0.44	0.52	0.31	0.46
PixArt- $\alpha$ ( Chen et al., 2024d )	0.45	0.50	0.48	0.49	0.56	0.34	0.47
Playground-v2.5 ( Li et al., 2024a )	0.49	0.58	0.55	0.43	0.48	0.33	0.49
FLUX.1 [Dev] ( BlackForest, 2024 )	0.48	0.58	0.62	0.42	0.51	0.35	0.50

Table 3: Quantitative evaluation on WISE ( Niu et al., 2025 ) . Per-domain and overall world-knowledge grounding scores.

Model	Global	Entity	Attribute	Relation	Other	Overall $\uparrow$
SD v1.5 ( Rombach et al., 2022 )	74.63	74.23	75.39	73.49	67.81	63.18
PixArt- $\alpha$ ( Chen et al., 2024d )	74.97	79.32	78.60	82.57	76.96	71.11
Lumina-Next ( Zhuo et al., 2024 )	82.82	88.65	86.44	80.53	81.82	74.63
SDXL ( Podell et al., 2023 )	83.27	82.43	80.91	86.76	80.41	74.65
Hunyuan-DiT ( Li et al., 2024b )	84.59	80.59	88.01	74.36	86.41	78.87
Janus ( Wu et al., 2025b )	82.33	87.38	87.70	85.46	86.41	79.68

Table 4: Quantitative evaluation on DPG-Bench. NFE notation follows Table 2 .

Idx	Model	NFE	OneIG	GenEval	DPG-Bench	LongText-CN	LongText-EN
Distribution Matching Loss Ablation
\raisebox{-0.05ex}{\tiny2}⃝	00 w/o $\mathcal{L}_{\text{score}}$ (DMD loss)	2	52.58	76.90	86.15	93.33	89.62
\raisebox{-0.05ex}{\tiny3}⃝	00 w/o $\mathcal{L}_{\text{pop}}$ (feature-moment loss)	2	52.87	77.46	86.54	93.87	89.79
\raisebox{-0.05ex}{\tiny4}⃝	00 w/o $\mathcal{L}_{\text{score}}$ and $\mathcal{L}_{\text{pop}}$	2	52.25	76.24	85.68	92.70	88.88
DMD Regression Ablation
\raisebox{-0.05ex}{\tiny5}⃝	00 $v$ -loss only	2	52.43	76.64	85.93	93.03	89.17

Table 5: Overall ablation results across benchmarks (2 NFE). Scores are percentages. Row \raisebox{-0.05ex}{\tiny1}⃝ is the full configuration and the shared control; rows \raisebox{-0.05ex}{\tiny2}⃝ – \raisebox{-0.05ex}{\tiny11}⃝ vary the losses, DMD regression weights, or feature map. The

v

-only and

x

-only controls retain

\mathcal{L}_{\text{pop}}

. APEX ( Liu et al., 2026a ) is the published Z-Image reference from its comparison with TwinFlow; rows \raisebox{-0.05ex}{\tiny12}⃝ – \raisebox{-0.05ex}{\tiny16}⃝ are from Liu et al. (2026b) , with row \raisebox{-0.05ex}{\tiny16}⃝ evaluated at 8 NFE.

Figure 5: Reference methods and IDM loss configurations. The two prompts are evaluated at 1 and 2 NFE. The $v{+}x$ control omits $\mathcal{L}_{\text{pop}}$ , while full IDM includes EMA feature-moment matching. APEX is an external reference trained on a smaller collection; the IDM loss ablations share the same training setting.

Idx	Model	NFE	Overall	Two obj.	Color attr.	Position	Counting	Colors	Single obj.
Distribution Matching Loss Ablation
\raisebox{-0.05ex}{\tiny2}⃝	00 w/o $\mathcal{L}_{\text{score}}$ (DMD loss)	2	76.90	90.40	64.50	49.25	70.31	87.23	99.69
\raisebox{-0.05ex}{\tiny3}⃝	00 w/o $\mathcal{L}_{\text{pop}}$ (feature-moment loss)	2	77.46	90.91	65.25	50.75	70.94	87.23	99.69
\raisebox{-0.05ex}{\tiny4}⃝	00 w/o $\mathcal{L}_{\text{score}}$ and $\mathcal{L}_{\text{pop}}$	2	76.24	90.15	63.75	49.25	68.44	86.17	99.69
DMD Regression Ablation
\raisebox{-0.05ex}{\tiny5}⃝	00 $v$ -loss only	2	76.64	90.40	64.00	50.00	69.06	86.70	99.69

Table 6: Detailed GenEval ablation results. Scores are percentages; Overall is the mean of the six category scores. Configurations, row groups, and sources follow Table 5 .

Idx	Model	NFE	Overall	Attribute	Global	Relation	Other	Entity
Distribution Matching Loss Ablation
\raisebox{-0.05ex}{\tiny2}⃝	00 w/o $\mathcal{L}_{\text{score}}$ (DMD loss)	2	86.15	89.33	80.23	94.32	89.54	92.58
\raisebox{-0.05ex}{\tiny3}⃝	00 w/o $\mathcal{L}_{\text{pop}}$ (feature-moment loss)	2	86.54	89.54	79.94	94.48	89.41	92.88
\raisebox{-0.05ex}{\tiny4}⃝	00 w/o $\mathcal{L}_{\text{score}}$ and $\mathcal{L}_{\text{pop}}$	2	85.68	89.07	80.01	94.12	89.37	92.23
DMD Regression Ablation
\raisebox{-0.05ex}{\tiny5}⃝	00 $v$ -loss only	2	85.93	89.21	79.45	94.22	89.05	92.42

Table 7: Detailed DPG-Bench ablation results. Scores are percentages. Overall follows the standard prompt-level aggregation over the benchmark question hierarchy, rather than the mean of the five category columns. Configurations, row groups, and sources follow Table 5 .

Idx	Model	NFE	Overall	Alignment	Text	Diversity	Style	Reasoning
Distribution Matching Loss Ablation
\raisebox{-0.05ex}{\tiny2}⃝	00 w/o $\mathcal{L}_{\text{score}}$ (DMD loss)	2	52.58	86.13	97.09	12.74	38.04	28.90
\raisebox{-0.05ex}{\tiny3}⃝	00 w/o $\mathcal{L}_{\text{pop}}$ (feature-moment loss)	2	52.87	86.31	97.73	12.81	38.16	29.34
\raisebox{-0.05ex}{\tiny4}⃝	00 w/o $\mathcal{L}_{\text{score}}$ and $\mathcal{L}_{\text{pop}}$	2	52.25	85.94	96.37	12.67	37.90	28.37
DMD Regression Ablation
\raisebox{-0.05ex}{\tiny5}⃝	00 $v$ -loss only	2	52.43	86.04	96.76	12.71	37.97	28.67

Table 8: Detailed OneIG ablation results. Scores are percentages; Overall is the mean of the five category scores. Configurations, row groups, and sources follow Table 5 .

Figure 6: Sampling-budget comparison on Z-Image. Final outputs from Z-Image-Turbo, DMD2, TwinFlow, APEX, and IDM at the indicated NFE budgets, grouped by two prompts. Each column represents a sampling budget, not an intermediate state of one trajectory.

Panel of Figure 7 — Figure 7: IDM (Z-Image), 2 NFE. Additional samples across styles and content. Every panel uses the same model and sampling budget; the grid is a gallery of outputs rather than a method or NFE comparison.

T2I Generation

Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.

See all 303 papers

DMA $^2$ : Pixel-space Distribution Matching with Adversarial and Anchor Losses
Sep 29, 2026Xin Lin, Zhifei Zhang, Yuqian Zhou +6Diffusion Model Distillation Pixel-Space Diffusion Models
UC San Diego · Adobe Research · University of Virginia +1
Unifying Distributional Training for One-Step Visual Generation
Sep 28, 2026Chi Zhang, Shi Haoyang, Yueyi Liu +9Gaussian Mixture Models T2I Generation
Tsinghua University · Fudan University · Xi’an Jiaotong University +4
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model Distillation Few-Step Diffusion Sampling
Texas A&M University · Intelligent Creation, ByteDance

Sep 29, 2026J/K move · Enter open · S save

DMA $^2$ : Pixel-space Distribution Matching with Adversarial and Anchor Losses

Xin Lin, Zhifei Zhang, Yuqian Zhou, Haitian Zheng+5

UC San Diego · Adobe Research · University of Virginia+1

Diffusion Model Distillation Pixel-Space Diffusion Models Few-Step Diffusion Sampling Distribution Matching Distillation T2I Diffusion Models

Open paper AskPDF arXiv

Distribution matching distillation (DMD) provides a general framework for few-step diffusion generation, but its modern text-to-image instantiations have been developed primarily around latent diffusion. It therefore overlooks key properties and design opportunities of native RGB. We revisit two DMD interfaces for pixel-space teachers. On the teacher-matching side, diagnostics show low-noise RGB matching is dominated by a local-texture cue, motivating a fixed high-noise matching band. On the real-data side, native clean-RGB outputs allow guidance from an external visual representation without traversing a decoder or sharing the heavy fake-score critic. DINO-Adv removes this critic from the adversarial gradient path and supplies local parametric patch guidance. For distribution-level guidance, we introduce AF-Loss, a parameter-free auxiliary semantic distribution-field objective designed for text-to-image DMD. It operates on detached rolling real and generated supports in the shared DINOv2 space while preserving prompt-conditioned teacher supervision. AF-Loss adds no learnable parameters or inference-time computation. Together these designs form DMA

^2

. Across DPG-Bench, GenEval, VQAScore, and COCO30K, the four-step DMA

^2

student performs better than the 25-step teacher and evaluated few-step distillers.

Abstract

T2I Generation

Explore similar work

DMA2^22: Pixel-space Distribution Matching with Adversarial and Anchor Losses

Unifying Distributional Training for One-Step Visual Generation

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

DMA $^2$ : Pixel-space Distribution Matching with Adversarial and Anchor Losses