ROAD: Modern Adversarial Training for Simple One-Step Distillation at Scale

Xiaoou Li; Shan You; Tao Huang

cs.CVOct 29, 2026

ROAD: Modern Adversarial Training for Simple One-Step Distillation at Scale

Authors: Xiaoou Li, Shan You, Tao Huang

Organizations: Shanghai Jiao Tong University (sjtu.edu.cn) · SenseTime (sensetime.com)

Abstract

One-step distillation turns pretrained diffusion models into fast generators, but strong results often require substantial training machinery and compute. Pure adversarial distillation offers a simpler route, yet has struggled at modern text-to-image scale. To address this limitation, we propose ROAD (Representation-grounded One-step Adversarial Distillation), a one-stage framework that distills text-to-image models with one adversarial objective and a lightweight representation-space critic. First, we replace the classical convolutional head with a Lipschitz-controlled Transformer head with token-level text conditioning, which matches the semantic tokens of modern visual encoders. Second, we observed a fixed random feature mixing leads to long-horizon degradation. We attribute it to generator-critic co-adaptation around one realization-specific view, and periodically re-draw the mixing to keep the randomization effective. Third, for our dense and vector-valued logits, we introduce Jacobian R1 which penalizes the full input-logit Jacobian, overcoming pooled R1's limitation of neglecting most of logit sensitivities. A single adversarial objective then suffices: in one step, ROAD reaches 11.03 FID on SDXL and 13.73 on SD3.5-M on COCO-10K, surpassing the multi-step teachers at a fraction of distribution matching's training compute.

Figures & tables

Figure 2: Overall architecture and critic head’s ViT block of ROAD. Left: generated and real images are encoded into multi-level semantic features, transformed by periodically re-drawn random mixings $M$ , and scored by Transformer critic heads. Right: a Lipschitz-controlled ViT block uses spectral-normalized scaled-cosine self-attention, token-level cross-attention, and an FFN, together with RMSNorm and learnable residual scales.

	Critic head	Param.	FID $\downarrow$	CLIP $\uparrow$
1	Conv	24.8M	19.02	0.317
2	Deep Conv	54.8M	21.52	0.311
3	Naive Transformer	54.4M	Collapse	–
4	ViTGAN	54.3M	15.40	0.319
5	ViT (ours)	54.3M	14.54	0.331

Table 1: Critic head design and text conditioning. Fixed mixing and pooled R1 are used with 2B generator (SDXL). Left: Conv follows the convolutional head of Projected GAN. Raising its capacity does not improve performance, whereas our ViT head substantially improves both fidelity and alignment; the naive Transformer head collapses. Right: Pooled-text projection provides only a minor gain over the unconditional critic, whereas token-level cross-attention yields a stronger improvement in text–image alignment.

Figure 3: Diagnosing co-adaptation under fixed random mixing. (a) FID oscillates and eventually deteriorates. (b) PFUC settles into a persistent per-channel pattern. (c) The change between consecutive PFUC profiles, measured by total variation (TV), decays toward a floor. (d) The logit margin stays high and nearly stationary even as FID degrades. The twin run refreshed every $5$ k steps is faded for reference. Sec. B.3 describes the measurements in detail.

Figure 4: Periodic mixing refresh breaks the persistent critic state. The refreshed run (blue, every $5$ k steps) is foregrounded, with fixed mixing faded for reference. (a) FID improves after re-draws and avoids the late deterioration of fixed mixing. (b) PFUC reorganizes under each new view. (c) The PFUC change spikes after each re-draw and then decays; filled dots mark the value at each re-draw. (d) The logit margin resets toward zero at each re-draw and re-forms as the heads adapt.

Method	Training		FID $\downarrow$	CLIP $\uparrow$	HPSv2 $\uparrow$	Pick $\uparrow$	Aes $\uparrow$
Method	Tech.	Stage	FID $\downarrow$	CLIP $\uparrow$	HPSv2 $\uparrow$	Pick $\uparrow$	Aes $\uparrow$
SDXL base (cfg 6)	–	–	19.29	0.331	28.33	22.11	6.19
Hyper-SDXL	Traj.+Reward+ ScoreD+GAN	6	33.66	0.328	29.97	22.71	6.59
DMD2	ScoreD+GAN	2	19.01	0.336	28.36	22.09	6.09
DMDX	Adv. ScoreD	2	–	0.352	27.70	22.27	–
SiDa	ScoreD+GAN	1	15.23	0.326	27.80	21.72	6.16

Table 2: Evaluation of one-step SDXL. All results are reproduced under our protocol with official checkpoints, except DMDX, whose values are from its paper. Unlike prior methods that require multiple objectives and multi-stage training (Sec. D ), ROAD uses a single stage with a pure GAN loss.

Method	Tech.	NFE	FID $\downarrow$	CLIP $\uparrow$	HPSv2 $\uparrow$	IR $\uparrow$	Pick $\uparrow$	Aes $\uparrow$	GE $\uparrow$
SD3.5-M (cfg 4.5)	–	40	18.01	0.341	29.19	1.00	22.38	6.12	0.69
SD3.5-M-Turbo	–	4	23.22	0.330	26.80	0.47	21.59	5.86	0.55
SenseFlow	ScoreD+GAN+Traj.	4	28.07	0.328	29.54	0.93	22.59	6.28	0.62
SwD	ScoreD+GAN+MMD	4	26.56	0.338	29.48	1.15	22.91	6.39	0.71
SiD-DiT	ScoreD	4	22.02	0.338	29.36	1.01	22.59	6.24	0.64
ROAD (ours)	GAN	1	13.73	0.342	28.85	0.94	22.00	6.15	0.66

Table 3: Evaluation of few-step SD3.5-M. All results are reproduced under our protocol with official 4-step checkpoints; there are no 1-step methods yet, to the best of our knowledge.

Panel of Figure 5 — Figure 5: Analysis of key components of ROAD.

Figure 9: Augmentation under fixed mixing. Representative discriminator anti-overfitting interventions do not materially alleviate late-stage FID degradation.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	SDXL	SD3.5-M
Teacher model	SDXL-Base-1.0	SD3.5-Medium
Teacher arch.	UNet	MMDiT
Conditioning timestep / $\sigma$	$t{=}999$	$\sigma{=}1.0$
GPUs $\times$ batch	$8{\times}8=64$	$8{\times}4=32$
G learning rate	$5{\times}10^{-7}$	$5{\times}10^{-7}$
D learning rate	$5{\times}10^{-6}$	$5{\times}10^{-6}$

Appendix

Table 6: Training configurations of SDXL and SD3.5-M.

Figure 11: ROAD samples on SD3.5-M. Qualitative generations from the distilled model.

Diffusion Model Distillation

Diffusion model distillation transfers a large model’s generative behavior to faster or smaller models that need fewer sampling steps.

See all 194 papers

One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration
May 20, 2026Chaoyang Wang, Yunhai TongDiffusion Model Distillation Few-Step Diffusion Sampling
Peking University
Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations
May 8, 2026Yuan Zhang, Chenyi Li, Guoqing Ma +7Diffusion Model Distillation Diffusion Sampling
JD Explore Academy, China · The Hong Kong University of Science and Technology · Tsinghua University +1
Data-free Multistep Diffusion Distillation
Oct 13, 2026Ngoc Duy Tran, Kien Do, Duc Thanh Nguyen +1Diffusion Model Distillation Diffusion Distillation
Deakin Applied Artificial Intelligence Initiative, Deakin University, Australia · VinSmart Future, Vietnam · School of Information Technology, Deakin University, Australia

May 20, 2026J/K move · Enter open · S save

One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration

Chaoyang Wang, Yunhai Tong

Peking University

Diffusion Model Distillation Few-Step Diffusion Sampling Diffusion-Based Image Generation Diffusion Models

Open paper AskPDF arXiv

Discrete diffusion models excel at visual synthesis but rely on slow, iterative decoding. Existing single-step distillation methods attempt to bypass this bottleneck, either by training auxiliary score networks that effectively double compute, or by introducing specialized parameterizations and multi-stage pipelines that fragment optimization. In this paper, we introduce Fixed-Point Distillation (FPD), an end-to-end framework that constructs local correction targets by partially corrupting the student's one-step draft and refining it with a single teacher step. To compute the training objective in a semantically meaningful space, we lift discrete tokens into continuous features and apply a multi-bandwidth drift loss that iteratively accumulates these corrections. To backpropagate through the discrete bottleneck, we employ a straight-through estimator that feeds exact hard-sampled tokens to the teacher and decoder during the forward pass, ensuring that training and inference operate on the same codebook manifold, while routing continuous gradients back to the student logits. This fully differentiable pathway additionally accommodates an optional unconditional adversarial objective to enhance perceptual realism. Evaluations on both class- and text-conditional generation validate the effectiveness of our framework. FPD achieves competitive visual fidelity and structural alignment within a single inference step, narrowing the gap to multi-step teachers while outperforming existing discrete distillation baselines.

Abstract

Diffusion Model Distillation

Explore similar work

One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration

Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations

Data-free Multistep Diffusion Distillation