cs.AINov 9, 2026

Disagreement-Aware Distribution Matching Distillation for Few-Step Diffusion Models

Authors: Zikai Zhou, Zhengpeng Xie, Shitong Shao, Lichen Bai, Tianhao Zhang, Bojun Cheng, Zeke Xie

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Catnip AI · The Hong Kong University of Science and Technology

Abstract

Distribution Matching Distillation~(DMD) enables strong few-step diffusion generation by matching the student distribution to a pretrained teacher. Existing improvements to DMD often augment its supervision or modify its optimization machinery. We ask a complementary question: can DMD diagnose its own updates using signals already produced during training? DMD inherently computes the discrepancy between a pretrained teacher and a fake model tracking the student distribution, but conventionally uses it primarily to determine the distribution matching direction. We find that the magnitude of this teacher--fake disagreement provides additional diagnostic information. Across extensive experiments, large disagreement in the high-noise regime consistently coincides with substantially higher-energy DMD updates. This reveals a dual role of disagreement: it determines not only where the student should move, but also provides an intrinsic signal of how aggressively the update should be applied. Based on this observation, we introduce Disagreement-Aware Distribution Matching Distillation (DA-DMD), which activates only in the high-noise regime and continuously modulates the update strength according to teacher--fake disagreement. DA-DMD reuses quantities already available during DMD training and requires no additional model, evaluator, or inference-time modification. We show that it exactly recovers vanilla DMD outside the selected region, reduces update energy only through selected samples, and exhibits redescending influence under extreme disagreement. Across multiple diffusion backbones, DA-DMD consistently improves few-step generation at the same inference cost. On SD3.5-L, DA-DMD improves HPSv2 from 28.2328.23 to 30.9330.93, while achieving a 64.4%64.4\% average win rate over DMD2.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

    Nov 12, 2026Tong Zhao, Xianfang Zeng, Yucheng Han +4Diffusion Model DistillationFew-Step Diffusion Sampling

  2. DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

    Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model DistillationFew-Step Diffusion Sampling