cs.CVOct 30, 2026

A Better Critic Matters for Distribution Matching Distillation

Authors: Zongze Wu, Xianbang Wang, Zhengyang Geng, Mingdeng Cao, Zhoutong Zhang, Richard Zhang, Eli Shechtman, Ruiming Cao

Organizations: Adobe · MIT

Abstract

Distribution Matching Distillation (DMD) trains a few-step student using the difference between a teacher's score and an online critic's estimate of the student distribution's score. We investigate what the critic needs, in capacity and in information, to provide a reliable training signal. On the capacity side, controlled experiments across seven SiT model sizes on ImageNet show that increasing teacher size while keeping a small critic fixed can degrade student quality and cause training collapse. Scaling the critic alongside the teacher, a strategy we call joint scaling, prevents collapse in our experiments and allows the student to benefit from larger teachers. On the information side, rather than forcing the critic to model a step-agnostic mixture of student outputs, we introduce a step-aware critic (STAC) that conditions on the student's generation step to estimate step-specific scores. Together, joint scaling and STAC consistently improve few-step distillation across class-conditional ImageNet generation, text-to-image generation with SDXL and FLUX.2, and text-to-video generation with Wan2.1.

Figures & tables

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

    Nov 12, 2026Tong Zhao, Xianfang Zeng, Yucheng Han +4Diffusion Model DistillationFew-Step Diffusion Sampling

  2. DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

    Oct 1, 2026Zhengming Yu, Junkun Yuan, Haotian Yang +8Diffusion Model DistillationFew-Step Diffusion Sampling

  3. Disagreement-Aware Distribution Matching Distillation for Few-Step Diffusion Models

    Nov 9, 2026Zikai Zhou, Zhengpeng Xie, Shitong Shao +4Diffusion Model DistillationDistribution Matching Distillation