eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation
Organizations: Zhejiang University · AGI Lab, Westlake University · StepFun
Abstract
Distribution matching distillation (DMD) has emerged as a leading paradigm for distilling diffusion models into few-step students, where an auxiliary fake score model tracks the student's distribution and the discrepancy between the teacher and fake predictions drives the student toward the teacher. In multi-step distillation, although only the student's final output is delivered, existing methods train the fake model on intermediate states or across all denoising steps. Consequently, the student's update signal becomes entangled with intermediate distributions, which can distort the matching direction and remain nonzero even after the student's final outputs already match the teacher distribution. To resolve this mismatch, we propose Endpoint Distribution Matching Distillation (eDMD), which trains a single shared fake model exclusively on the student's final outputs and uses it to correct every denoising step. Our theoretical analysis proves that this endpoint reference guarantees that the distribution matching term vanishes if and only if the student's final output distribution matches the teacher's, regardless of intermediate distributions. Beyond the fake reference, we further strengthen teacher supervision through a precise teacher (PT) that replaces single-step guided predictions with short multi-step trajectories on the final noise interval, paired with a calibrated query density for intermediate steps. Experiments on SD3-Medium, FLUX.2-klein (4B and 9B), and Qwen-Image-2.1 demonstrate that eDMD consistently outperforms existing distribution matching methods for 4-step generation.
Figures & tables
| Scores at the peak checkpoint | Fixed steps | |||||||||
| Method | HPSv3++ | HPSv3 | PickScore | Aesthetic | HPSv2.1 | ImageReward | CLIP | DPG | @1k | @2k |
| SD3-Medium, teacher default | ||||||||||
| Teacher (28 steps) | 8.554 | 9.257 | 22.429 | 5.350 | 0.2849 | 0.964 | 0.3263 | 85.93 | – | – |
| DMD2 [ 61 ] | 10.270 | 10.129 | 22.455 | 5.547 | 0.3052 | 1.136 | 0.3210 | 84.77 | 10.209 | 9.354 |
| TDM ∗ [ 34 ] | 10.386 | 10.418 | 22.495 | 5.499 | 0.2996 | 1.016 | 0.3183 | 83.59 | – | – |
| Method | HPS v3++ | HPS v3 | Pick | Aes. | HPS v2.1 | IR | CLIP |
|---|---|---|---|---|---|---|---|
| Teacher (40 steps) | 13.767 | 12.493 | 23.097 | 5.671 | 0.2974 | 1.057 | 0.3144 |
| Turbo ∗ (8 steps) | 13.850 | 12.171 | 23.023 | 5.675 | 0.2985 | 1.064 | 0.3164 |
| DMD2 | 14.212 | 13.073 | 23.090 | 5.781 | 0.3032 | 1.049 | 0.3126 |
| TDM | 13.787 | 12.804 | 23.101 | 5.743 | 0.3003 | 1.072 | 0.3129 |
| eDMD | 14.691 | 13.186 | 23.175 | 5.788 | 0.3109 | 1.104 | 0.3138 |
| Variant | HPS v3++ | HPS v3 | Pick | Aes. | HPS v2.1 | IR |
|---|---|---|---|---|---|---|
| SD3-Medium | ||||||
| Full | 11.171 | 11.336 | 22.589 | 5.547 | 0.3097 | 1.115 |
| w/o E | 10.122 | 11.166 | 22.480 | 5.500 | 0.3051 | 1.056 |
| w/o PT | 11.026 | 11.092 | 22.547 | 5.497 | 0.3096 | 1.115 |
| w/o Q | 10.798 | 10.706 | 22.562 | 5.490 | 0.3063 | 1.110 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Fake | Training target | Optimal prediction | Extra condition |
|---|---|---|---|
| Stage law ( Prop. 2 ) | score of | , | |
| TDM, exact weight | from re-noised state, weight | for every | |
| TDM, released | from re-noised state, weight | not met along the teacher’s flow | |
| Endpoint | final output | none |
| Item | Value |
|---|---|
| Student | LoRA [ 11 ] on the teacher, rank 32, |
| Fake | as the student; FLUX.2-klein-9B: all parameters |
| Student optimizer | AdamW [ 28 ] , lr (FLUX.2-klein-9B: ), , weight decay 0.01, clip 3 |
| Student schedule | 100 warm-up updates, then constant; SD3-Medium and FLUX.2-klein-9B: half cosine to 0 from 1,200 |
| Fake optimizer | AdamW, lr (FLUX.2-klein-9B: ), , weight decay 0.01, clip 1, constant |
| Fake updates | 5 per student update, each on fresh prompts and a fresh rollout; the student reuses the last rollout |
| Mean level | ||||||
| Teacher | uniform | Support | ||||
| SD3-Medium | 0.900 | 18.8 | 13.6 | 0.913 | 0.945 | 0.39 |
| 0.750 | 33.3 | 26.9 | 0.793 | 0.870 | 0.46 | |
| 0.500 | 64.7 | 52.7 | 0.608 | 0.745 | 0.55 | |
| FLUX.2-klein-4B | 0.967 | 1.57 | 0.36 | 0.978 | 0.979 | 0.99 |
| Implementation | HPSv3++ | HPSv3 | Pick | Aes. | HPSv2.1 | CLIP |
|---|---|---|---|---|---|---|
| SD3-Medium, trained locally | ||||||
| TDM (official recipe) | 9.634 | 10.473 | 22.461 | 5.441 | 0.3002 | 0.3212 |
| TDM ( re-impl.) | 9.993 | 10.510 | 22.397 | 5.527 | 0.3029 | – |
| CDM | 10.586 | 11.787 | 22.554 | 5.469 | 0.3012 | 0.3210 |
| FLUX.2-klein-4B, trained locally | ||||||
| TDM (official recipe) | 9.333 | 9.098 | 22.224 | 5.446 | 0.2783 | 0.3145 |
| Method | Step | HPS v3++ | HPS v3 | Pick | Aes. | HPS v2.1 | IR | CLIP |
|---|---|---|---|---|---|---|---|---|
| DMD2 | 100R | 14.212 | 13.073 | 23.090 | 5.781 | 0.3032 | 1.049 | 0.3126 |
| DMD2 + PAG | 500E | 14.094 | 13.104 | 23.012 | 5.859 | 0.3061 | 1.047 | 0.3145 |
| TDM | 500R | 13.787 | 12.804 | 23.101 | 5.743 | 0.3003 | 1.072 | 0.3129 |
| TDM + PAG | 600R | 13.674 | 13.189 | 23.201 | 5.846 | 0.3003 | 1.048 | 0.3122 |
| eDMD | 400E | 14.691 | 13.186 | 23.175 | 5.788 | 0.3109 | 1.104 | 0.3138 |
| Method | SD3 | FLUX 4B | FLUX 9B | Qwen |
|---|---|---|---|---|
| DMD2 | 1100E | 1600R | 700R | 100R |
| TDM | official | 1600R | 900R | 500R |
| D-DMD | 1800R | 1400E | 1400R | – |
| CDM | official | 600R | 1300R | – |
| eDMD | 1000R | 2000R | 1400R | 400E |
| SD3-Medium | FLUX.2-klein-4B | FLUX.2-klein-9B | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Run | Peak | @1k | @2k | Peak | @1k | @2k | Peak | @1k | @2k |
| TDM re-impl. | 9.99 | 9.74 | 9.97 | 10.26 | 10.08 | 10.25 | 11.08 | 11.06 | 11.00 |
| TDM re-impl. + E | 10.60 | 10.46 | 10.59 | 11.70 | 11.50 | 11.58 | 11.62 | 11.27 | 11.59 |
| eDMD | 11.17 | 11.17 | 10.82 | 12.70 | 12.07 | 12.70 | 12.43 | 11.89 | 12.28 |
Diffusion Model Distillation
Diffusion model distillation transfers a large model’s generative behavior to faster or smaller models that need fewer sampling steps.
See all 211 papers