Distribution-matching distillation enables high-quality image generation in only a few denoising steps. However, distilled student models often map different initial noise samples to similar outputs under the same prompt. To locate where diversity is lost, we compare the teacher and student along the sampling trajectory. The largest measured reduction in the response to different initial noises occurs during the first, high-noise step. When we replace the first step with the teacher's corresponding trajectory and leave the final three student steps unchanged, most of the output diversity returns. This suggests that the first step is a key point for restoring diversity. Motivated by this finding, we introduce ReSeed, a post-distillation method that adds a seed-response bias adapter only to the first student step and aligns its velocity with the teacher's first-interval secant velocity. All student weights and later denoising steps remain unchanged. We evaluate ReSeed on three text-to-image models (SD3-Medium, SD3.5-Medium, and FLUX.1-dev) and one text-to-video model (Wan2.1-T2V-1.3B). For a distilled 4-step FLUX.1-dev model, ReSeed requires only 0.00196% additional trainable parameters relative to the base model and is trained on 8,000 prompt--noise pairs, while keeping the distilled student fully frozen and requiring no target images, perceptual losses, or adversarial losses. It improves DINO diversity by 48.3% while maintaining competitive visual quality. These results show that substantial diversity can be recovered through a lightweight, data-efficient tuning of only the first denoising step, without finetuning the full student model.
Figures & tables
Figure 1: ReSeed recovers visible same-prompt diversity. All panels use the prompt “A smiling woman with long dark brown hair, hazel eyes, and a warm expression” and the same 25 fixed initial noises, matched cell by cell. (a) The 40-step SD3.5-Medium teacher at CFG 4.5 . (b) The frozen 4-step DMD student, and (c) The same frozen 4-step DMD student with ReSeed.
Figure 2: First-step replacement on SD3-Medium. (a) With student steps 2–4 fixed, replacing step 1 by the teacher raises diversity for 98/100 prompts; ReSeed moves them toward the teacher-prefix result. (b,c) Replacing the first k steps shows that k=1 gives 91% of the diversity gain. CLIPScore changes little until later steps are replaced. Bands show the standard deviation over three seeds.
Figure 3: Overview of ReSeed . (a) First-step contraction can merge seed trajectories; the teacher and ReSeed retain alternative paths. (b) Training fits first-step block-output biases to the teacher’s interval-average velocity with frozen base weights. (c) The bias is off in steps 2–4.
Figure 4: Seed-response error versus DINOv3 diversity on COCO5K ( 5,000 prompts × 10 seeds). The error axis is reversed; black curve is checkpoint means.
Method
NFEs
Diversity
Quality
DINO ↑
CLIP dist. ↑
LPIPS ↑
CLIP ↑
Pick ↑
HPS ↑
AeS ↑
FID ↓
SD3-Medium (teacher CFG =7 )
Base Model
56
0.269
0.148
0.696
0.3534
22.516
0.2895
5.33
32.15
DMD
4
0.214±0.001
0.134±0.0003
0.655±0.006
0.3442±0.0011
22.654±0.015
0.3094±0.0007
5.61±0.01
37.38±0.40
DP-DMD
4
0.282±0.007
0.167±0.002
0.666±0.016
0.3276±0.0004
21.983±0.096
0.2805±0.0078
5.20±0.09
43.29±2.12
GAD
4
0.234±0.002
0.132±0.002
0.668±0.017
0.3398±0.0020
22.587±0.057
0.3046±0.0003
5.53±0.01
35.82±0.39
Table 1: Results on SD3-Medium and SD3.5-Medium under a common evaluation protocol.
Model
NFEs
Diversity
Quality
DINO ↑
CLIP dist. ↑
LPIPS ↑
CLIP score ↑
Pick ↑
HPS ↑
AeS ↑
FID ↓
FLUX.1-dev
28
0.242
0.140
0.649
0.3472
23.050
0.3033
5.70
33.49
SenseFlow
4
0.158
0.090
0.533
0.3431
23.152
0.3128
5.87
40.22
SenseFlow + ReSeed (ours)
4
0.234±0.001
0.126±0.002
0.618±0.003
0.3450±0.0003
23.121±0.011
0.3118±0.0003
5.85±0.01
34.65±0.15
Table 2: FLUX quality–diversity comparison.
Figure 6: 4-step FLUX.1-dev student during bias adapter training.
Method
Steps
Video quality (VBench)
Text align. ↑
Video diversity
Subject ↑
Backg. ↑
Flicker ↑
Smooth. ↑
Dynamic ↑
AeS ↑
Imaging ↑
CLIP mean ↑
CLIP frame ↑
DINO mean ↑
DINO frame ↑
Teacher
50
0.970
0.961
0.983
0.991
0.371
0.590
0.703
0.2425
0.1634
0.1899
0.5005
0.5352
DMD2
4
0.970
0.962
0.983
0.991
0.531
0.615
0.698
0.2419
0.1267
0.1530
0.3374
0.3748
DFD
4
0.967
0.955
0.984
0.991
0.529
0.611
0.711
0.2398
0.1351
0.1659
0.3358
0.3775
DMD2 + ReSeed (ours)
4
0.967
0.961
0.986
0.992
0.329
0.631
0.675
0.2365
0.1437
0.1733
0.4290
0.4716
Table 3: Wan2.1-T2V-1.3B on 320 VBench prompts. Bold marks the best 4-step result
Adapter
DINO diversity
Peak act.
Latency (ms)
Student model
Params
of denoiser
Before → after
Gain
change
Step 1
Per sample
Overhead
SD3-Medium
72,192
0.00356%
0.2142→0.2936
+37.1%
+0.00 MiB
37.10→37.68
148.4→149.0
+0.410%
SD3.5-Medium
72,192
0.00322%
0.2356→0.3282
+39.3%
+0.00 MiB
55.69→56.25
222.8→223.2
+0.194%
FLUX.1-dev
233,472
0.00196%
0.1575→0.2336
+48.3%
+1.00 MiB
253.4→255.9
1013.6→1016.2
+0.250%
Wan2.1-T2V-1.3B
46,080
0.00325%
0.3374→0.4290
+27.1%
+0.00 MiB
992.7→996.1
3978.9→3984.3
+0.137%
Table 4: ReSeed training and inference overhead.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Source type
Repository or checkpoint
SD3-Medium teacher
Public
stabilityai/stable-diffusion-3- medium-diffusers
SD3.5-Medium teacher
Public
stabilityai/ stable-diffusion-3.5-medium
FLUX.1-dev teacher
Public
black-forest-labs/FLUX.1-dev
SenseFlow-FLUX student
Public
domiso/SenseFlow , SenseFlow-FLUX/ transformer [ Ge et al., 2026 ]
Wan2.1-T2V-1.3B teacher
Public
Wan-AI/ Wan2.1-T2V-1.3B-Diffusers
Wan2.1 DMD2 student
Public
csusupergear/ wan_t2v_checkpoints_release , dmd2_t2v_vipe/0001000 [ Chen et al., 2026 ]
Appendix
Table 5: Model and checkpoint sources. Repository identifiers link to the public weight releases; locally trained students are listed separately.
Category
Teacher
DMD2
DFD
+ ReSeed
Change [95% CI]
Improved
Animal
0.4235
0.2555
0.2471
0.3313
+0.0758[+0.053,+0.100]
34/40
Architecture
0.5948
0.4391
0.4374
0.5278
+0.0888[+0.056,+0.125]
31/40
Food
0.4371
0.2797
0.2719
0.3904
+0.1107[+0.087,+0.136]
40/40
Human
0.4518
0.2564
0.2635
0.3800
+0.1236[+0.097,+0.152]
37/40
Lifestyle
0.4792
0.2837
0.3117
0.4093
+0.1255[+0.096,+0.155]
38/40
Plant
0.5813
0.4125
0.4146
0.4818
+0.0693[+0.034,+0.104]
31/40
Appendix
Table 6: Video-level DINOv2 diversity by VBench category (40 prompts each). The paired change and the number of improved prompts compare ReSeed with DMD2.
Figure 7: Training trajectories for five teacher guidance scales.
Teacher w
Diversity
Quality
DINO ↑
CLIP dist. ↑
LPIPS ↑
CLIP score ↑
Pick ↑
HPS ↑
AeS ↑
FID-1K ↓
1
0.3132
0.1813
0.7441
0.33832
22.318
0.2998
5.617
71.35
3
0.3060
0.1764
0.7391
0.33952
22.384
0.3012
5.616
71.54
5
0.2990
0.1737
0.7332
0.34072
22.433
0.3019
5.619
72.08
7
0.2951
0.1697
0.7279
0.34049
22.454
0.3026
5.617
72.47
10
0.2873
0.1676
0.7230
0.34094
22.471
0.3027
5.621
72.63
Appendix
Table 7: Teacher guidance for the first-interval regression target, SD3-Medium, read at 2,000 updates. Only the guidance used to build the target changes; the student, its sampler at CFG 1 , the COCO1K prompts, the 100×10 diversity bank, the seed and the block range are shared. Bold marks the best column entry. Diversity falls and alignment generally increases with w , and FID is best at the lowest guidance.
Variant
CLIP score
Pick
HPS
AeS
FID ↓
DINO ↑
CLIPd ↑
LPIPS ↑
4-step student (frozen)
0.34290
22.637
0.3100
5.618
37.471
0.2142
0.1334
0.6609
28-step teacher
0.35343
22.516
0.2895
5.325
32.148
0.2686
0.1478
0.6961
α=−1
0.34346
22.597
0.3118
5.623
38.468
0.1815
0.1230
0.5855
α=0.5
0.34222
22.605
0.3081
5.628
35.598
0.2545
0.1509
0.6979
α=1
0.34051
22.471
0.3041
5.623
33.376
0.2914
0.1698
0.7283
α=1.5
0.33918
22.304
0.2980
5.600
31.476
0.3190
0.1862
0.7535
Appendix
Table 8: COCO5K quality and diversity.
Figure 8: ReSeed training dynamics. Left: training loss for the three image models. Middle and right: bias magnitude and cosine with the final bias during training.
Figure 9: SD3-Medium and SD3.5-Medium over ReSeed training.
Figure 10: Prompt-level trajectories for one FLUX.1-dev adapter on 100 prompts × 10 initial noises. Each point compares one prompt at a training checkpoint (vertical axis) with the original student (horizontal axis); colour denotes the checkpoint and the black line shows the prompt mean. Left: DINOv3 diversity improves for 95 of 100 prompts, from 0.158 to 0.242 on average. Right: CLIP returns to its initial mean ( 0.33748→0.33753 ).
Step
LHS
RHS
LHS/RHS
s
DINOv3 div. ↑
CLIP div. ↑
0
5.70
8.53
0.68
0.39
0.213
0.122
100
3.09
3.66
0.85
0.23
0.253
0.139
200
2.74
3.08
0.90
0.19
0.267
0.145
300
2.60
2.93
0.89
0.20
0.277
0.149
400
2.46
2.73
0.91
0.18
0.279
0.150
700
2.28
2.49
0.92
0.17
0.281
0.152
Appendix
Table 9: Pairwise response error, its bound, and final-image diversity on COCO5K ( 5,000 prompts × 10 seeds) for one SD3-Medium ReSeed run. Step 0 is the DMD 4-step student. LHS and RHS are the empirical pairwise error P and uncorrected bound B , averaged over prompts and reported in units of 10−3 . LHS/RHS and the estimated mean-error share s are medians over prompts.
Figure 11: Empirical analysis of Proposition 1 on COCO5K. Each point is one prompt at one checkpoint, coloured by training step; step 0 is the frozen student. The black curve shows per-step means over prompts. The left-hand side of Equation 7 is plotted against its right-hand side 2(t1−t0)2LReSeed . The diagonal marks P=B ; the exact finite-sample bound is given in Equation 13 . The median LHS/RHS rises from 0.68 to 0.92 during training (Table 9 ).
Figure 12: Matched-noise SD3.5-Medium comparisons. Each method uses the same six seeds. The figure continues on the next page.
Figure 21
Figure 13: Matched-noise SD3-Medium comparisons. Each method uses the same six noises.
Figure 14: Matched-seed FLUX.1-dev comparisons. Each method uses the same nine noises for each prompt.
Figure 16: Matched-noise frame strips for Wan2.1-T2V-1.3B. In each panel, the top, middle, and bottom blocks show the 50-step teacher, the 4-step DMD2 student, and the same student with ReSeed (300 updates); within a block, the eight rows are seeds 0–7 with the same initial noise across blocks, and the twelve columns are twelve evenly spaced frames from 0 to 80. Prompts and video-level DINO diversity (teacher / DMD2 / ReSeed, eight seeds): (a) “A barista pours latte art into a cappuccino, viewed from directly above as a delicate rosetta forms” , 0.122/0.058/0.245 ; (b) “an elegant ceramic plant pot and hanging plant on indoor” , 0.607/0.247/0.588 ; (c) “A young boy lights sparklers in his backyard as twilight settles, his face illuminated in flickering gold” , 0.113/0.082/0.132 ; (d) “scenic view from wooden pathway” , 0.599/0.339/0.475 .
Figure 17: SD3-Medium: a gallery of 4-step ReSeed samples at 1024×1024 .
Figure 18: SD3.5-Medium: a gallery of 4-step ReSeed samples at 1024×1024 .
Figure 19: FLUX.1-dev: a gallery of 4-step ReSeed samples at 1024×1024 .
Diffusion models produce high-quality text-to-image results, but their iterative denoising is computationally expensive.Distribution Matching Distillation (DMD) emerges as a promising path to few-step distillation, but suffers from diversity collapse and fidelity degradation when reduced to two steps or fewer. We present 1.x-Distill, the first fractional-step distillation framework that breaks the integer-step constraint of prior few-step methods and establishes 1.x-step generation as a practical regime for distilled diffusion models.Specifically, we first analyze the overlooked role of teacher CFG in DMD and introduce a simple yet effective modification to suppress mode collapse. Then, to improve performance under extreme steps, we introduce Stagewise Focused Distillation, a two-stage strategy that learns coarse structure through diversity-preserving distribution matching and refines details with inference-consistent adversarial distillation. Furthermore, we design a lightweight compensation module for Distill--Cache co-Training, which naturally incorporates block-level caching into our distillation pipeline.Experiments on SD3-Medium and SD3.5-Large show that 1.x-Distill surpasses prior few-step methods, achieving better quality and diversity at 1.67 and 1.74 effective NFEs, respectively, with up to 33x speedup over original 28x2 NFE sampling.
Haoyu Li, Tingyan Wen, Lin Qi +6
Tsinghua University · Central Media Technology Institute, Huawei
Few-step distillation accelerates diffusion models but must balance diversity and fidelity: trajectory-based distillation preserves mode coverage, while distribution matching sharpens samples but can reduce diversity. We show that this tension can be exploited in a noise-regime-dependent way: high-noise steps largely determine global modes, whereas low-noise steps refine local details. We propose CrossDistill, a trajectory-level hybrid distillation framework that splits the sampling trajectory at a crossover point, applies a trajectory-preserving objective on the high-noise interval and a distribution-matching objective on the low-noise interval, and couples the two stages through the crossover state. In contrast to loss-level mixing, and complementarily to training-time two-stage recipes, CrossDistill explicitly assigns complementary objectives along the noise axis, so that global branching is preserved before local statistics are sharpened. CrossDistill is a noise-level scheduling policy: PCM and DMD are plug-in instantiations, while the noise partition, crossover coupling, and objective ordering are the key design elements. Experiments on text-to-video diffusion models and qualitative image-to-video results show that CrossDistill expands the few-step quality-diversity frontier, retaining seed-level variation while achieving competitive visual fidelity.
Yuxi Liu, Haoyu Li, Yixiang Cai +8
Peking University, Melon Group · Alibaba Group · Tsinghua University
Sampling from pretrained diffusion and flow-matching models typically requires many forward passes to generate diverse and high-fidelity images. Existing distillation methods often rely on multiple auxiliary networks, carefully designed training stages, or complex optimization pipelines. In this work, we revisit the recently proposed Drifting Model objective and show that a single drifting loss can be directly used to simplify one step distillation. A key observation is that the pretrained diffusion teacher itself already provides a strong representation space. Unlike the original Drifting Model, which relies on an additional pretrained feature extractor, we use intermediate hidden states of the pretrained teacher model as the feature representation. This removes the need for training or introducing an extra representation network while preserving a semantically meaningful feature geometry for drifting. Furthermore, we introduce a lightweight mode coverage loss to mitigate mode collapse during distillation and encourage the student generator to cover diverse teacher-supported regions. Extensive experiments on ImageNet and SDXL demonstrate that our method achieves efficient one step generation with competitive image quality and diversity, achieving FID scores of 1.58 on ImageNet-64×64 and 18.4 on SDXL, while substantially simplifying the overall distillation framework.
Yuan Zhang, Chenyi Li, Guoqing Ma +7
JD Explore Academy, China · The Hong Kong University of Science and Technology · Tsinghua University +1