The recent wave of one-step generative models, which compress the multi-step trajectory of diffusion via either distillation or learned flow maps, has reached an inflection point where they can generate high-quality images. Here, we ask a natural question that follows from these advances: what happens to the denoising trajectory of multi-step diffusion when generation is compressed into a single forward pass? We offer an empirical observation we call depth as time: the denoising computation that multi-step diffusion performs across sampling steps appears to unfold across the depth of a single forward pass, and can be recovered by decoding intermediate layers with the model's own output head. Most interestingly, we show that this depthwise computation depends on the transport task a flow map is trained to solve. The most surprising case is MeanFlow, where probing shorter transport intervals reveals both denoising and renoising within a single network evaluation. In contrast, generators trained without a time-indexed transport task, such as drifting models, do not exhibit the same depthwise denoising. Consequently, we show that models that exhibit the depthwise denoising phenomenon are more compressible across the layerwise computation: a MeanFlow SiT-L/2 model can be compressed by 16.6× in parameters into a single time-conditioned block. We offer an explanation for this denoise-then-renoise behavior and show that, when we treat the layerwise computation explicitly as a flow, a single time-conditioned block can be trained to denoise across layers, compressing a MeanFlow SiT-L/2 model by 16.6× in parameters. Together, these results suggest that the temporal computation of diffusion is not eliminated by one-step generation, but reorganized across network depth.
Figures & tables
Figure 1: A one-step toy student distilled from a multi-step teacher appears to denoise across depth on the 8-Gaussians toy example. Normalized distance (right) shows that, although both progress toward the same endpoint, the student’s depthwise progression need not trace the teacher’s sampling-time trajectory. k/K is the normalized sampling step for the teacher, here K=64 steps.
Figure 2: Depthwise denoising-like progression persists in large-scale one-step image generators. Layerwise endpoint predictions from FLUX-schnell and MeanFlow progressively reveal cleaner image structure across network depth, while Drifting provides a contrasting τ -independent model. For FLUX-schnell, normalized endpoint distance decreases similarly under the model’s output head and an independently trained probe, whereas a random probe does not reproduce the same progression.
Figure 3: Depthwise mixing across one-step generators. A shared linear mixture of the input noise and final prediction closely reconstructs intermediate predictions for MeanFlow, FLUX-schnell, and Shortcut, with residuals decreasing across depth. In contrast, Drifting exhibits substantially larger residuals, indicating that the same two-component mixing model does not explain its depthwise trajectory as well.
Figure 4: For Meanflows, the recovered depthwise schedule is not the flow-matching interpolation schedule. Drifting, on the other hand, maintains σi≈0 at every block.
Figure 5: Each sampler step exhibits its own depthwise denoising progression. Layerwise endpoint predictions {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{r}^{i}} for two-step FLUX-schnell, decoded after each single block (sb). Within each NFE, the prediction is refined across depth toward the endpoint prescribed by the sampler: a still-noisy intermediate state in step 1, and the clean image in step 2.
Figure 6: Sampling time and network depth encode distinct but nested progressions. Normalized distance ρ(i) (Equation ( 4 )) for four-step FLUX-schnell and SANA-Sprint, computed from the endpoint predictions {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{r}^{i}} and the diagnostic clean predictions {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{0}^{i}} . Red curves show the layerwise progression within each NFE, gray curves show the multi-step teacher’s sampling trajectory, and dashed lines mark sampler steps. The {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{r}^{i}} curves reset at each step as the sampler advances, whereas the {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{0}^{i}} curves largely reach the final clean image within the first step.
Figure 7: Multi-step MeanFlow models trained on CIFAR-10 and ImageNet exhibit the denoise-then-renoise pattern. (a) Visualizing {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{r}} -predictions on the two-step generation trajectory across depth for a MeanFlow trained on CIFAR-10 2 2 2 Most CIFAR-10 diffusion and one-step generation codebases use DDPM++/NCSN++-style U-Nets ( Song et al., 2021 ) ; we train our own DiT-B/2 on CIFAR-10 for this experiment. reveals the denoise-then-renoise pattern ( top ). Quantitatively, the recovered depth schedule ρ(i) for a five-step generation trajectory , which decreases and then increases within each step ( bottom ). (b) We observe the depth as time on a MeanFlow with SiT-B/2 trained on ImageNet with one step generation ( top ). Using the same model but on three-step generation, we observe the same denoise and renoise pattern with {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{r}} -predictions ( middle ) and {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{0}} -predictions ( bottom ).
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Readout controls on one-step FLUX-schnell. Normalized distance ρ(i) to the final prediction under the model’s own readout Wout (blue), a trained linear probe (red), and a random linear probe (grey), alongside the FLUX-dev teacher’s sampling trajectory (black). The dashed line marks the transition from double-stream (db) to single-stream (sb) blocks.
Figure 9: Recovered depthwise mixing coefficients (αi,σi) . For the task-conditioned models, σi rises toward zero across depth; for Drifting, σi≈0 at every block.
Figure 10: Centering ablation. For MeanFlow, removing centering leaves the held-out reconstruction error essentially unchanged across intermediate blocks. For Drifting, adding centering does not improve the fit and instead makes it marginally worse. The final block is excluded from the normalized Ri comparison because its baseline error is zero by construction.
Figure 11
Figure 11: A single application of Bθ over a coarse interval (τa,τb) replaces several fine-grained transitions.
Figure 13
Model
Unique blocks
Params
K
Compression
FID (pre-FD)
FID (post-FD)
ω
MeanFlow SiT-B/2 (teacher)
12
131M
12
–
6.17
–
–
Ours
1
16.0M
12
8.2×
15.61
8.17
1.4
Ours
2
26.6M
12
4.9×
12.43
5.84
1.2
MeanFlow SiT-L/2 (teacher)
24
459M
24
–
4.01
–
–
Ours
1
27.6M
24
16.6×
11.75
4.87
1.4
Ours
2
46.5M
24
9.9×
7.19
4.04
1.3
Appendix
Table 1: Parameter compression on ImageNet 256×256 . FID-50k of fixed- K students at the teacher’s depth, before and after FD-loss post-training, each at its optimal CFG scale ω . Compression is relative to the teacher’s parameter count. Highlighted rows use a single shared block and are the results reported in Section 5 ; grey rows are the teachers. Adding unique blocks trades compression for quality.
Figure 12: Parameter efficiency of depth as time on ImageNet 256×256 . FID-50k is reported at each model’s optimal CFG scale. Hollow markers denote models before FD-loss post-training; filled markers denote the same models after. All baselines are as reported in their original papers. Our students reach competitive FID with 4.9 – 16.6× fewer parameters than their teachers.
B/2 (teacher FID 6.17 )
K
FLOPs ↓
elastic ↓
fixed- K↓
native, probed ↓
12 (native)
1.00×
8.75
5.84
5.84
6
0.50×
10.17
8.02
294.68
4
0.33×
15.24
11.31
313.64
2
0.17×
48.01
23.65
436.60
Appendix
Table 2: Compute compression on ImageNet 256×256 . FID-50k after FD-loss post-training for two-block students, each at its optimal CFG scale. FLOPs are relative to the same student at its native depth, with CFG in both cases. Elastic is a single model trained over the set K and run at each K . Fixed- K is a separate model trained for each K , and degrades gracefully as compute decreases. Native, probed is the native-depth model run with fewer, coarser applications than it was trained for, which collapses.
B/2 elastic , pre-FD
K
12
6
4
2
FID-50k
17.89
24.06
37.36
82.02
ω
1.5
1.5
1.5
1.7
Appendix
Table 3: Pre-FD FID-50k of the elastic students in Table 2 , at the optimal CFG scale ω for each K .
Figure 13: Compute compression of depth as time. FID-50k at the optimal CFG scale against the number of block applications K per generation; FLOPs scale approximately linearly with K . Fixed- K is a separate tied model trained for each depth; elastic is a single tied model run at every K . Solid markers are pre-FD, hollow markers post-FD.
Model
Blocks
K
Params
Clock
LR (schedule)
Batch
Steps
FID-50k
Tied + clock
1
20
14.4M
✓
10−4 (cosine)
64
400k
21.27
Tied, no clock
1
20
14.4M
✗
10−4 (cosine)
64
400k
45.45
Appendix
Table 4: Internal-clock ablation on CIFAR-10 , trained from scratch. The two models share architecture, parameters, compute, and optimization settings, and differ only in the internal clock. Removing it more than doubles FID-50k ( 21.27→45.45 ), showing that the clock is essential to the tied block.
Figure 14: Applying Wout to the residual states of pixel- and latent-space drifting DiT-B/2 models does not reveal the depth-as-time structure observed in task-conditioned models.
Model
Params
Compression
FID-50k
Degradation
Drifting B/2 (teacher)
132.7M
–
1.75
–
one tied block, K=12
15.8M
8.4×
47.09
≈27×
MeanFlow SiT-B/2 (teacher)
131M
–
6.17
–
one tied block, K=12
16.0M
8.2×
15.61
2.5×
Appendix
Table 5: Compressing a drifting model versus a MeanFlow model into a single time-conditioned block (pre-FD, ImageNet 256×256 ). Both teachers are compressed by a similar factor, yet the drifting student degrades far more than the MeanFlow student, despite distilling from a stronger teacher.
Figure 15: Samples from the MeanFlow SiT-L/2 teacher and its compressed students on ImageNet 256×256 . Each row within a panel is one class, and corresponding positions across the three columns use the same initial noise and class label. Students are fixed- K models with K=24 after FD-loss post-training (Table 1 ). Despite using 16.6× and 9.9× fewer parameters, the students largely preserve the teacher’s composition, pose, and color for each sample, with differences mostly in fine detail.
Figure 16: Additional samples from the MeanFlow SiT-L/2 teacher and its compressed students , arranged as in Figure 15 : each row within a panel is one class, and corresponding positions across the three columns use the same initial noise and class label.
Figure 17: Additional samples from the MeanFlow SiT-L/2 teacher and its compressed students , arranged as in Figure 15 : each row within a panel is one class, and corresponding positions across the three columns use the same initial noise and class label.
Figure 25
Figure 18: Depth as time in a Shortcut model. Top: endpoint predictions {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{t-d}^{\,i}} across depth during a two-step rollout. Within the first step, whose endpoint is noisy, predictions first reveal cleaner structure and then return toward the prescribed endpoint. Bottom left: normalized distance ρ(i) computed from the diagnostic clean predictions {\color[rgb]{0.6914,0.1406,0.0938}\tilde{\mathbf{x}}_{0}^{\,i}} . Bottom right: radially averaged power spectra of the endpoint predictions in the first step. Intermediate blocks move toward the clean-image spectrum before later blocks return toward the noisy endpoint spectrum.
DiffC provides a principled way to reuse pre-trained diffusion models for lossy compression, but its encoding and decoding procedures remain slow because they require many discretized forward and reverse steps. We study whether few-step generative models -- Rectified Flow, Consistency Trajectory Models (CTM), and MeanFlow -- can be cast as codecs within the same reverse channel coding (RCC) framework. The main challenge is that RCC requires posterior and shared distribution parameters, whereas these models do not explicitly parameterize intermediate conditional distributions. For Rectified Flow and MeanFlow, we use the equivalence between velocity parameterization and diffusion-style denoising parameterization to derive the quantities required by RCC. For CTM, which is distilled from EDM, we adopt the EDM noise parameterization together with local Gaussian approximations of the sender and shared distributions at intermediate states. This yields a proof-of-concept probabilistic formulation that enables compression with pre-trained few-step generative models without retraining. On low-resolution benchmarks, the resulting codecs reduce encoding and decoding time and improve realism in the low-bit-rate regime.
Modern diffusion and flow-based models are increasingly moving toward few-step, latent-free generation to bypass the computational overhead of multi-step sampling and the reconstruction bottlenecks of external autoencoders. We propose the Interval Denoiser, a theoretically rigorous framework for latent-free generation. Derived directly from the flow matching ODE, it establishes an exact analytical mapping for intermediate trajectory states. Unlike prior formulations, our prediction is shown to reside on a low-dimensional manifold across any time interval, making the regression tractable for a network operating directly on pixels. Furthermore, by avoiding empirical algebraic substitutions, our formulation correctly isolates the pure time derivative to prevent biased gradient evaluations and ensure exact first-order optimization. By analyzing this objective, we equip our framework with residual clipping and a time-sampling curriculum, enabling effective long-interval training and improving few-step performance. Trained from scratch on ImageNet 256x256, our model achieves an FID of 4.55 in one step (1-NFE) and 3.98 in two steps (2-NFE) without perceptual losses.
Alexander Zaytsev, Dmitry Baranchuk, Alexander Korotin +1
HSE University · Yandex Research · Applied AI Institute +2
Recently, diffusion models (DMs) have made significant strides in high-quality image generation. However, the multi-step denoising process often results in considerable computational overhead, impeding deployment on resource-constrained edge devices. Existing methods mitigate this issue by compressing models and adjusting the time step sequence. However, they overlook input redundancy and require lengthy search times. In this paper, we propose Coarse-to-Fine Diffusion Models with Time Step Sequence Redistribution. Recognizing indistinguishable early-stage generated images, we introduce Coarse-to-Fine Denoising (C2F) to reduce computation during coarse feature generation. Furthermore, we design Time Step Sequence Redistribution (TRD) for efficient sampling trajectory adjustment, requiring less than 10 minutes for search. Experimental results demonstrate that the proposed methods achieve near-lossless performance with an 80% to 90% reduction in computation on CIFAR10 and LSUN-Church.
Yu-Shan Tai, An-Yeu, Wu
Andy · Graduate Institute of Electrical Engineering, National Taiwan University, Taipei, Taiwan