Organizations: University of California, Irvine, CA 92697, USA · University of Alabama at Birmingham, Birmingham, AL 35294, USA · Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA
Existing image generation models face critical challenges regarding the trade-off between computation and fidelity. Specifically, models relying on a pretrained Variational Autoencoder (VAE) suffer from information loss, limited detail, and the inability to support end-to-end training. In contrast, models operating directly in the pixel space incur prohibitive computational cost. Although cascade models can mitigate computational cost, stage-wise separation prevents effective end-to-end optimization, hampers knowledge sharing, and often results in inaccurate distribution learning within each stage. To address these challenges, we introduce a unified multi-stage generative framework formulated under the stochastic interpolant formalism with our Conditional Dependent Coupling strategy as a Flow-Matching variant. It decomposes the generative process into interpolant trajectories at multiple stages, ensuring accurate distribution learning while enabling end-to-end optimization. Importantly, the entire process is modeled as a single unified Diffusion Transformer, eliminating the need for disjoint modules and also enabling knowledge sharing. Under explicitly stated assumptions, we provably reduce both transport cost and asymptotic inference time; we empirically validate the underlying NFE-transport-cost and per-evaluation-cost relations. Extensive experiments demonstrate that our method achieves competitive fidelity and substantially lower wall-clock cost in the low-NFE regime across multiple resolutions.
Figures & tables
Figure 1: Data Distribution Learning and Transition.
Method
Params
FID ↓
IS ↑
Prec. ↑
Rec. ↑
NFE
Speed
GAN
BigGAN [ 39 ]
112M
6.95
224.5
0.89
0.38
1
1.46
GigaGAN [ 40 ]
569M
3.45
225.5
0.84
0.61
1
1.32
StyleGAN [ 41 ]
166M
2.30
265.1
0.78
0.53
1
0.96
Diffusion / Flow
ADM [ 42 ]
554M
10.94
101.0
0.69
0.63
250×2
9.46
U - ViT [ 9 ]
287M
3.40
219.9
0.83
0.52
-
-
LDM - 4 - G [ 4 ]
400M
3.60
247.7
-
-
250×2
4.18
Table 1: Comprehensive comparison of generative models on ImageNet-1k/256. Missing values are denoted “-”. In the NFE field, “×2" indicates that CFG incurs an NFE of 2 per sampling step.
Figure 3Figure 4
Figure 6: FID under varying γ and σ .
Architecture Comparison
Fusion Ablation
Method
FID
Params
Time
Variant
FID
Unified
1.81
677M
76h
w/ Attn
1.81
Stage-spec.
2.68
130×3+458 M
78h
w/o Attn
1.84
Table 2: Ablation studies on Unified Architecture and Fusion Mechanism.
Interpolant Variant
Hierarchy Setting
Type
Definition
FID
Stages
Res. Path
FID
Linear
αt=1−τ;βt=τ
1.81
4
32…256
1.81
Trig.
αt=cos2πτ;βt=sin2πτ
2.11
2
64, 256
1.97
VP-SDE
αt=1−τ2;βt=τ
2.03
-
-
-
Table 3: Ablation studies on Interpolants (Left) and Hierarchy Stages (Right).
Figure 8: Visualization of each stage in CDC-FM, including the output at every stage, the details added at each stage, and the corresponding hierarchical structure of those details.
Figure 9: The per-stage FID under varying Scfg .
Resolution dk
322
642
1282
Time / function-eval (ms)
5.6
22.0
87.4
Normalized to d1
1.0×
≈4.0×
≈16.0×
Appendix
Table 4: Per-function-evaluation cost across resolutions (single forward pass of the unified DiT, in milliseconds, averaged over 200 calls on a single H100 GPU at batch size 1 ). The cost scales approximately linearly with the generation dimension dk , which empirically supports Assumption G.5 (G.4) used in the proof of Theorem M.1 .
(t1,t2)
(0.0,1.0)
(0.0,0.75)
(0.0,0.5)
(0.0,0.25)
(0.25,0.5)
Transport cost C
1.00
0.563
0.250
0.063
0.063
Required NFE
64
36
16
4
4
NFE / C
64
63.9
64.0
63.5
63.5
Appendix
Table 5: Empirical relationship between transportation cost C(t1,t2)=E[∣x1−x0∣2] and the NFE required to reach a fixed target FID ( 1.85 ) using SiT as the generative model on ImageNet-1k/256. For each (t1,t2) pair we draw 1,000 pairs of intermediate states from a linear interpolant and sweep the NFE upward until the target FID is hit. Lower transport cost consistently leads to proportionally fewer NFEs, and the trend is consistent across different intermediate states; this empirically supports Assumption G.6 (G.5).
Method
FID ↓
IS ↑
NFE
Speed (s/img) ↓
Single-stage Pixel ( K=1 )
1.82
305.8
250×2
8.73
CDC-FM (Ours)
1.81
304.3
16×4×2
5.18
Appendix
Table 6: Single-stage pixel baseline ( K=1 ) vs. CDC-FM under an identical backbone, training schedule, optimizer, and ODE solver. Removing only the conditional dependent coupling and re-collapsing the model to one stage costs ∼1.7× wall-clock at matched FID, isolating the efficiency gain attributable to the proposed coupling. Speed is reported in seconds per image at batch size 1 on a single H100 GPU.
Stages
Resolution Path
FID ↓
2
32→256
2.06
2
64→256
1.97
3
32→128→256
1.83
4
32→64→128→256
1.81
Appendix
Table 7: Impact of the stage count K and the resolution path on ImageNet-1k/256. Excessively large per-stage resolution ratios degrade FID (rows 1–2). With moderate ratios, fewer stages remain viable (rows 3–4): the gap between K=3 and K=4 is small, while K=4 obtains the best FID. Choices below the 322 floor yield insufficient structural/semantic content and were excluded.
N1
N2
N3
N4
Total NFE
FID ↓
16
16
16
16
64
1.81
8
16
16
16
56
1.83
4
16
16
16
52
1.85
16
8
16
16
56
1.85
16
4
16
16
52
1.87
16
16
8
16
56
1.88
Appendix
Table 8: Per-stage NFE allocation ablation on ImageNet-1k/256 with the 32→64→128→256 hierarchy. We hold three of the per-stage NFEs at 16 and reduce the fourth to {8,4} . Reducing the NFE of any single stage degrades FID, but the degradation is substantially larger at later, higher-resolution stages, indicating that solver-step budget should be biased toward high-resolution stages.
Model/Method
Params
FID ↓
IS ↑
BigGAN [ 39 ]
-
8.43
177.9
ADM [ 42 ]
554M
7.72
172.7
VDM++ [ 11 ]
2B
2.65
278.1
DiT - XL/2 [ 5 ]
675M
3.04
240.8
U - ViT [ 9 ]
501M
4.05
263.8
SiD2 [ 10 ]
2.0B
2.19
-
Appendix
Table 9: Comparison on ImageNet-1k/512 conditional generation.