Diffusion models have achieved remarkable progress in high-fidelity image, video, and audio generation, yet inference remains computationally expensive. Nevertheless, current diffusion acceleration methods based on distributed parallelism suffer from noticeable generation artifacts and fail to achieve substantial acceleration proportional to the number of GPUs. Therefore, we propose a hybrid parallelism framework that combines a novel data parallel strategy, condition-based partitioning, with an optimal pipeline scheduling method, adaptive parallelism switching, to reduce generation latency and achieve high generation quality in conditional diffusion models. The key ideas are to (i) leverage the conditional and unconditional denoising paths as a new data-partitioning perspective and (ii) adaptively enable optimal pipeline parallelism according to the denoising discrepancy between these two paths. Our framework achieves 2.31× and 2.07× latency reductions on SDXL and SD3, respectively, using two NVIDIA RTX~3090 GPUs, while preserving image quality. This result confirms the generality of our approach across U-Net-based diffusion models and DiT-based flow-matching architectures. Our approach also outperforms existing methods in acceleration under high-resolution synthesis settings. Code is available at https://github.com/kaist-dmlab/Hybridiff.
Figures & tables
Figure 2: Comparison of parallel strategies for diffusion inference. (a) Patch-based data parallel frameworks suffer from bottlenecks caused by all-gather operations and artifacts at patch boundaries, leading to limited acceleration and quality degradation. (b) Pipeline parallel frameworks incur excessive asynchronous communication overhead and accumulate estimate errors. (c) Our hybrid parallelism , which incorporates condition-based data parallelism, adaptively combines both paradigms to achieve high fidelity and fast generation.
Figure 3: Overview of the proposed diffusion inference hybrid parallel framework. Our method adaptively switches parallelism modes at τ1 and τ2 , optimizing the trade-off between computational efficiency and consistency of conditional guidance, and demonstrates superior inference acceleration performance while preserving high generation quality.
Figure 4: Illustration of the rel-MAEt(ϵc,ϵu) curve. The rel-MAEt(ϵc,ϵu) value is relatively large before τ1 and after τ2 , while it converges near zero between them, indicating stable alignment between conditional and unconditional branches during the parallelism phase.
Base Model
Devices
Methods
Latency (s) ↓
Speed-Up ↑
Comm. (GB) ↓
FID ↓
LPIPS ↓
PSNR ↑
w/ G. T.
w/ Orig.
w/ G. T.
w/ Orig.
w/ G. T.
w/ Orig.
Stable Diffusion XL
1
Original Model
16.49
-
-
23.977
-
0.797
-
9.618
-
2
DistriFusion [ 11 ]
13.53
1.22 ×
0.525
24.164
4.864
0.7978
0.146
9.597
24.634
AsyncDiff [ 2 ] (stride=1)
12.54
1.31 ×
9.830
23.941
4.103
0.797
0.108
9.586
26.387
Ours ( k =5)
7.12
2.31 ×
0.516
23.831
4.100
0.796
0.107
9.665
26.640
Stable Diffusion 3
1
Original Model
19.36
-
-
33.433
-
0.810
-
8.086
-
Table 1: Quantitative comparison of parallelism methods on the Stable Diffusion XL and Stable Diffusion 3 models. We compare our method with existing distributed inference techniques under 1- and 2-GPU. We report both the baseline latency and the corresponding acceleration ratio ( Speed-Up ), Communication efficiency (Comm.) , and quantitative metrics assessing generation fidelity. Here, w/ G.T. denotes comparison with the ground-truth image, and w/ Orig. indicates comparison with the original (single-GPU) model output.
Figure 5: Qualitative results of the main experiments. We compare 1024 × 1024 image generations from the SDXL model. Our method achieves the best acceleration and FID performance, while producing visuals most similar to the original.
Figure 6: Visualization of speed–quality trade-off across different parallelism intervals k . Smaller k values preserve higher fidelity, whereas larger k achieve greater acceleration. Our method consistently dominates prior works across the trade-off frontier. All experiments were conducted on 2 GPUs.
Methods
Latency (s) ↓
Speed-Up ↑
FID ↓
(w/ Orig.)
Original Model
16.49
-
-
Full Condition-Based Partitioning
9.24
1.78 ×
3.623
Ours (Hybrid Parallelism)
7.12
2.31 ×
4.100
Table 2: Ablation on hybrid parallel components. All experiments are conducted on the SDXL model at 1024 × 1024 resolution, comparing the single-GPU baseline, full condition-based partitioning, and our hybrid parallelism framework.
Figure 7: Comparison of high-resolutions tasks. We compare different parallel inference methods on the SDXL model using NVIDIA H200 GPUs across 1024 × 1024, 2048 × 2048, and 2560 × 2560 high-resolutions.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
Speed-Up ↑
Image Quality ↑
Model General. ↑
High-res Synth. ↑
Comm. Efficiency ↑
Distrifusion
2.5
3.5
2.5
3.3
5.0
AsyncDiff
3.0
4.5
5.0
3.5
1.0
Ours
4.7
4.5
5.0
4.4
5.0
Appendix
Table 3: Quantitative metrics comparison across five evaluation aspects. Scores are normalized to a 5-point scale. Higher values ( ↑ ) indicate better performance.
Figure 8: Empirical visualization of denoising discrepancy curve.
Figure 9: Extensibility to many GPU configurations structures. This figure illustrates two strategies for scaling the proposed hybrid parallelism framework to larger GPU configurations. These structures demonstrate how the proposed framework naturally generalizes from the 2 GPUs setting to both batch-level and layer-wise many GPU configurations.
Parallelism Interval k
Latency (s) ↓
Speed-Up ↑
FID ↓
(w/ Orig.)
k =5
7.12
2.31 ×
4.100
k =10
6.89
2.39 ×
5.942
k =20
6.44
2.56 ×
7.966
k =30
5.94
2.78 ×
9.191
Appendix
Table 4: Effect of speed-quality trade-off across different parallelism intervals k . All experiments are conducted on the SDXL model at 1024 × 1024 resolution with various parallelism intervals.
Figure 10: Additional qualitative results of the main experiments. We compare 1024 × 1024 image generations from the SDXL model. Our method achieves the best acceleration and FID performance, while producing visuals most similar to the original.
Figure 11: Additional qualitative comparisons across different k values. We compare 1024 × 1024 image generations from the SDXL model across various parallelism intervals. Smaller k values preserve higher visual fidelity, whereas larger k gradually reduce local detail due to the extended parallelism window. Although the overall appearance remains similar, fine-grained conditional attributes become subtly blurred as k increases.
Diffusion models have achieved remarkable success in generating high-fidelity content but suffer from slow, iterative sampling, resulting in high latency that limits their use in interactive applications. We introduce DRiffusion, a parallel sampling framework that parallelizes diffusion inference through a draft-and-refine process. DRiffusion employs skip transitions to generate multiple draft states for future timesteps and computes their corresponding noises in parallel, which are then used in the standard denoising process to produce refined results. Theoretically, our method achieves an acceleration rate of n1 or n+12, depending on whether the conservative or aggressive mode is used, where n denotes the number of devices. Empirically, DRiffusion attains 1.4×-3.7× speedup across multiple diffusion models while incur minimal degradation in generation quality: on MS-COCO dataset, both FID and CLIP remain largely on par with those of the original model, while PickScore and HPSv2.1 show only minor average drops of 0.17 and 0.43, respectively. These results verify that DRiffusion delivers substantial acceleration and preserves perceptual quality.
Runsheng Bai, Chengyu Zhang, Yangdong Deng
CSAIL, MIT · IST, Nanjing University · School of Software, Tsinghua University; FuturististAI Lab, Shanghai Tsinghua International Innovation Center
Diffusion models have achieved remarkable success in generative AI, yet their computational efficiency remains a significant challenge, particularly for Diffusion Transformers (DiTs) requiring intensive full-attention computation. While existing acceleration approaches focus on content-agnostic uniform optimization strategies, we observe that different regions in generated content exhibit heterogeneous convergence patterns during the denoising process. We present Jano, a training-free framework that leverages this insight for efficient region-aware generation. Jano introduces an early-stage complexity recognition algorithm that accurately identifies regional convergence requirements within initial denoising steps, coupled with an adaptive token scheduling runtime that optimizes computational resource allocation. Through comprehensive evaluation on state-of-the-art models, Jano achieves substantial acceleration (average 2.0 times speedup, up to 2.4 times) while preserving generation quality. Our work challenges conventional uniform processing assumptions and provides a practical solution for accelerating large-scale content generation. The source code of our implementation is available at https://github.com/chen-yy20/Jano.
Yuyang Chen, Linqian Zeng, Yijin ZHou +2
Shanghai Jiaotong University · Tsinghua University
Existing text-to-image diffusion models excel at generating high-quality images, but face significant efficiency challenges when scaled to high resolutions, like 4K image generation. While previous research accelerates diffusion models in various aspects, it seldom handles the inherent redundancy within the latent space. To bridge this gap, this paper introduces DC-Gen, a general framework that accelerates text-to-image diffusion models by leveraging a deeply compressed latent space. Rather than a costly training-from-scratch approach, DC-Gen uses an efficient post-training pipeline to preserve the quality of the base model. A key challenge in this paradigm is the representation gap between the base model's latent space and a deeply compressed latent space, which can lead to instability during direct fine-tuning. To overcome this, DC-Gen first bridges the representation gap with a lightweight embedding alignment training. Once the latent embeddings are aligned, only a small amount of LoRA fine-tuning is needed to unlock the base model's inherent generation quality. We verify DC-Gen's effectiveness on SANA and FLUX.1-Krea. The resulting DC-Gen-SANA and DC-Gen-FLUX models achieve quality comparable to their base models but with a significant speedup. Specifically, DC-Gen-FLUX reduces the latency of 4K image generation by 53x on the NVIDIA H100 GPU. When combined with NVFP4 SVDQuant, DC-Gen-FLUX generates a 4K image in just 3.5 seconds on a single NVIDIA 5090 GPU, achieving a total latency reduction of 138x compared to the base FLUX.1-Krea model. Code: https://github.com/dc-ai-projects/DC-Gen.