ParaLin++: Accelerating Parallel Diffusion Integrator via Intrinsic Partially Linear Structure

Jianrong Lu; Zhiyu Zhu; Hui Liu; Junhui Hou

cs.CVOct 22, 2026

ParaLin++: Accelerating Parallel Diffusion Integrator via Intrinsic Partially Linear Structure

Authors: Jianrong Lu, Zhiyu Zhu, Hui Liu, Junhui Hou

Organizations: the Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · City University of Hong Kong (Dongguan), Dongguan, China · the School of Computing and Information Sciences, Saint Francis University, Hong Kong SAR, China

Abstract

Training-free few-step samplers are often described as nearly quality-lossless under conventional FID/CLIP-style evaluation. It remains unclear whether that conclusion transfers to more realistic and complex generation-quality benchmarks. We first systematize the quality metrics used to evaluate training-free sampling acceleration for diffusion and flow-matching models. We find that representative training-free step-shortening methods often remain nearly lossless under FID/CLIP-class metrics, yet can suffer substantial quality degradation when evaluated with semantic-composition, dense-instruction, and long-text-rendering benchmarks such as GenEval, DPG-Bench, and LongTextBench. Parallel sampling is a notable exception: by retaining the full-step discretization and removing only the solver's temporal dependency, it can preserve near-lossless quality while accelerating sampling. Building on this observation, we propose ParaLin++, a step-parity parallel sampler that preserves the full-step discretization while accelerating the solver iterations. Experiments across Stable Diffusion-v2, LSUN Church, FLUX.1-dev, and Ming-Flash-Omni 2.0 show that ParaLin++ preserves the full-step Euler quality profile at $2.6\times$ -- $4.8\times$ , $1.68\times$ on FLUX, and $1.21\times$ on Ming. Project page: https://paralinplus.github.io/.

Figures & tables

Method	Dist.	T2I-align.	Complex
DDPM [ 22 ]	✓	✗	✗
DDIM [ 57 ]	✓	✗	✗
DPM-Solver [ 42 ]	✓	✗	✗
DPMSolver++ [ 41 ]	✓	✓	✗
DPM-Solver-v3 [ 76 ]	✓	✗	✗
UniPC [ 73 ]	✓	✓	✗

TABLE I: Quality metrics used to evaluate training-free diffusion/flow-matching acceleration in the literature and in this work. “Dist.” = distributional, reference-based metrics of the FID class (FID, sFID, clean-FID, IS, Precision–Recall); “T2I-align.” = text–image alignment metrics (CLIP score, CLIP-FID); “Complex” = generation-quality benchmarks that assess semantic composition, dense instruction following, or in-image text rendering. The acceleration literature usually stops at distributional/text-alignment proxies, whereas this work adds a quality-focused audit. ✓/✗: metric reported / not reported by the method’s own evaluation.

	EDM [ 29 ]	VE [ 60 ]	VP [ 60 ]
Schedule settings	$\begin{aligned} \alpha_{t}&=1\\ \sigma_{t}&=t\end{aligned}$	$\begin{aligned} \alpha_{t}&=1\\ \sigma_{t}&=\sqrt{t}\end{aligned}$	$\begin{aligned} \alpha_{t}&=1/\sqrt{e^{\frac{1}{2}\beta_{d}t^{2}+\beta_{\min}t}}\\ \sigma_{t}&=\alpha_{t}\sqrt{e^{\frac{1}{2}\beta_{d}t^{2}+\beta_{\min}t}-1}\end{aligned}$
Substitution $\sigma_{t}/\alpha_{t}=$	$t$	$\sqrt{t}$	$\sqrt{e^{\frac{1}{2}\beta_{d}t^{2}+\beta_{\min}t}-1}$
Final error bound	$\left\\|\mathcal{Z}_{s}\right\\|\leq\zeta(s-t)=\zeta h$	$\left\\|\mathcal{Z}_{s}\right\\|\leq\zeta(\sqrt{s}-\sqrt{t})$	$\left\\|\mathcal{Z}_{s}\right\\|\leq\zeta\alpha_{t}(\sigma_{s}/\alpha_{s}-\sigma_{t}/\alpha_{t})$

TABLE II: Practical error bounds for diffusion frameworks in the EDM design space. Parameters follow Table 1 of [ 29 ] , with

\beta_{\min}=0.1

\beta_{d}=19.9

, and

\left\|\mathbf{S}_{\theta^{*}}(\mathbf{X}_{t},t)-\mathbf{\epsilon}_{t}\right\|<\zeta

for a small step size

h=s-t<1

. Rows give the schedule settings, the

\sigma_{t}/\alpha_{t}

substitution, and the resulting bound; columns are the diffusion frameworks.

Fig. 1: Illustration of ParaLin++. During iteration $k$ , ParaLin++ performs simultaneous denoising updates across multiple steps. From left to right: existing full nonlinear parallel methods, our partial linear method, and our method for faster convergence.

Steps	Method	Stable Diffusion-v2
Steps	Method	Iters $\downarrow$	NFE $\downarrow$	CLIP $\uparrow$	FID $\downarrow$	Time (s) $\downarrow$	Speedup $\uparrow$
50	DDIM [ 57 ]	50	50	25.6	57.2	6.3	1.0 $\times$
	DDIM + ParaDiGMS [ 56 ]	21	132	25.6	56.9	3.3	1.9 $\times$
	DDIM + ParaSolver [ 45 ]	13	83	25.6	56.9	1.9	3.3 $\times$
	DDIM + ParaLin [ 46 ]	13	73	25.6	57.1	1.7	3.7 $\times$
	DDIM + ParaLin++	10	69	25.5	57.0	1.3	4.8 $\times$

TABLE III: Comparisons on Stable Diffusion-v2, with classifier guidance w = 7.5. The visual comparisons are shown in the supplementary figures.

Steps	Method	LSUN Church
Steps	Method	Iters $\downarrow$	NFE $\downarrow$	FID $\downarrow$	Time (s) $\downarrow$	Speedup $\uparrow$
50	DDIM [ 57 ]	50	50	15.5	1.8	1.0 $\times$
	DDIM + ParaDiGMS [ 56 ]	23	202	15.8	1.5	1.2 $\times$
	DDIM + ParaSolver [ 45 ]	14	73	15.7	0.8	2.2 $\times$
	DDIM + ParaLin [ 46 ]	14	67	15.7	0.6	3.0 $\times$
	DDIM + ParaLin++	9	64	15.7	0.4	4.5 $\times$

TABLE IV: Quantitative comparisons of different methods on LSUN Church over

5000

samples. DPMSolver is not integrated with the LSUN model in the Diffusers library; thus we exclude it.

Method	Iters	FID $\downarrow$	Speedup $\uparrow$
DPM-Solver-v3 [ 76 ]	25	2.64	$1.00\times$
DPM-Solver-v3 [ 76 ]	10	3.40	$2.43\times$
UniPC [ 73 ]	10	3.40	$2.41\times$
PD [ 54 ]	10	3.10	$2.48\times$
CD [ 58 ]	10	2.90	$2.46\times$
ParaLin++ + DPM-Solver-v3	10	2.64	$2.41\times$

TABLE V: Less-step sampling on the CIFAR-10 model with 50k samples.

Method	Sampling regime	Discrete steps	Iters $\downarrow$	GenEval $\uparrow$	DPG-Bench $\uparrow$	LongTextBench $\uparrow$	Speedup $\uparrow$
Euler [ 4 ]	Sequential reference	30	30	0.66	83.97	0.27	$1.00\times$
UniPC [ 73 ]	Few-step	16	16	0.52	77.46	0.09	$1.81\times$
DPMSolver++ [ 41 ]	Few-step	16	16	0.50	75.75	0.08	$1.85\times$
ParaLin++	Step-parity parallel	30	16.37	0.64	84.24	0.26	$1.68\times$

TABLE VI: Motivating comparison on FLUX. Few-step sequential solvers alter the 30-step discretization and sharply degrade semantic composition, complex prompt following, and long-text rendering. ParaLin++ keeps the same 30 discrete diffusion steps and parallelizes the solver iterations. For sequential solvers, “Iters” equals the number of discrete steps; for ParaLin++, it reports the average number of parallel solver iterations.

FLUX.1-dev [ 4 ]
Method	Sampling regime	Steps	Iters	GenEval $\uparrow$	DPG-Bench $\uparrow$	LongTextBench $\uparrow$	Speedup $\uparrow$
Euler [ 4 ]	Sequential reference	30	30	0.66	83.97	0.27	$1.00\times$
UniPC [ 73 ]	Few-step	16	16	0.52	77.46	0.09	$1.81\times$
		20	20	0.55	79.67	0.13	$1.49\times$
		24	24	0.57	80.98	0.15	$1.26\times$
		26	26	0.58	81.35	0.16	$1.16\times$

TABLE VII: Detailed generation-quality benchmark results on FLUX and Ming-Flash-Omni 2.0. Within each model block, speedup is measured against that model’s 30-step Euler baseline on the same GenEval generation pass, excluding model loading and scoring. For sequential samplers, “Iters” equals the number of discrete steps; for ParaLin++, it denotes the number of parallel solver iterations.

CFG	Method	Iters	GenEval $\uparrow$	DPG $\uparrow$	LTB $\uparrow$	Speedup $\uparrow$
0	Euler [ 4 ]	30	0.60	82.25	0.17	$1.00\times$
	UniPC [ 73 ]	20	0.46	78.97	0.08	$1.49\times$
	DPM++ [ 41 ]	20	0.43	77.71	0.06	$1.49\times$
	ParaLin++	14.79	0.59	82.18	0.15	$1.86\times$
3.5	Euler [ 4 ]	30	0.66	83.97	0.27	$1.00\times$
	UniPC [ 73 ]	20	0.55	79.67	0.13	$1.49\times$

TABLE VIII: FLUX.1-dev CFG-sweep results. All benchmark cells (GenEval, DPG-Bench, and LongTextBench) are complete for all five CFG values; LTB abbreviates LongTextBench and DPM++ abbreviates DPMSolver++. “Iters” equals the discrete-step count for sequential baselines and denotes the average measured parallel solver iterations for ParaLin++. Speedup is measured against the 30-step Euler reference at the same CFG on the GenEval generation pass, excluding model loading and scoring.

Fig. 2: Generation-quality benchmark visual comparison on FLUX.1-dev. Each row is one representative benchmark prompt, and the columns show 30-step Euler, 16-step UniPC, 16-step DPMSolver++, and 30-step ParaLin++ (17 parallel solver iterations). Few-step examples retain plausible low-level appearance but can violate semantic composition, dense instructions, or in-image text requirements; ParaLin++ remains visually aligned with the 30-step Euler reference.

Fig. 3: GenEval CFG-sweep visual comparison on FLUX.1-dev. Rows are CFG $0$ , $3.5$ , $7$ , $10.5$ , and $14$ ; columns are 30-step Euler, 20-step UniPC, 20-step DPMSolver++, and 30-step ParaLin++. All cells use prompt 00000 and sample 0000.

Fig. 4: LongTextBench CFG-sweep visual comparison on FLUX.1-dev, using the English prompt 0000 with variant 1. Rows are CFG $0$ , $3.5$ , $7$ , $10.5$ , and $14$ ; columns are 30-step Euler, 20-step UniPC, 20-step DPMSolver++, and 30-step ParaLin++. The reported LongTextBench metric averages the English and Chinese scores.

Fig. 5: DPG-Bench CFG-sweep visual comparison on FLUX.1-dev. Rows are CFG $0$ , $3.5$ , $7$ , $10.5$ , and $14$ ; columns are 30-step Euler, 20-step UniPC, 20-step DPMSolver++, and 30-step ParaLin++. All cells use prompt 100. The complete CFG- $0$ – $14$ DPG-Bench matrix is reported in Table VIII .

Sampler	Method	Iters $\downarrow$	NFE $\downarrow$	CLIP $\uparrow$	FID $\downarrow$	Time (s) $\downarrow$	Speedup $\uparrow$
DDIM–50	Sequential	50	50	25.6	57.2	6.3	1.0 $\times$
	ParaDiGMS	34	67	25.6	56.9	7.8	0.8 $\times$
	ParaSolver	18	53	25.6	56.9	6.0	1.05 $\times$
	ParaLin	14	51	25.6	57.1	5.7	1.1 $\times$
	ParaLin++	8	32	25.6	57.1	3.2	2.0 $\times$
DDIM–100	Sequential	100	100	25.6	56.3	10.1	1.0 $\times$

TABLE IX: Comparisons on Stable Diffusion-v2 when only using one GPU, with classifier guidance

w=7.5

. “DPM” denotes DPM-Solver; ParaDiGMS, ParaSolver, ParaLin, and ParaLin++ indicate the corresponding parallel wrappers of the sequential sampler.

Configuration	Solver components			100-step DDIM results
Configuration	Nonlinearity	Stopping criterion	SOR	Iters $\downarrow$	NFE $\downarrow$	Time (s) $\downarrow$	Speedup $\uparrow$
Sequential DDIM	–	–	–	100	100	10.6	$1.0\times$
Full nonlinearity	Full ( $\varrho=100$ )	Fixed ( $\beta=1.0$ )	No ( $\omega=1.0$ )	72	2100	24.4	$0.43\times$
Partial nonlinearity	Partial ( $\varrho=8$ )	Fixed ( $\beta=1.0$ )	No ( $\omega=1.0$ )	18	140	2.1	$5.0\times$
Partial + adaptive stopping	Partial ( $\varrho=8$ )	Adaptive ( $\beta=0.999$ )	No ( $\omega=1.0$ )	17	139	2.0	$5.3\times$
ParaLin++ (partial + adaptive + SOR)	Partial ( $\varrho=8$ )	Adaptive ( $\beta=0.999$ )	Yes ( $\omega=1.04$ )	15	117	1.6	$\mathbf{6.5\times}$

TABLE X: Ablation study on Stable Diffusion v2. The table isolates nonlinearity, adaptive stopping, and SOR in the 100-step DDIM task; speedup is measured relative to the sequential DDIM baseline.

Method	EDM Process		VE Process		VP Process
Method	FID	Iters	FID	Iters	FID	Iters
Baseline	11.1649	25	423.0628	25	104.14673	25
$\varrho=4$	10.8811	22	422.9794	14	35.8963	12
$\varrho=6$	10.8718	21	422.9293	14	41.1196	10
$\varrho=8$	10.8621	20	422.8247	13	58.5588	8
$\varrho=10$	10.8626	15	422.7672	10	58.5215	7

TABLE XI: The impact of nonlinear degree

\varrho

on FID across EDM, VE, and VP processes over ImageNet dataset. We set

\delta=0.01,\beta=9e-4

for the EDM process,

\delta=0.01,\beta=9e-7

for VP processes, and

\delta=0.1,\beta=9e-5

for the VE process.

Method	FID	CLIP	Iters	NFEs	Speedup
Baseline	57.23	25.64	50	50	1 $\times$
$\varrho=5$	57.18	25.55	18	82	2.9 $\times$
$\varrho=10$	57.25	25.59	12	101	3.0 $\times$
$\varrho=15$	57.14	25.61	11	123	3.1 $\times$
$\varrho=20$	57.22	25.60	12	160	2.6 $\times$
$\varrho=25$	57.14	25.53	12	202	2.1 $\times$

TABLE XII: Impact of varying

\varrho

on generation quality (FID, CLIP) and performance (NFEs, Speedup).

Method	$\delta=0.01$		$\delta=0.05$		$\delta=0.1$
Method	FID	Iters	FID	Iters	FID	Iters
DDIM	57.2	50	57.2	50	57.2	50
$\beta=0.9$	59.7	9	58.6	9	57.8	9
$\beta=0.99$	57.8	13	57.1	13	56.9	11
$\beta=0.999$	57.0	16	56.9	13	56.8	11
$\beta=1.0$	57.0	21	57.1	13	57.2	11

TABLE XIII: The effect of the tolerance

\delta

and EMA decay rate

\beta

on FID using Stable Diffusion-v2 over

5000

random samples. Our ParaLin++ (

\varrho=8,p=40

) is employed for DDIM.

Method	FID	CLIP	Iters	NFEs	Speedup
DDIM	57.23	25.64	50	50	1 $\times$
$\omega=1$	57.20	25.59	11	83	3.81 $\times$
$\omega=1.02$	57.38	25.57	11	79	3.89 $\times$
$\omega=1.04$	57.38	25.57	10	70	4.34 $\times$
$\omega=1.06$	57.24	25.56	9	67	4.76 $\times$
$\omega=1.08$	58.13	25.12	9	67	4.94 $\times$

TABLE XIV: Impact of varying

\omega

on generation quality and performance. We set

\varrho=8,p=40,\beta=0.999

Panel of Figure 20 — Fig. 6: Ablation study on parameters. The first row shows the results for different values of $\varrho$ . The second row shows the results for different values of $\beta$ . The third row shows the results for different values of $\omega$ . The last image in each row represents the baseline comparison.

Diffusion Sampling

Diffusion sampling studies how to turn a trained diffusion model into high-quality samples efficiently and reliably.

See all 264 papers

DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease
Mar 26, 2026Runsheng Bai, Chengyu Zhang, Yangdong DengDiffusion Model Sampling Diffusion Model Inference Acceleration
CSAIL, MIT · IST, Nanjing University · School of Software, Tsinghua University; FuturististAI Lab, Shanghai Tsinghua International Innovation Center
Unlocking Few-Step Diffusion for Faithful Previews
Sep 28, 2026Jing Jia, Sifan Liu, Guanyang WangDiffusion Sampling Few-Step Diffusion Sampling
Department of Computer Science, Rutgers University · Department of Statistical Science, Duke University · Department of Statistics, Rutgers University
LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
Jun 15, 2026Amr Mohamed, Guokan Shang, Michalis VazirgiannisMasked Diffusion Models Diffusion Language Model Decoding
1MBZUAI · 2Ecole Polytechnique

Mar 26, 2026J/K move · Enter open · S save

DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease

Runsheng Bai, Chengyu Zhang, Yangdong Deng

CSAIL, MIT · IST, Nanjing University · School of Software, Tsinghua University; FuturististAI Lab, Shanghai Tsinghua International Innovation Center

Diffusion Model Sampling Diffusion Model Inference Acceleration Diffusion Sampling Diffusion-Based Image Generation Diffusion-Based Denoising

Open paper AskPDF arXiv

Diffusion models have achieved remarkable success in generating high-fidelity content but suffer from slow, iterative sampling, resulting in high latency that limits their use in interactive applications. We introduce DRiffusion, a parallel sampling framework that parallelizes diffusion inference through a draft-and-refine process. DRiffusion employs skip transitions to generate multiple draft states for future timesteps and computes their corresponding noises in parallel, which are then used in the standard denoising process to produce refined results. Theoretically, our method achieves an acceleration rate of

\tfrac{1}{n}

\tfrac{2}{n+1}

, depending on whether the conservative or aggressive mode is used, where

n

denotes the number of devices. Empirically, DRiffusion attains 1.4

\times

-3.7

\times

speedup across multiple diffusion models while incur minimal degradation in generation quality: on MS-COCO dataset, both FID and CLIP remain largely on par with those of the original model, while PickScore and HPSv2.1 show only minor average drops of 0.17 and 0.43, respectively. These results verify that DRiffusion delivers substantial acceleration and preserves perceptual quality.

Abstract

Diffusion Sampling

Explore similar work

DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease

Unlocking Few-Step Diffusion for Faithful Previews

LESS Is More: Mutual-Stability Sampling for Diffusion Language Models