Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction

Soochul Park; Yeon Ju Lee

cs.LGMar 4, 2026

Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction

Authors: Soochul Park, Yeon Ju Lee

Organizations: SteAI · MODULABS · Korea University

Abstract

Diffusion models achieve state-of-the-art image quality. However, sampling is costly at inference time because it requires a large number of function evaluations (NFEs). To reduce NFEs, classical ODE numerical methods have been adopted. Yet, the choice of prediction type and integration domain leads to different sampling behaviors. To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms. It retains the standard predictor-corrector structure while preserving second-order local accuracy. These parameters are learned via a classification-based objective using a frozen pretrained classifier (e.g., MobileNet or CLIP). For ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image generation (SANA, PixArt- $α$ ), Dual-Solver improves FID and CLIP scores in the low-NFE regime ( $3 \le$ NFE $\le 9$ ) across backbones.

Figures & tables

Panel of Figure 1 — Figure 1: Sampling results. SANA ( Xie et al., 2024 ) , NFE=3, CFG=4.5. See Fig. 18 for further results.

	Differential form	Integral form on $[t_{i},t_{i+1}]$
Noise	$\displaystyle\frac{d{{\bm{x}}}_{t}}{dt}=\tfrac{d\log\alpha_{t}}{dt}\,{\bm{x}}_{t}-\sigma_{t}\,\tfrac{d\lambda_{t}}{dt}\,\bm{\epsilon}_{\theta}({\bm{x}}_{t},t)$	$\displaystyle{\bm{x}}_{t_{i+1}}=\tfrac{\alpha_{t_{i+1}}}{\alpha_{t_{i}}}\,{\bm{x}}_{t_{i}}-\alpha_{t_{i+1}}\!\int_{t_{i}}^{t_{i+1}}\tfrac{\sigma_{t}}{\alpha_{t}}\,\tfrac{d\lambda_{t}}{dt}\,\bm{\epsilon}_{\theta}({\bm{x}}_{t},t)\,dt$
Velocity	$\displaystyle\frac{d{{\bm{x}}}_{t}}{dt}=\tfrac{d\alpha_{t}}{dt}\,{\bm{x}}_{\theta}({\bm{x}}_{t},t)+\tfrac{d\sigma_{t}}{dt}\,\bm{\epsilon}_{\theta}({\bm{x}}_{t},t)$	$\displaystyle{\bm{x}}_{t_{i+1}}={\bm{x}}_{t_{i}}+\!\int_{t_{i}}^{t_{i+1}}\!\!\big[\tfrac{d\alpha_{t}}{dt}\,{\bm{x}}_{\theta}({\bm{x}}_{t},t)+\tfrac{d\sigma_{t}}{dt}\,\bm{\epsilon}_{\theta}({\bm{x}}_{t},t)\big]\,dt$
Data	$\displaystyle\frac{d{{\bm{x}}}_{t}}{dt}=\tfrac{d\log\sigma_{t}}{dt}\,{\bm{x}}_{t}+\alpha_{t}\,\tfrac{d\lambda_{t}}{dt}\,{\bm{x}}_{\theta}({\bm{x}}_{t},t)$	$\displaystyle{\bm{x}}_{t_{i+1}}=\tfrac{\sigma_{t_{i+1}}}{\sigma_{t_{i}}}\,{\bm{x}}_{t_{i}}+\sigma_{t_{i+1}}\!\int_{t_{i}}^{t_{i+1}}\tfrac{\alpha_{t}}{\sigma_{t}}\,\tfrac{d\lambda_{t}}{dt}\,{\bm{x}}_{\theta}({\bm{x}}_{t},t)\,dt$

Table 1: Differential and integral forms for noise, velocity, and data predictions. (

\alpha_{t}

: signal rate,

\sigma_{t}

: noise rate,

\lambda_{t}:=\log(\alpha_{t}/\sigma_{t})

)

Panel of Figure 2 — Figure 2: Euler updates for noise, velocity, and data predictions.

Figure 4

Figure 3: Learned parameters. Values of $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ across sampling steps, learned on DiT ( Peebles and Xie, 2023 ) at NFE = 5. See Figs. 11 , 12 , 13 , and 14 for further results.

Panel of Figure 5 — Figure 5: Main quantitative results. FID and CLIP score; evaluated on 50k (DiT/GM - DiT) and 30k (SANA/PixArt- $\alpha$ ) samples; CFG: DiT=1.5, GM - DiT=1.4, SANA=4.5, PixArt- $\alpha$ =3.5.

	Cross-entropy loss (↓)
NFE	3	5	7	9
p1	0.667	0.225	0.183	0.175
p1c2	0.574	0.197	0.178	0.173
p2	1.023	0.253	0.222	0.181
p2c2	5.009	0.317	0.203	0.191

Table 2: Ablation of predictor–corrector configurations on DiT.

	Cross-entropy loss (↓)
NFE	3	5	7	9
$\gamma=1$ fixed	0.816	0.223	0.182	0.176
$\gamma=0$ fixed	0.600	0.202	0.183	0.180
$\gamma=-1$ fixed	7.871	7.676	0.238	0.196
$\tau=1$ fixed	0.601	0.217	0.175	0.178
$\kappa=0$ fixed	0.944	0.256	0.202	0.190

Table 3: Ablation of the

(\gamma,\tau,\kappa)

parameter settings on DiT.

	FID (↓)
NFE	3	5	7	9
Type 1	23.38	4.09	3.56	3.64
Type 2	24.91	3.52	2.75	2.67

Table 4: Ablation of Domain Change Functions on DiT.

Figure 6: Classifier Accuracy vs. FID on GM-DiT.

Regression-based learning (Sec. 5.1 )
	FID (↓)
NFE	3	5	7	9
Sample	107.13	11.71	4.60	2.99
Trajectory	100.89	11.59	3.66	2.84
Feature (AlexNet)	47.75	7.24	3.42	2.91
Feature (VGG)	41.58	5.48	3.23	2.88

Table 5: Comparison of parameter learning methods on DiT.

Figure 7: FID results for parameter interpolation across NFEs with DiT.

Panel of Figure 8 — Figure 8: Sampling results. PixArt- $\alpha$ ( Chen et al., 2023 ) , NFE=5, CFG=3.5. See Fig. 19 for further results.

Panel of Figure 9 — Figure 9: Sampling results. GM-DiT ( Chen et al., 2025 ) , NFE=3, CFG=1.4. See Fig. 17 for further results.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Method	3	4	5	6	7	8	9
DDIM ( Song et al., 2021a )	89.33	56.33	32.91	20.06	13.64	9.55	7.42
DPM-Solver++ ( Lu et al., 2022a )	88.46	47.64	22.19	11.49	7.06	5.19	4.43
BNS-Solver ( Shaul et al., 2024 )	103.26	38.20	14.53	6.37	4.25	3.37	3.05
DS-Solver ( Wang et al., 2025 )	67.31	17.31	7.66	5.46	3.79	3.08	3.02
Dual-Solver (Ours)	24.91	6.05	3.52	3.13	2.75	2.60	2.67

Appendix

Table 6: DiT ( Peebles and Xie, 2023 ) : FID (↓) vs. NFE. ImageNet generation, 50k samples.

Method	3	4	5	6	7	8	9
DDIM ( Song et al., 2021a )	70.15	35.98	19.70	12.55	9.03	7.01	5.78
DPM-Solver++ ( Lu et al., 2022a )	63.24	19.53	7.85	4.74	3.65	3.15	2.89
BNS-Solver ( Shaul et al., 2024 )	57.88	26.64	10.31	5.02	3.43	2.66	2.44
DS-Solver ( Wang et al., 2025 )	34.15	11.60	5.64	3.49	2.70	2.48	2.41
Dual-Solver-R (Ours)	45.53	14.43	7.49	3.75	2.77	2.44	2.32
Dual-Solver-C (Ours)	6.81	3.76	3.09	2.97	2.77	2.70	2.60

Appendix

Table 7: GM-DiT Chen et al. (2025) : FID (↓) vs. NFE. ImageNet generation, 50k samples.

Method	3	4	5	6	7	8	9
FID (↓)
DDIM ( Song et al., 2021a )	45.05	27.72	23.93	23.06	22.99	22.96	22.97
DPM-Solver++ ( Lu et al., 2022a )	45.33	26.12	22.56	22.48	22.79	22.90	23.11
BNS-Solver ( Shaul et al., 2024 )	48.16	26.37	21.04	20.66	20.79	21.13	21.64
DS-Solver ( Wang et al., 2025 )	48.65	29.15	21.66	20.65	21.43	21.80	22.27
Dual-Solver (Ours)	21.79	19.40	18.81	18.52	19.57	19.43	19.77

Appendix

Table 9: SANA ( Xie et al., 2024 ) : FID (↓) and CLIP score (RN101, ↑) vs. NFE. Text-to-image on MSCOCO 2014 ( Lin et al., 2014 ) with 30k samples.

Weights 3 3 3 https://docs.pytorch.org/vision/main/models.html	Top-5 Acc. (%)	FID@3	FID@9	Precision@3	Precision@9	Recall@3	Recall@9	GFLOPS
ViT_H_14_Weights.IMAGENET1K_SWAG_E2E_V1	98.694	10.71	6.72	0.8376	0.8960	0.7420	0.7311	1016.72
RegNet_Y_128GF_Weights.IMAGENET1K_SWAG_LINEAR_V1	97.844	12.99	5.61	0.8230	0.8996	0.7327	0.7336	127.52
RegNet_Y_16GF_Weights.IMAGENET1K_SWAG_LINEAR_V1	97.244	9.97	5.32	0.8573	0.9007	0.7392	0.7467	15.91
ConvNeXt_Base_Weights.IMAGENET1K_V1	96.870	10.28	5.78	0.8529	0.9040	0.7359	0.7327	15.36
EfficientNet_B5_Weights.IMAGENET1K_V1	96.628	11.53	6.01	0.8379	0.9048	0.7393	0.7310	10.27
RegNet_Y_16GF_Weights.IMAGENET1K_V2	96.328	9.63	5.21	0.8559	0.9061	0.7396	0.7455	15.91

Appendix

Table 10: ImageNet per-classifier metrics. Top-5 accuracy, FID, precision/recall at NFE=3 and 9, and GFLOPs; FID is measured on 10k samples after training Dual-Solver for each classifier.

Panel of Figure 10 — Figure 10: Accuracy vs. precision/recall. We select 20 TorchVision classifiers sorted by accuracy, learn the Dual-Solver for each, and report precision/recall at NFE=3 and 9 on 10k samples.

Weights	MSCOCO Acc. (%)	FID@3	FID@6	GFLOPs
ViT-H-14-378-quickgelu, dfn5b	63.76	23.98	23.28	1054.05
coca_ViT-L-14, mscoco_finetuned_laion2b_s13b_b90k	60.28	23.09	21.22	214.52
EVA02-E-14, laion2b_s4b_b115k	58.92	22.41	21.12	1007.93
convnext_xxlarge, laion2b_s34b_b82k_augreg	58.34	21.05	20.86	800.88
ViT-B-16-SigLIP-256, webli	57.24	21.03	19.87	57.84
EVA02-L-14-336, merged2b_s6b_b61k	56.05	23.15	23.28	167.50

Appendix

Table 11: OpenCLIP per-classifier metrics. MSCOCO accuracy, FID at NFE = 3 and 6, and GFLOPs; FID is measured on 30k samples after learning Dual-Solver for each classifier.

Panel of Figure 11 — Figure 11: Learned parameters. $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ for DiT ( Peebles and Xie, 2023 ) .

Panel of Figure 12 — Figure 12: Learned parameters. $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ for GM-DiT ( Chen et al., 2025 ) .

Panel of Figure 13 — Figure 13: Learned parameters. $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ for SANA ( Xie et al., 2024 ) .

Panel of Figure 14 — Figure 14: Learned parameters. $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ for PixArt- $\alpha$ ( Chen et al., 2023 ) .

Panel of Figure 15 — Figure 15: Interpolated parameters. $\{\gamma,\tau_{u},\tau_{v},\kappa_{u},\kappa_{v}\}$ for DiT ( Peebles and Xie, 2023 ) .

Panel of Figure 16 — Figure 16: Additional sampling results. DiT-XL/2 256 $\times$ 256 (NFE=4, CFG=1.5)

Panel of Figure 17 — Figure 17: Additional sampling results. GM-DiT 256 $\times$ 256 (NFE=3, CFG=1.4)

Panel of Figure 18 — Figure 18: Additional sampling results. SANA ( Xie et al., 2024 ) , NFE=3, CFG=4.5.

Panel of Figure 19 — Figure 19: Additional sampling results. PixArt- $\alpha$ ( Chen et al., 2023 ) , NFE=5, CFG=3.5.

Explore similar work

Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization
Aug 18, 2026Travis Zhang, Christian Belardi, Justin Lovelace +4Diffusion Timestep Scheduling Few-Step Diffusion Sampling
Cornell University
The Diffusion Duality, Chapter II: $Ψ$ -Samplers
Feb 24, 2026Justin Deschenaux, Caglar Gulcehre, Subham Sekhar SahooDiffusion Models Diffusion Sampling
EPFL, Lausanne, Switzerland · Microsoft AI · Cornell Tech, NY
RBF-Solver: A Multistep Sampler for Diffusion Probabilistic Models via Radial Basis Functions
Mar 5, 2026Soochul Park, Yeon Ju Lee, SeongJin Yoon +3Diffusion Sampling Diffusion Model Inference Acceleration
SteAI · MODULABS · Korea University +3

Aug 18, 2026J/K move · Enter open · S save

Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization

Travis Zhang, Christian Belardi, Justin Lovelace, Jin Peng Zhou+3

Cornell University

Diffusion Timestep Scheduling Few-Step Diffusion Sampling Diffusion Model Inference Acceleration T2I Diffusion Models

Open paper AskPDF arXiv

Sampling from a diffusion model typically requires many forward passes through a large neural network, making generation computationally expensive. While much work has focused on efficient solvers and samplers, comparatively little attention has been paid to selecting the sampling timesteps themselves. A recent line of work optimizes theoretically derived surrogates for sample quality rather than the quality metric itself. We propose Optimizing Your Sampling (OYS), which instead treats timestep selection as a black-box optimization problem, optimizing the target metric directly with Bayesian optimization. OYS outperforms both the default schedules and those of Align Your Steps on text-to-image generation, and improves over the default schedules on inpainting and other image tasks, in both quantitative and human evaluations. OYS requires no additional training, is applicable even to distilled models, and improves both simple and sophisticated samplers such as Euler and DPM-Solver++. A 5-step OYS schedule retains 89%-94% of the quality of a 50-step schedule while reducing inference cost by 10x.

Abstract

Explore similar work

Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization

The Diffusion Duality, Chapter II: ΨΨΨ-Samplers

RBF-Solver: A Multistep Sampler for Diffusion Probabilistic Models via Radial Basis Functions

The Diffusion Duality, Chapter II: $Ψ$ -Samplers