Diffusion probabilistic models (DPMs) are widely adopted for their outstanding generative fidelity, yet their sampling is computationally demanding. Polynomial-based multistep samplers mitigate this cost by accelerating inference; however, despite their theoretical accuracy guarantees, they generate the sampling trajectory according to a predefined scheme, providing no flexibility for further optimization. To address this limitation, we propose RBF-Solver, a multistep diffusion sampler that interpolates model evaluations with Gaussian radial basis functions (RBFs). By leveraging learnable shape parameters in Gaussian RBFs, RBF-Solver explicitly follows optimal sampling trajectories. At first order, it reduces to the Euler method (DDIM). At second order or higher, as the shape parameters approach infinity, RBF-Solver converges to the Adams method, ensuring its compatibility with existing samplers. Owing to the locality of Gaussian RBFs, RBF-Solver maintains high image fidelity even at fourth order or higher, where previous samplers deteriorate. For unconditional generation, RBF-Solver consistently outperforms polynomial-based samplers in the high-NFE regime (NFE >= 15). On CIFAR-10 with the Score-SDE model, it achieves an FID of 2.87 with 15 function evaluations and further improves to 2.48 with 40 function evaluations. For conditional ImageNet 256 x 256 generation with the Guided Diffusion model at a guidance scale 8.0, substantial gains are achieved in the low-NFE range (5-10), yielding a 16.12-33.73% reduction in FID relative to polynomial-based samplers.
Figures & tables
Figure 1: (a) During sampling, the three model evaluations at time points t3 , t2 , and t1 are interpolated using both radial basis functions (RBFs) and a constant basis, each basis being multiplied by its weight. Only the values of the central green pixels are plotted; the points used for interpolation are shown in black, while unused future and past points are shown in gray. (b) For second- and third-order schemes, the curves show how the interpolation coefficients vary with the shape parameter γ . As logγ→−∞ , all coefficients converge to the same value, whereas as logγ→∞ , they approach the coefficients of the Adams method.
Figure 2
Figure 2: Unconditional sampling results. FID ↓ on 50k samples.
Table 1: From Adams method to RBF-Solver. Sample quality measured by FID ↓ on ImageNet 64 × 64 (Improved-Diffusion), evaluated on 50k samples.
NFE
5
10
15
20
25
30
35
40
Shared- γ
116.37
26.07
21.02
19.46
18.95
18.58
18.40
18.28
Split- γ / Indep.
117.35
26.86
20.91
19.33
18.73
18.32
18.17
18.08
Split- γ / Joint
116.55
26.64
20.89
19.30
18.72
18.31
18.17
18.07
Table 2: Shape parameter optimization. Sample quality measured by FID ↓ on ImageNet 64 × 64 (Improved-Diffusion), evaluated on 50k samples.
NFE = 10
NFE = 20
NFE = 30
NFE = 40
RBF-Solver
UniPC
RBF-Solver
UniPC
RBF-Solver
UniPC
RBF-Solver
UniPC
p=3
26.47
27.75
19.30
19.51
18.31
18.60
18.07
18.34
p=4
24.72
33.60
18.67
19.37
18.08
18.39
17.95
18.25
p=5
25.37
40.55
18.58
20.25
17.92
18.48
17.92
18.27
p=6
25.30
40.55
18.44
29.93
17.99
23.89
17.79
20.69
p=7
26.39
55.96
18.62
58.40
18.52
68.87
18.97
80.48
Table 3: High-order stability. Sample quality measured by FID ↓ on the ImageNet 64 × 64 (Improved-Diffusion), evaluated on 50k samples.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Evolution of key quantities across guidance scales 2.0, 4.0, 6.0, and 8.0. Variations in the predictor’s shape parameter modulate the magnitude of its zeroth-index coefficient, which in turn helps interpolate the rapidly changing trajectory of the model evaluations.
Figure 5: Optimized predictor- and corrector-side shape parameters and their corresponding coefficient-magnitude ratios (CMRs). Rows represent guidance scales 2.0, 4.0, 6.0, and 8.0 (top → bottom); columns represent NFE= 5, 10, 15. Shape parameters are plotted on a logarithmic scale (blue = predictor, yellow = corrector). CMRs are color-coded as follows: green (0-index), red (1-index), purple (2-index), and brown (3-index).
Series
NFE
5
6
8
10
12
15
20
25
30
35
40
Model Trajectory
27.37
11.39
5.97
4.68
4.76
3.54
2.91
2.73
2.65
2.61
2.61
Target Trajectory
26.65
13.05
5.32
4.13
3.84
3.02
2.66
2.56
2.51
2.50
2.49
Appendix
Table 4: Comparison of model and target trajectory. Sample quality is measured by FID ↓ on the CIFAR-10 32×32 dataset (Score-SDE), evaluated on 50k samples.
Series
NFE
5
6
8
10
12
15
20
25
30
35
40
[−1,1]
26.34
16.34
5.28
4.11
3.80
3.01
2.66
2.56
2.51
2.50
2.49
[−2,2]
26.65
13.05
5.32
4.13
3.84
3.02
2.66
2.56
2.51
2.50
2.49
[−3,3]
26.64
12.45
5.36
4.16
3.92
3.02
2.66
2.56
2.51
2.50
2.50
Appendix
Table 5: Shape parameter range ablation study. Sample quality is measured by FID ↓ on CIFAR-10 32 × 32 dataset (Score-SDE), evaluated on 50k samples.
Series
NFE
5
6
8
10
12
15
20
25
30
35
40
[−1,1]
116.69
70.58
35.68
26.57
22.37
20.89
19.30
18.73
18.33
18.18
18.09
[−2,2]
116.55
70.68
34.15
26.47
22.72
20.89
19.30
18.72
18.31
18.17
18.07
[−3,3]
116.75
70.60
35.13
26.64
22.36
20.88
19.28
18.73
18.30
18.15
18.07
Appendix
Table 6: Shape parameter range ablation study. Sample quality is measured by FID ↓ on ImageNet 64 × 64 dataset (Improved-Diffusion), evaluated on 50k samples.
NFE
CIFAR-10
ImageNet 64 × 64
ImageNet 128 × 128
ImageNet 256 × 256
5
1.916±0.032
4.846±0.006
4.356±0.046
7.432±0.024
6
2.361±0.032
6.002±0.026
5.346±0.031
9.101±0.025
8
3.333±0.028
8.333±0.019
7.384±0.041
12.491±0.024
10
4.207±0.029
10.692±0.012
9.461±0.031
15.889±0.027
12
5.205±0.031
12.968±0.009
11.465±0.065
19.275±0.033
15
6.524±0.033
16.504±0.009
14.566±0.067
24.349±0.041
Appendix
Table 7: Runtime per NFE for shape-parameter optimization. Each per-NFE value is the time required to optimize one batch of size M . The Total row sums all per-NFE runtimes, multiplies the result by K , and then adds the target-set runtime. All values are reported as mean ± std over 10 runs.
Dataset
Model
# Eval. samples
Section
Unconditional
CIFAR-10 32 × 32 [ 28 ]
Score-SDE [ 3 ]
50k
Appendix I.1.1
CIFAR-10 32 × 32 [ 28 ]
EDM [ 34 ]
50k
Appendix I.1.2
ImageNet 64 × 64 [ 40 ]
Improved-Diffusion [ 41 ]
50k
Appendix I.1.3
Conditional
ImageNet 128 × 128 [ 29 ]
Guided-Diffusion [ 4 ]
50k
Appendix I.2.1
Appendix
Table 9: Summary of experimental settings used in Additional Main Results
Model
NFE
5
6
8
10
12
15
20
25
30
35
40
DPM-Solver++ (3)
28.50
13.46
5.32
4.01
4.04
3.32
2.89
2.75
2.68
2.65
2.62
UniPC– B1(h) (3)
23.69
10.38
5.15
3.96
3.93
3.05
2.73
2.64
2.60
2.59
2.57
RBF-Solver (3)
26.65
13.05
5.32
4.13
3.84
3.02
2.66
2.56
2.51
2.50
2.49
RBF-Solver (4)
20.17
12.45
5.71
4.26
3.90
2.87
2.60
2.54
2.49
2.49
2.48
Appendix
Table 10: Sample quality measured by FID ↓ on CIFAR-10 32 × 32 dataset (Score-SDE), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Table 11: Qualitative comparison of different samplers on CIFAR-10 (Score-SDE). Each image is randomly sampled and solver order 3. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
Model
NFE
5
6
8
10
12
15
20
25
30
35
40
DPM-Solver++ (3)
24.54
11.87
4.36
2.91
2.45
2.17
2.05
2.02
2.01
2.00
2.00
UniPC- B1(h) (3)
23.75
12.74
4.22
3.01
2.41
2.08
2.01
2.00
2.00
2.00
2.00
UniPC- B2(h) (3)
23.52
11.11
3.86
2.85
2.37
2.07
2.01
2.00
1.99
1.99
1.99
RBF-Solver (3)
22.25
10.78
3.77
2.88
2.37
2.06
1.99
1.98
1.98
1.98
1.98
RBF-Solver (4)
14.91
7.67
4.90
3.71
2.50
2.05
2.00
1.99
1.99
1.99
1.98
Appendix
Table 12: Sample quality measured by FID ↓ on CIFAR-10 32 × 32 dataset (EDM), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Model
NFE
5
6
8
10
12
15
20
25
30
35
40
DPM-Solver++ (3)
110.23
66.00
38.48
28.95
24.59
21.62
19.94
19.36
18.91
18.70
18.57
UniPC- B1(h) (3)
109.81
65.06
35.11
27.75
24.30
21.15
19.51
18.98
18.60
18.42
18.34
RBF-Solver (3)
116.55
70.68
34.15
26.47
22.72
20.89
19.30
18.72
18.31
18.17
18.07
RBF-Solver (4)
126.26
76.92
31.71
24.72
21.70
19.61
18.67
18.30
18.08
17.99
17.95
Appendix
Table 13: Sample quality measured by FID ↓ on ImageNet 64 × 64 dataset (Improved-Diffusion), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Table 14: Qualitative comparison of different samplers on ImageNet 64 × 64. Each image is randomly sampled and solver order = 3. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
NFE
5
6
8
10
12
15
20
25
Guidance Scale = 2.0
DPM-Solver++ (2)
15.32
10.60
7.16
6.01
5.41
4.93
4.50
4.30
UniPC- B2(h) (2)
13.43
9.19
6.50
5.69
5.23
4.85
4.48
4.29
RBF-Solver (2)
12.83
8.80
6.34
5.61
5.22
4.85
4.49
4.28
RBF-Solver (3)
12.19
8.15
5.98
5.51
5.16
4.84
4.51
4.30
Appendix
Table 15: Sample quality measured by FID ↓ on ImageNet 128 × 128 dataset (Guided-Diffusion), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Table 16: Qualitative comparison of various samplers on ImageNet 128 × 128. Each image is randomly sampled with guidance scale 6.0 and solver order = 2. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
NFE
5
6
8
10
12
15
20
25
30
Guidance Scale = 2.0
DPM-Solver++ (2)
16.27
12.59
9.62
8.54
8.11
7.68
7.44
7.33
7.27
UniPC- B2(h) (2)
15.00
11.53
9.05
8.20
7.83
7.62
7.36
7.23
7.23
RBF-Solver (2)
15.05
11.45
9.11
8.28
7.87
7.71
7.43
7.31
7.28
RBF-Solver (3)
14.77
11.11
9.00
8.40
8.05
7.78
7.45
7.31
7.28
Appendix
Table 17: Sample quality measured by FID ↓ on ImageNet 256 × 256 dataset (Guided-Diffusion), evaluated on 10k samples. Numbers in parentheses indicate the solver order.
Table 18: Qualitative comparison of different samplers on ImageNet 256 × 256. Each image is randomly sampled with guidance scale set to 8.0 and solver order = 2. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
Table 20: Qualitative comparison of different samplers on four shared generation scripts extracted from MS-COCO 2014. Each cell displays a 512×512 image generated using Stable Diffusion v1.4, with classifier-free guidance scale set to 7.5 and NFE = 5.
Dataset
Model
Compared Samplers
Section
Unconditional
CIFAR-10 32 × 32 [ 28 ]
Score-SDE [ 3 ]
DPM-Solver-v3 [ 39 ] , DC-Solver [ 38 ]
J.2.1
CIFAR-10 32 × 32 [ 28 ]
EDM [ 34 ]
DPM-Solver-v3 [ 39 ] , DC-Solver [ 38 ]
J.2.2
CIFAR-10 32 × 32 [ 28 ]
EDM [ 34 ]
AMED-Solver, AMED-Plugin [ 37 ]
J.2.3
FFHQ 64 × 64 [ 47 ]
EDM [ 34 ]
AMED-Solver, AMED-Plugin [ 37 ]
J.2.4
Conditional
Appendix
Table 21: Summary of experimental settings for auxiliary-tuning samplers.
Method
Parameter Type
Param. Count
Dataset Size
Offline Tuning Time *
Additional Network
DPM-Solver-v3 [ 39 ]
Empirical Model Stats.
1.1M–295M
1k–4k
< 11 h / 8 GPUs
✗
AMED-Solver [ 37 ]
Neural-Net Weights
≈ 9k
10k
< 3 h / 1 GPU
✓
DC-Solver [ 38 ]
Compensation Ratios
NFE
10
< 5 min / 1 GPU
✗
RBF-Solver (Ours)
Shape Parameters
2(NFE−1)
128
1 min - 1 h / 1 GPU **
✗
Appendix
Table 22: Comparison of solvers that employ auxiliary-parameter tuning. Param. Count is the total number of auxiliary-tuning parameters stored after the offline step; Dataset Size is the amount of data each sampler uses for tuning; Additional Network indicates whether a separate network is trained.
Model
NFE
5
6
8
10
12
15
20
25
30
35
40
DPM-Solver-v3 (3)
12.76
7.39
3.93
3.39
3.23
2.89
2.69
2.62
2.58
2.58
2.57
DC-Solver (3)
48.27
22.65
6.46
4.64
5.51
4.39
3.86
2.68
2.59
2.52
2.48
RBF-Solver (3)
26.65
13.05
5.32
4.13
3.84
3.02
2.66
2.56
2.51
2.50
2.49
RBF-Solver (4)
20.17
12.45
5.71
4.26
3.90
2.87
2.60
2.54
2.49
2.49
2.48
Appendix
Table 23: Sample quality measured by FID ↓ on CIFAR-10 32 × 32 dataset (Score-SDE), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Model
NFE
5
6
8
10
12
15
20
25
30
35
40
DPM-Solver-v3 (3)
12.19
8.56
3.50
2.51
2.23
2.10
2.02
2.00
2.00
1.99
1.99
DC-Solver (3)
29.69
12.52
5.21
2.88
3.15
2.37
2.07
2.16
2.15
2.11
2.17
RBF-Solver (3)
22.25
10.78
3.77
2.88
2.37
2.06
1.99
1.98
1.98
1.98
1.98
RBF-Solver (4)
14.91
7.67
4.88
3.71
2.50
2.05
2.00
1.99
1.99
1.98
1.98
Appendix
Table 24: Sample quality measured by FID ↓ on CIFAR-10 32 × 32 dataset (EDM), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Table 25: Qualitative comparison of different samplers on CIFAR-10 (EDM). Each image is randomly sampled and solver order = 3. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
Model
NFE
5
7
9
11
15
21
25
31
35
41
AMED-Solver (2)
7.28
4.20
3.78
4.00
3.40
3.35
3.26
3.06
2.95
2.62
AMED-Plugin (4)
6.77
3.80
2.66
2.31
2.00
1.97
1.99
2.07
2.01
1.99
RBF-Solver (3)
22.63
5.94
3.43
2.68
2.08
2.00
1.99
1.98
1.98
1.97
RBF-Solver (4)
15.01
5.29
5.06
3.01
2.09
2.01
1.99
1.98
1.98
1.97
Appendix
Table 26: Sample quality measured by FID ↓ on CIFAR-10 32 × 32 (EDM), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
Model
NFE
5
7
9
11
15
21
25
31
35
41
AMED-Solver (2)
14.80
8.82
6.30
5.29
4.40
3.85
3.53
3.33
3.21
2.93
AMED-Plugin (4)
12.48
6.64
4.24
3.70
2.93
2.61
2.57
2.53
2.50
2.55
RBF-Solver (3)
19.58
7.07
4.61
3.49
2.70
2.42
2.37
2.34
2.34
2.34
RBF-Solver (4)
15.58
7.93
8.71
4.19
2.60
2.34
2.30
2.29
2.30
2.31
Appendix
Table 27: Sample quality measured by FID ↓ on FFHQ 64 × 64 (EDM), evaluated on 50k samples. Numbers in parentheses indicate the solver order.
NFE
5
6
8
10
12
15
20
25
Guidance Scale = 2.0
DPM-Solver-v3 (2)
14.75
10.99
8.79
8.05
7.74
7.61
7.35
7.23
DC-Solver (3)
15.78
11.60
9.29
8.19
7.77
7.61
7.46
7.47
RBF-Solver (2)
15.05
11.45
9.11
8.28
7.87
7.71
7.43
7.31
RBF-Solver (3)
14.77
11.11
9.00
8.40
8.05
7.78
7.45
7.31
Appendix
Table 28: Sample quality measured by FID ↓ on ImageNet 256 × 256 (Guided-Diffusion), evaluated on 10k samples. Numbers in parentheses indicate the solver order.
Table 29: Qualitative comparison of different samplers on ImageNet 256 × 256. Each image is randomly sampled with guidance scale 8.0. DPM-Solver-v3 and RBF-Solver use order 2; DC-Solver uses order 3. Columns correspond to samplers; rows indicate NFE values of 5, 10, and 15.
Diffusion-based generative models have achieved remarkable performance across various domains, yet their practical deployment is often limited by high sampling costs. While prior work focuses on training objectives or individual solvers, the broader sampling design problem, specifically solver selection and scheduling, remains largely governed by static heuristics. We propose SDM, a principled, training-free sampling framework that adapts both the numerical solver and the timestep schedule to the intrinsic properties of the diffusion trajectory. By analyzing the PF-ODE dynamics, we show that velocity variation is small in high-noise stages and increases near the data manifold, identifying intervals where solver order is most consequential. In parallel, we introduce an offline-calibrated adaptive scheduling method that explicitly controls the local Wasserstein discretization error and projects the calibrated trajectory to a prescribed NFE budget. We further extend the formulation to a mixed-transition Wasserstein error bound, providing a unified error-propagation view of adaptive scheduling and solver selection within the overall SDM framework. Across standard benchmarks, with extensions to modern ODE samplers, high-resolution synthesis, and text-to-image generation, SDM achieves improved sample quality compared to baseline methods, attaining an FID of 1.93 on CIFAR-10, 2.41 on FFHQ, and 1.98 on AFHQv2, with a reduced number of function evaluations compared to existing samplers. Our code is available at https://github.com/aiimaginglab/sdm.
Sangwoo Jo, Sungjoon Choi
Department of Artificial Intelligence, Korea University, Seoul, South Korea.
Diffusion models achieve state-of-the-art image quality. However, sampling is costly at inference time because it requires a large number of function evaluations (NFEs). To reduce NFEs, classical ODE numerical methods have been adopted. Yet, the choice of prediction type and integration domain leads to different sampling behaviors. To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms. It retains the standard predictor-corrector structure while preserving second-order local accuracy. These parameters are learned via a classification-based objective using a frozen pretrained classifier (e.g., MobileNet or CLIP). For ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image generation (SANA, PixArt-α), Dual-Solver improves FID and CLIP scores in the low-NFE regime (3≤ NFE ≤9) across backbones.
Diffusion models rely on numerical solvers requiring time-discretization, which has a large influence on the tradeoff between sampling cost and quality. However, the computational difficulty of the reverse process varies along the sampling trajectory and across data distributions, making the choice of discretization important. We adapt proportional-integral (PI) step-size control to diffusion, using our diffusion noise-normalised error estimator. Unlike existing adaptive methods in diffusion that respond only to the current error, the PI solver also incorporates the previous error, yielding smoother step adaptation. We further show that these per-sample trajectories exhibit shared structure and can be aggregated into a fixed schedule that retains much of the benefit of adaptive sampling. We evaluate both approaches on natural-image and language datasets, in terms of quality, measured by FID at a matched number of neural network evaluations (NFE), comparing them with widely used stochastic solvers and schedules. For images, our fixed discretization outperforms the commonly used EDM schedule in terms of sample quality when used with the stochastic Heun sampler, and with the EDM-churn sampler at low NFE. Additionally, our PI adaptive solver obtains better FID than most stochastic and adaptive baselines, although it does not beat the EDM-churn sampler at low NFE. Moreover, we find our solver outperforms both the EDM and the entropy schedule on language diffusion at low-to-medium NFE in terms of perplexity, with the drawback of lower token entropy. Lastly, we find that the benefit of per-sample adaptivity is problem-dependent. It is highly beneficial in 1D toy examples, while only marginal for image and language data, where the average schedule sometimes even outperforms the PI-adaptive solver. Code is available at https://github.com/ellakemperman/adaptive-second-order-diffusion-solvers