Fast and Scalable Analytical Diffusion

Xinyi Shang; Peng Sun; Jingyu Lin; Zhiqiang Shen

cs.LGFeb 18, 2026

Fast and Scalable Analytical Diffusion

Authors: Xinyi Shang, Peng Sun, Jingyu Lin, Zhiqiang Shen

Organizations: University College London · Mohamed bin Zayed University of Artificial Intelligence · Zhejiang University · Westlake University ·

Abstract

Analytical diffusion models offer a mathematically transparent path to generative modeling by formulating the denoising score as an empirical-Bayes posterior mean. However, this interpretability comes at a prohibitive cost: the standard formulation necessitates a full-dataset scan at every timestep, scaling linearly with dataset size. In this work, we present the first systematic study addressing this scalability bottleneck. We challenge the prevailing assumption that the entire training data is necessary, uncovering the phenomenon of Posterior Progressive Concentration: the effective golden support of the denoising score is not static but shrinks asymptotically from the global manifold to a local neighborhood as the signal-to-noise ratio increases. Capitalizing on this, we propose Dynamic Time-Aware Golden Subset Diffusion (GoldDiff), a training-free framework that decouples inference complexity from dataset size. Instead of static retrieval, GoldDiff uses a coarse-to-fine mechanism to dynamically pinpoint the ''Golden Subset'' for inference. Theoretically, we derive rigorous bounds guaranteeing that our sparse approximation converges to the exact score. Empirically, GoldDiff achieves a $\bf 71 \times$ speedup on AFHQ while matching or achieving even better performance than full-scan baselines. Most notably, we demonstrate the first successful scaling of analytical diffusion to ImageNet-1K, unlocking a scalable, training-free paradigm for large-scale generative modeling.

Figures & tables

Figure 1: The Phenomenon of Posterior Progressive Concentration. $\bigstar$ denotes the initial noise, and the target distribution is the Moons data ( Pedregosa et al., 2011 ) . As the diffusion process reverses from pure noise to data (Left to Right), the golden support of the posterior distribution dynamically shrinks from the global manifold to a localized neighborhood.

Figure 2: Impact of Biased Weight Estimation. Due to biased weight estimation, PCA ( Lukoianov et al., 2025 ) produces inherently over-smoothed outputs even with sufficient denoising steps.

Panel of Figure 3 — Figure 3: Analysis of the SOTA method PCA ( Lukoianov et al., 2025 ) for Subset Selection. (a) Evolution of the weight distribution during the denoising process. (b)–(c) Sensitivity analysis: (b) Performance evaluation of the analytical denoiser across varying random subset sizes ( $N_{\text{sub}}\in\{10,100,1000,5000\}$ ) compared to the full CIFAR-10 dataset. (c) Visualization of intermediate generation outputs.

Method	Optimal	Wiener Filter	Kamb	PCA	GoldDiff (Ours)
Complexity	$\mathcal{O}(ND)$	$\mathcal{O}(D^{2})$	$\mathcal{O}(Np_{t}D^{2})$	$\mathcal{O}(Np_{t}D)$	$\mathbf{\mathcal{O}(Nd+m_{t}p_{t}D)}$

Table 1: Algorithmic Complexity Comparison.

D

: flattened image dimension,

N

: dataset size,

p_{t}

: patch size at step

t

Figure 4: Qualitative Comparison. We compare our GoldDiff (5th row) against four baselines: Optimal (1st row), Wiener filter (2nd row), Kamb ( Kamb and Ganguli, 2024 ) (3rd row), and the PCA model (4th row). All images are generated from the same initial noise using 10 steps of DDIM ( Song et al., 2020a ) . The last row displays reference samples generated by a trained U-Net ( Ho et al., 2020 ) .

Dataset	CIFAR-10				CelebA-HQ				AFHQ
Metric	Efficacy		Efficiency		Efficacy		Efficiency		Efficacy		Efficiency
Method	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	Memory	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	Memory	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	Memory
Optimal ( De Bortoli, 2022 )	0.030	-0.401	2.353	0.020	0.021	0.010	1.244	0.055	0.039	-0.713	0.642	0.055
Wiener ( Wiener, 1949 )	0.008	0.553	0.011	0.290	0.012	0.781	0.153	4.509	0.010	0.687	0.156	4.509
Kamb ( Kamb and Ganguli, 2024 )	0.011	0.411	5.980	1.889	0.011	0.720	37.158	7.547	0.034	0.576	15.094	7.5471
PCA ( Lukoianov et al., 2025 )	0.008	0.670	2.802	0.835	0.009	0.802	6.040	5.460	0.008	0.703	24.896	4.743

Table 2: Quantitative Comparison of Analytical Denoisers. Here, Time denotes the sampling time cost (s) per step, and Memory denotes the peak memory (GB) usage during sampling. All metrics are averaged over 128 samples. The best and second-best results are highlighted in bold and underlined , respectively. Efficacy gain (%) denotes the improvement relative to the second-best performance.

Total Steps	Method	Unconditional			Conditional
Total Steps	Method	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time
10	PCA	0.045	0.412	110.798	0.033	0.467	0.451
	PCA (Unbaised)	0.042	0.435	110.986	0.032	0.473	0.453
	GoldDiff	0.039	0.458	2.607	0.031	0.490	0.100
100	PCA	0.032	0.436	110.798	0.025	0.521	0.451
	PCA (Unbaised)	0.036	0.425	110.986	0.035	0.411	0.453

Table 3: Quantitative Comparison on ImageNet-1K. We evaluate performance in both unconditional and conditional settings. All metrics are averaged over 128 samples, with the best results highlighted in bold . “Total Steps” denotes the total number of denoising steps, and we report results for 10 and 100 steps.

Figure 5: Qualitative Comparison of conditional Denoising on ImageNet-1K. We visualize samples generated by our method compared to two PCA-based baselines: Original PCA ( Lukoianov et al., 2025 ) and Unbiased PCA (PCA-U) for the class “Tench”.

		CIFAR-10		AFHQ
		MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )
EDM-VP	Optimal	0.060	-0.238	0.053	0.028
	Wiener	0.026	0.472	0.033	0.448
	Kamb	0.031	0.350	0.034	0.461
	PCA	0.023	0.594	0.029	0.524
	GoldDiff	0.0195	0.654	0.024	0.588

Table 4: Validation on Diverse Neural Denoisers. All metrics are averaged over 128 samples.

	Celeba-HQ			AFHQ
Method	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time
Optimal	0.021	0.010	1.244	0.039	-0.713	0.642
+ GoldDiff	0.014	0.134	0.338	0.035	-0.270	0.153
Kamb	0.011	0.720	37.158	0.034	0.576	15.094
+ GoldDiff	0.009	0.729	1.336	0.031	0.586	1.434

Table 5: Orthogonality to Existing Analytical Denoisers. All metrics are averaged over 128 samples.

Panel of Figure 6 — Figure 6: Sensitivity Analysis of the coarse candidate set size and golden subset size across multiple datasets. Dashed lines mean the baseline PCA ( Lukoianov et al., 2025 ) .

		Celaba-HQ		AFHQ
		MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )
GoldDiff	+ WSS	0.009	0.818	0.008	0.715
GoldDiff	+SS	0.008	0.836	0.007	0.731

Table 6: Biased (WSS) vs. unbiased weight (SS) estimation. All metrics are averaged over 128 samples.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	MNIST				Fashion-MNIST
Metric	Efficacy		Efficiency		Efficacy		Efficiency
Method	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	Mem.	MSE ( $\downarrow$ )	$r^{2}$ ( $\uparrow$ )	Time	Mem.
Optimal ( De Bortoli, 2022 )	0.047	0.435	2.806	0.013	0.033	0.514	2.608	0.013
Wiener ( Wiener, 1949 )	0.034	0.594	0.001	0.026	0.031	0.589	0.001	0.026
Kamb ( Kamb and Ganguli, 2024 )	0.043	0.489	4.115	2.188	0.051	0.294	3.903	2.188
PCA ( Lukoianov et al., 2025 )	0.027	0.684	2.798	0.204	0.015	0.795	2.791	0.204

Appendix

Table 7: Quantitative Comparison of Analytical Denoisers on MNIST and Fashion-MNIST. Time (s) and Memory (GB) are averaged over 128 samples. Best and second-best results are bolded and underlined , respectively.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,169 papers

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement
May 19, 2026Taegu Kang, Jaesik Yoon, Sungjin AhnInference-Time Scaling Diffusion Models
KAIST · KAIST & SAP · KAIST & NYU
Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning
Jul 23, 2026Rogerio Guimaraes, Pietro PeronaDiffusion Model Sampling Diffusion Sampling
California Institute of Technology, Pasadena CA 91125, USA
From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
Jul 25, 2026Jinshu Huang, Yiming Jiang, Chunlin WuDiffusion Model Sampling Score-Based Generative Modeling
School of Mathematical Sciences, Nankai University, Tianjin, China

May 19, 2026J/K move · Enter open · S save

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement

Taegu Kang, Jaesik Yoon, Sungjin Ahn

KAIST · KAIST & SAP · KAIST & NYU

Inference-Time Scaling Diffusion Models

Open paper AskPDF arXiv

Inference-time scaling has emerged as a major approach for improving reasoning capabilities, and has been increasingly applied to diffusion models. However, existing inference-time scaling methods for diffusion models typically rely on external verifiers or reward models to rank and select samples, limiting their scalability to settings where such evaluators are available and reliable. Moreover, while recent diffusion models perform sequential inference with region-wise, mixed-noise conditioning, inference-time scaling tailored to this setting remains relatively underexplored. We propose Iterative Partial Refinement (IPR), an inference-time scaling method for sequential diffusion that requires no external verifier. Starting from an already-generated sample, IPR re-noises a subset of regions and regenerates them conditioned on the remaining regions, enabling the model to revise earlier decisions under a richer context than was available during the initial generation. This iterative partial refinement produces more globally consistent samples without external verification. On reasoning tasks requiring global constraint satisfaction, IPR consistently improves performance: on MNIST Sudoku, the valid solution rate increases from 55.8% to 75.0%. These results show that iterative partial refinement alone can serve as an effective inference-time scaling strategy for diffusion models in sequential, mixed-noise settings. Code is available at: https://github.com/ahn-ml/IPR

Abstract

Diffusion Models

Explore similar work

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement

Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning

From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models