ManifoldCache: Training-Free Diffusion Acceleration via Constraint Manifold Caching

Prashant Pandey; Devineni Sri Venkatraya Chowdary; Brejesh Lall

cs.AIOct 3, 2026

ManifoldCache: Training-Free Diffusion Acceleration via Constraint Manifold Caching

Authors: Prashant Pandey, Devineni Sri Venkatraya Chowdary, Brejesh Lall

Organizations: Indian Institute of Technology, Delhi · Indian School of Mines, Dhanbad

Abstract

Diffusion models for structured scientific generation must produce samples satisfying hard geometric constraints imposed by physics, chemistry, or biology, yet inference in these settings is prohibitively slow, demanding hundreds to thousands of neural-function evaluations per sample. We unify eight state-of-the-art models spanning medical volumetrics, molecular conformations, protein backbone design, crystal structure prediction, and multi-view 3D scenes under a single abstraction, Constraint-Manifold Diffusion Models (CMDMs), in which the target distribution is supported on a manifold defined by an externally specified constraint map. All existing acceleration families fail on this class: quantization exhausts memory on high-dimensional volumetric operators; pruning breaks constraint fidelity; fast ODE solvers allow trajectories to drift off the constraint manifold; and feature-caching heuristics are blind to constraint geometry, inducing mode confusion in the high-noise regime. We introduce ManifoldCache, the first training-free, data-free accelerator designed from first principles for CMDMs. The key insight is that the conditional score decomposes orthogonally into a normal component, which enforces constraint satisfaction, and a tangential component, which navigates within the manifold. Exploiting this structure, we prove that the noise-schedule midpoint is a sharp safe-caching boundary: caching before it incurs provably bounded error, while caching after it guarantees a strictly positive fraction of trajectories suffer mode confusion, a gap that persists up to the boundary. We further prove that deeper network blocks admit provably larger certified cache strides within the safe phase, as a consequence of the score decomposition propagating through block Jacobians. The resulting schedule requires no calibration data, along with zero training overhead.

Figures & tables

Figure 1: Quality–speed trade-off on Med-DDPM ( Dorjsembe et al., 2024 ) across quantization, pruning, solver, and caching baselines. ManifoldCache achieves 1 min 31 sec per sample (full inference: 2 min 45 sec); all baselines either degrade quality, slow down, or incur OOM.

Figure 2: Geometric picture behind ManifoldCache . (a) Near $\mathcal{M}_{c}$ , the score splits into a normal part $s_{t}^{\perp}$ , which pulls the sample toward $\mathcal{M}_{c}$ , and a tangential part $s_{t}^{\parallel}$ , which moves it within $\mathcal{M}_{c}$ toward one of the valid modes (here $\mu_{1}$ and $\mu_{2}$ , separated by $\delta_{c}$ ; § 3.2 ). (b) The threshold $T^{*}=\max\{t:\bar{\alpha}_{t}\geq\tfrac{1}{2}\}$ splits the schedule: for $t>T^{*}$ every block is recomputed, and caching is enabled only for $t\leq T^{*}$ (Theorem 1 ); the $\kappa_{t}$ curve is schematic. (c) For $t\leq T^{*}$ , the cache stride grows with block depth (Eq. ( 6 )): shallow blocks are recomputed often, and deep blocks are reused for longer.

Method	Time (sec) $\downarrow$	Mean scRMSD $\downarrow$	VRAM (MiB) $\downarrow$
Full Inference	90.47	1.93	15438
SmoothCache ( Liu et al., 2025c )	60.96	2.11	16430
FORA ( Selvaraju et al., 2024 )	70.62	2.31	15980
TeaCache ( Liu et al., 2025a )	61.10	8.07	17800
TaylorSeer ( Liu et al., 2025b )	61.08	9.11	16700
ManifoldCache $(0)\,(2,4)$	34.40	6.24	16192

Table 1: Proteina ( Geffner et al., 2025 ) (de novo

C_{\alpha}

backbone generation, flow-matching DiT trained on AFDB+PDB): runtime, mean self-consistency RMSD ( scRMSD ; ProteinMPNN

\to

ESMFold), and peak VRAM . ManifoldCache

(T^{*})\,(2,4)

has the lowest scRMSD among the accelerators (2.02 vs. 1.93 for full inference).

Method	Time (sec) $\downarrow$	COV-R (mean) $\uparrow$	COV-P (mean) $\uparrow$	VRAM (MiB) $\downarrow$
Full Inference	621	0.9577	0.9529	30706
TeaCache ( Liu et al., 2025a )	565	0.9473	0.9424	32491
AB-Cache ( Yu et al., 2025 )	501	0.9460	0.9430	31900
PFC (ABC-Forecasting) ( Sommer et al., 2025 )	561	0.9517	0.9428	32998
ManifoldCache $(T^{*})\,(2,4)$	410	0.9545	0.9513	32360

Table 2: DiT-MC ( Frank et al., 2026 ) (3D molecular conformer generation on GEOM-QM9, graph-conditioned DiT): runtime, mean recall ( COV-R ) and precision ( COV-P ) at a

0.75

Å threshold, and VRAM .

Method	Time (sec) $\downarrow$	AAR $\uparrow$	C-RMSD $\downarrow$	L-RMSD $\downarrow$	VRAM (MiB) $\downarrow$
Full Inference	687	0.2562	3.14	1.59	28072
SmoothCache ( Liu et al., 2025c )	578	0.1890	6.10	6.04	28126
DeepCache ( Ma et al., 2024 )	555	0.1622	4.98	3.10	28960
ManifoldCache $(T^{*})\,(2,4)$	490	0.2507	3.37	1.90	28148

Table 3: UniMoMo ( Kong et al., 2025 ) (peptide binder design via E(3)-equivariant latent diffusion): runtime, amino-acid recovery ( AAR , mean), complex ( C-RMSD , median,

C_{\alpha}

after target alignment), ligand ( L-RMSD , median, after peptide alignment), and VRAM .

Method	Match Rate $\uparrow$	RMSD $\downarrow$	Time (sec) $\downarrow$	VRAM (MiB) $\downarrow$
Full Inference	0.7880	0.0269	1240	1886
DeepCache ( Ma et al., 2024 )	0.7650	0.0305	825	1894
ManifoldCache $(0)\,(2,4)$	0.7100	0.0346	716	2014
ManifoldCache $(T^{*}/2)\,(2,4)$	0.7700	0.0303	732	2015
ManifoldCache $(T^{*})\,(4,2)$	0.7790	0.0344	802	2019
ManifoldCache $(T^{*})\,(2,4)$	0.7800	0.0273	805	2014

Table 4: DiffCSP++ ( Jiao et al., 2024 ) (space-group constrained crystal structure prediction on MP-20): Match Rate and RMSD , runtime, and VRAM . ManifoldCache

(T^{*})\,(2,4)

has the highest Match Rate and the lowest RMSD among the accelerators.

Method	Time per sample (sec) $\downarrow$	FID $\downarrow$	VRAM (MiB) $\downarrow$
Full Inference	165	1.21	22392
DeepCache ( Ma et al., 2024 )	143	20.14	22994
FasterDiffusion ( Li et al., 2024 )	123	13.37	25780
ManifoldCache $(0)\,(2,4)$	75	21.41	30525
ManifoldCache $(T^{*})\,(4,2)$	89	11.31	30531
ManifoldCache $(T^{*})\,(2,5)$	88	9.75	30550

Table 5: Med-DDPM ( Dorjsembe et al., 2024 ) (semantic 3D brain MRI synthesis on BraTS,

192^{3}

Conv3D U-Net): runtime per sample and 3D FID between generated and real volumes. ManifoldCache

(T^{*})\,(2,4)

1.8{\times}

faster than full inference and attains the lowest FID among all accelerators; Figure 3 shows samples.

Method	Time (sec) $\downarrow$	Score $\uparrow$	Consistency Score $\uparrow$	VRAM (MiB) $\downarrow$
Full Inference	1323.49	0.8050	0.2083	15584
FORA ( Selvaraju et al., 2024 )	0 867.61	0.5970	0.0799	17761
ToCa ( Zou et al., 2024 )	0 975.87	0.6118	0.1237	18450
dKV-Cache ( Ma et al., 2026 )	1111.50	0.4381	0.0780	16109
ManifoldCache $(T^{*})\,(2,5)$	0 814.00	0.6090	0.0948	18460
ManifoldCache $(T^{*})\,(2,4)$	0 829.00	0.6250	0.0980	18119

Table 6: DemoDiff ( Liu et al., 2026 ) (in-context molecular design, 0.7B discrete graph diffusion transformer with motif-token NPE): runtime, official task-averaged design Score , and ICL Consistency Score . ManifoldCache

(T^{*})\,(2,3)

retains

89\%

of the design Score at

1.5{\times}

speedup; alternatives lose

24{-}46\%

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time (sec) $\downarrow$
Full Inference	–	–	–	5577.64
DeepCache ( Ma et al., 2024 )	31.42	0.9416	0.0890	3522.21
PAB ( Zhao et al., 2025 )	36.85	0.9750	0.0552	4744.55
AdaCache ( Kahatapitiya et al., 2025 )	34.35	0.9734	0.0910	3491.62
ManifoldCache $(T^{*})\,(2,3)$	39.27	0.9817	0.0564	3416.05

Table 7: Lyra ( Bahmani et al., 2026 ) (single-image

\to

3D Gaussian Splatting via camera-controlled video DiT, evaluated on multi-view synthesis of dynamic scenes): PSNR , SSIM , LPIPS , and runtime. ManifoldCache has the best PSNR, SSIM, and runtime; PAB has a slightly lower LPIPS but is

1.4{\times}

slower.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time (sec) $\downarrow$	VRAM (MiB) $\downarrow$
Full Inference	–	–	–	409	16520
DeepCache ( Ma et al., 2024 )	26.88	0.8730	0.1891	350	17885
FrDiff ( So et al., 2024 )	31.94	0.9192	0.0764	347	18673
ManifoldCache $(T^{*})\,(2,3)$	40.75	0.9909	0.0579	336	18540

Table 8: SV4D 2.0 ( Yao et al., 2025 ) (multi-view video diffusion for dynamic 4D asset generation, 8 view pairs on Objaverse): PSNR , SSIM , LPIPS , and runtime. ManifoldCache

(T^{*})\,(2,3)

is fastest and delivers

+13.9

dB PSNR over DeepCache.

Figure 3: From left to right: ManifoldCache , FasterDiffusion, DeepCache, and Full Inference. ManifoldCache achieves visual quality most similar to Full Inference on Med-DDPM (3D Brain MRI).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Constraint encoded by $g(x;c)$	How it is maintained
Med-DDPM	anatomical intensity and continuity	statistical
DiffCSP++	space-group and Wyckoff symmetry	structural
Proteina	Ramachandran validity and steric-clash avoidance	learned geometry
SV4D 2.0, Lyra	multi-view photometric consistency	statistical
DiT-MC	bond-geometry validity	learned geometry
UniMoMo	$\mathrm{E}(3)$ -equivariant block-graph validity	structural

Appendix

Table 9: Constraint encoded by

g(x;c)

and how each evaluated model keeps its samples on

\mathcal{M}_{c}

Figure 4: Quality–speed trade-off on all eight CMDMs. Each panel plots sampling time against the model’s primary quality metric: mean scRMSD (Proteina), COV-R (DiT-MC), AAR (UniMoMo), Match Rate (DiffCSP++), FID (Med-DDPM), design Score (DemoDiff), and PSNR (Lyra, SV4D 2.0); the axis labels show whether higher or lower is better. “Ours” is the main ManifoldCache configuration in each table, the dotted lines pass through it, and grey points are the other ManifoldCache configurations and ablations. For Lyra and SV4D 2.0, quality is measured against full inference, so full inference appears only as a dashed line at its runtime. All values are taken from Tables 1 – 8 .

Model	Full Inference	Best baseline	ManifoldCache
Med-DDPM	$1.06\times 10^{17}$	$7.01\times 10^{16}$	$\mathbf{4.64\times 10^{16}}$
Proteina	$7.20\times 10^{14}$	$6.06\times 10^{14}$	$\mathbf{5.80\times 10^{14}}$
UniMoMo	$3.24\times 10^{14}$	$2.04\times 10^{14}$	$\mathbf{2.03\times 10^{14}}$
SV4D 2.0	$2.137\times 10^{15}$	$\mathbf{1.945\times 10^{15}}$	$1.977\times 10^{15}$
DiT-MC	$1.81\times 10^{14}$	$1.65\times 10^{14}$	$\mathbf{1.46\times 10^{14}}$
DemoDiff	$1.52\times 10^{16}$	$1.241\times 10^{16}$	$\mathbf{1.013\times 10^{16}}$

Appendix

Table 10: Inference FLOPs (lower is better). “Best baseline” is the caching baseline with the fewest FLOPs for each model.

Model	Full Inference	Best baseline	ManifoldCache
Med-DDPM	250	188	109
Proteina	200	171	164

Appendix

Table 11: Number of function evaluations (NFE; lower is better). “Best baseline” is the caching baseline with the fewest NFE for each model.

Method	Time (sec) $\downarrow$	Mean scRMSD $\downarrow$	FLOPs $\downarrow$
Full Inference	90.47	1.93	$7.20\times 10^{14}$
Speculative sampling [ De Bortoli et al., 2025 ]	281	16.2	$2.8\times 10^{15}$
Spiffy [ Agrawal et al., 2025 ]	377	18.2	$3.8\times 10^{15}$
ManifoldCache $(T^{*})\,(2,4)$	48.18	2.02	$5.80\times 10^{14}$

Appendix

Table 12: Speculative sampling and ManifoldCache on Proteina. Time and scRMSD for full inference and ManifoldCache are taken from Table 1 .

Figure 5: From left to right: ManifoldCache , PAB, and AdaCache. It can be seen that ManifoldCache achieves the best visual quality on Lyra.

Diffusion Model Inference Acceleration

Diffusion model inference acceleration reduces the time and compute needed to generate or reconstruct outputs while preserving quality.

See all 506 papers

Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCache
Feb 26, 2026Bowen Cui, Yuanbin Wang, Huajiang Xu +6Diffusion Model Inference Acceleration Diffusion Model Caching
Alibaba Group
OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference
Jul 31, 2026Zhikang Xie, Xichen Ye, Yifan Wu +5Diffusion Model Caching Diffusion Model Inference Acceleration
College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · 3Shanghai Key Laboratory of Intelligent Information Processing
OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models
Jun 30, 2026Huanlin Gao, Fang Zhao, Qiang Hui +8Diffusion Sampling Diffusion Model Caching
Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom · National Key Laboratory for Novel Software Technology, Nanjing University

Feb 26, 2026J/K move · Enter open · S save

Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCache

Bowen Cui, Yuanbin Wang, Huajiang Xu, Biaolong Chen+5

Alibaba Group

Diffusion Model Inference Acceleration Diffusion Model Caching Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Diffusion models have demonstrated remarkable success in image and video generation, yet their practical deployment remains hindered by the substantial computational overhead of multi-step iterative sampling. Among acceleration strategies, caching-based methods offer a training-free and effective solution by reusing or predicting features across timesteps. However, existing approaches rely on fixed or locally adaptive schedules without considering the global structure of the denoising trajectory, often leading to error accumulation and visual artifacts. To overcome this limitation, we propose DPCache, a novel training-free acceleration framework that formulates diffusion sampling acceleration as a global path planning problem. DPCache constructs a Path-Aware Cost Tensor from a small calibration set to quantify the path-dependent error of skipping timesteps conditioned on the preceding key timestep. Leveraging this tensor, DPCache employs dynamic programming to select an optimal sequence of key timesteps that minimizes the total path cost while preserving trajectory fidelity. During inference, the model performs full computations only at these key timesteps, while intermediate outputs are efficiently predicted using cached features. Extensive experiments on DiT, FLUX, and HunyuanVideo demonstrate that DPCache achieves strong acceleration with minimal quality loss, outperforming prior acceleration methods by

+

0.031 ImageReward at 4.87

\times

speedup and even surpassing the full-step baseline by

+

0.028 ImageReward at 3.54

\times

speedup on FLUX, validating the effectiveness of our path-aware global scheduling framework. Code is available at https://github.com/argsss/DPCache.

Abstract

Diffusion Model Inference Acceleration

Explore similar work

Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCache

OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inference

OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models