Thermodynamic Diffusion Inference with Minimal Digital Conditioning

Aditi De

cs.LGApr 15, 2026

Thermodynamic Diffusion Inference with Minimal Digital Conditioning

Authors: Aditi De

Abstract

Diffusion-model inference and overdamped Langevin dynamics are formally identical. A physical substrate that encodes the score function therefore equilibrates to the correct output by thermodynamics alone, requiring no digital arithmetic during inference and potentially achieving a $10{,}000\times$ reduction in energy relative to a GPU. Two fundamental barriers have until now prevented this equivalence from being realized at production scale: non-local skip connections, which locally coupled analog substrates cannot represent, and input conditioning, in which the coupling constants carry roughly $2{,}600\times$ too little signal to anchor the system to a specific input. We resolve both obstacles. Hierarchical bilinear coupling encodes U-Net skip connections as rank- $k$ inter-module interactions derived directly from the singular structure of the encoder and decoder Gram matrices, requiring only $O(Dk)$ physical connections instead of $O(D^2)$ . A minimal digital interface -- a 4-dimensional bottleneck encoder together with a 16-unit transfer network, totalling 2,560 parameters -- overcomes the conditioning barrier. When evaluated on activations drawn from a trained denoising U-Net, the complete system attains a decoder cosine similarity of 0.9906 against an oracle upper bound of 1.0000, while preserving theoretical net energy savings of approximately $10^7\times$ over GPU inference. These results constitute the first demonstration of trained-weight, production-scale thermodynamic diffusion inference.

Figures & tables

Figure 1: System architecture. Left: Conventional GPU inference executes the full score network digitally at $\sim$ 1–10 J per image. Right: A 2,560-parameter digital conditioning interface (0.032% of U-Net cost) computes the bias vectors $\mathbf{b}_{\rm enc}$ and $\mathbf{b}_{\rm dec}$ ; hierarchical bilinear skip coupling routes information physically; the Langevin substrate equilibrates under thermal noise to produce the denoised output (cosine similarity 0.9906 versus digital baseline). Theoretical net energy gain: $\sim\!10^{7}\times$ .

Figure 2: Skip-coupling analysis (trained regime). Trained U-Net, $D=64$ , real correlated activations. Panel (a): coupling matrix $\mathbf{J}_{\rm skip}$ . Panel (b): per-dimension decoder shift $\Delta y_{j}^{*}$ (mean $\pm$ std, 64 samples). Panel (c): $\rho_{\rm skip}$ versus rank $k$ . $\rho_{\rm skip}=12.74\%$ , CV $1.4\%$ . The larger shift confirms that training strengthens encoder–decoder singular alignment.

Figure 3: Signal-deficit analysis. Left: Gram-coupling eigenspectrum spanning six decades ( $\lambda_{\rm max}=0.034$ , mean $=0.009$ ), quantifying why naive biases are $\sim\!2{,}600\times$ too small. Right: Naive bias $b=xJ^{\top}$ (red) is uncorrelated with the target activation; the correct oracle bias $b=Ax^{*}$ (green) lies on the identity line. The standard-deviation ratio here is $13\times$ ; the full deficit across all $D$ dimensions reaches $2{,}600\times$ .

Figure 4: Conditioning-interface sweep. Left: Decoder cosine similarity saturates at the oracle level for $k=4$ with both linear and MLP encoders; the binding constraint is the quality of the Gram approximation. Right: Net energy savings ( $\sim\!10^{7}\times$ ) are independent of bottleneck dimension—encoder overhead remains negligible at every scale.

Figure 5: Production test (analytical regime, $D=128$ ). Left: Four conditioning regimes; oracle and full pipeline are visually indistinguishable. The skip-only collapse isolates the conditioning signal carried by rank-16 coupling in the absence of trained-weight structure. Center: Target versus equilibrium scatter lies on the identity line. Right: Per-sample alignment; full-pipeline variance is essentially zero.

Regime	$k=2$	$k=4$	$k=8$	$k=16$	$k=32$	$k=48$	$k=64$
Analytical ( $D=128$ )	21.3	12.5	8.9	7.5	9.0	9.1	9.0
Trained ( $D=64$ )	32.5	21.2	15.0	12.7	12.8	—	—

Table 1: Rank sweep —relative decoder shift

\rho_{\rm skip}

(%). Rank 16 is the operating point.

Regime	Configuration	Dec. cos. $\uparrow$	Params	Theory gain
Analytical $D=128$ random weights	Skip coupling (rank 16)	$7.48\%^{\dagger}$	—	—
	Oracle (ceiling)	0.9966	—	$\sim\!10^{7}\times$
	Skip only ‡	0.0144	256	—
	Full pipeline	0.9954	5,396	$\mathbf{\sim\!10^{7}\times}$
Trained $D=64$ real activations	Skip coupling (rank 16)	$12.74\%^{\dagger}$	—	—
	Oracle (ceiling)	1.0000	—	$\sim\!10^{7}\times$

Table 2: Main results. Both experimental regimes. † Relative shift, not cosine. ‡ Learned encoder + skip coupling as decoder conditioning (no transfer network).

Analog Computing

See all 24 papers

Apr 26, 2026

Symmetric Equilibrium Propagation for Thermodynamic Diffusion Training

The reverse process in score-based diffusion models is formally equivalent to overdamped Langevin dynamics in a time-dependent energy landscape. In our prior work we showed that a bilinearly-coupled analog substrate can physically realize this dynamics at a projected three-to-four orders of magnitude energy advantage over digital inference by replacing dense skip connections with low-rank inter-module couplings. Whether the training loop can be closed on the same substrate -- without routing gradients through an external digital accelerator -- has remained open. We resolve this affirmatively: Equilibrium Propagation applied directly to the bilinear energy yields an unbiased estimator of the denoising score-matching gradient in the zero-nudge limit. For finite nudging we derive a sharp bias bound controlled solely by substrate stiffness, local curvature, and the norm of the loss-gradient signal, with a bilinear-specific corollary showing that one dominant bias term vanishes identically for coupling-parameter updates. Symmetric nudging further upgrades the leading bias from

\mathcal{O}(β)

\mathcal{O}(β^2)

at negligible extra cost. Under realistic finite-relaxation budgets this upgrade is essential, as one-sided EqProp produces anti-correlated gradients while symmetric EqProp yields well-aligned updates. Bias-variance analysis determines the optimal operating point, and end-to-end physical-unit accounting projects a

10^3

10^4\times

energy advantage per training step over a matched GPU baseline. Symmetric bilinear EqProp is the first local, readout-only training rule that preserves the low-rank coupling enabling scalable thermodynamic diffusion models.

Aditi De

Equilibrium Propagation Thermodynamic Computing Score-Based Generative Modeling Diffusion Models

Ask PDF

Jun 30, 2026

Scaling Up Thermodynamic AI Models

Thermodynamic computing devices based on the Ising model show great promise for low-power AI inference and edge computing, but scalable methods for training large models for such hardware remain limited. Prior theory shows that the time-averaged behavior of high-temperature Gibbs-sampled Ising systems can implement feed-forward neural inference. We turn this theoretical correspondence into a scalable and purely backpropagation-based algorithm for training deep convolutional networks for thermodynamic inference on Ising machine hardware. Our image classification models achieve accuracies of 94.9% on CIFAR-10 and 76.0% on CIFAR-100 under binary Gibbs sampling. We then develop and experimentally validate a mathematical theory relating inference cost to accuracy and controlling autocorrelation times. Subsequently, we calculate asymptotic results showing that inference cost is bounded by a well-controlled tradeoff with performance and exhibit algorithms for computing optimal inference schedules. Finally, we discuss implications for hardware development and the future of high-temperature thermodynamic AI models.

Andrew G. Moore

Efficient Neural Network Inference AI Accelerator Inference Thermodynamic Computing

Ask PDF

May 19, 2026

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement

Inference-time scaling has emerged as a major approach for improving reasoning capabilities, and has been increasingly applied to diffusion models. However, existing inference-time scaling methods for diffusion models typically rely on external verifiers or reward models to rank and select samples, limiting their scalability to settings where such evaluators are available and reliable. Moreover, while recent diffusion models perform sequential inference with region-wise, mixed-noise conditioning, inference-time scaling tailored to this setting remains relatively underexplored. We propose Iterative Partial Refinement (IPR), an inference-time scaling method for sequential diffusion that requires no external verifier. Starting from an already-generated sample, IPR re-noises a subset of regions and regenerates them conditioned on the remaining regions, enabling the model to revise earlier decisions under a richer context than was available during the initial generation. This iterative partial refinement produces more globally consistent samples without external verification. On reasoning tasks requiring global constraint satisfaction, IPR consistently improves performance: on MNIST Sudoku, the valid solution rate increases from 55.8% to 75.0%. These results show that iterative partial refinement alone can serve as an effective inference-time scaling strategy for diffusion models in sequential, mixed-noise settings. Code is available at: https://github.com/ahn-ml/IPR

Taegu Kang, Jaesik Yoon, Sungjin Ahn

KAIST · KAIST & SAP · KAIST & NYU

Inference-Time Scaling Diffusion Models

Ask PDF

Abstract

Analog Computing

Explore similar work

Symmetric Equilibrium Propagation for Thermodynamic Diffusion Training

Scaling Up Thermodynamic AI Models

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement