End-to-End Differentiable Diffusion Conditioning for Physics-Informed Optimization

Ricardo Luna Gutierrez; Vineet Gundecha; Rahman Ejaz; Varchas Gopalaswamy; Riccardo Betti; Sahand Ghorbanpour; Aarne Lees; Soumyendu Sarkar

cs.LGOct 22, 2026

End-to-End Differentiable Diffusion Conditioning for Physics-Informed Optimization

Authors: Ricardo Luna Gutierrez, Vineet Gundecha, Rahman Ejaz, Varchas Gopalaswamy, Riccardo Betti, Sahand Ghorbanpour, Aarne Lees, Soumyendu Sarkar

Organizations: Hewlett Packard Enterprise · University of Rochester

Abstract

Optimizing high-dimensional physical systems under expensive evaluations and complex constraints is a central challenge in science and engineering. In offline model-based optimization (MBO), methods must improve designs using only a fixed dataset and no test-time access to the true objective. However, existing methods typically suffer from conservatism, unconstrained surrogate exploitation, or function as static inference-time samplers that cannot refine candidates against precise physical objectives. In this paper, we propose Differentiable Diffusion Conditioning (D2C) for offline MBO, which leverages the fact that deterministic sampling makes the reverse denoising chain differentiable with respect to the conditioning signal. D2C turns a frozen conditional diffusion model into an end-to-end gradient-based optimizer by directly optimizing conditioning variables at test time via gradients from frozen surrogate objectives and physics-informed penalties backpropagated through the reverse chain. We implement this idea in a composed proposer-evaluator framework and evaluate it on laser pulse optimization for inertial confinement fusion, sustainable data-center workload scheduling, and four Design-Bench tasks. Our method achieves the strongest results on the two real-world tasks and the best average rank across four Design-Bench tasks.

Figures & tables

Figure 1: Overview of D2C. A frozen conditional diffusion proposer generates designs from conditioning inputs, a frozen surrogate evaluator and physics-informed penalties define a differentiable test-time objective, and gradients are backpropagated through the deterministic DDIM chain to optimize the conditioning signal. Multiple noise trials and loss-weight samples produce a candidate pool, from which the best valid design is selected after domain-specific constraint checks.

Method	ICF Standard	ICF Top	DC
Gradient Ascent	$2.31\%\pm 0.31$	$1.46\%\pm 0.29$	$0.14\%\pm 0.04$
COMs	$3.14\%\pm 0.41$	$0.43\%\pm 0.11$	$1.33\%\pm 0.24$
BDI	$2.42\%\pm 0.37$	$1.33\%\pm 0.21$	$0.37\%\pm 0.08$
CbAS	$4.49\%\pm 0.63$	$1.86\%\pm 0.29$	$1.85\%\pm 0.31$
MINs	$3.71\%\pm 0.67$	$1.51\%\pm 0.43$	$1.21\%\pm 0.23$
REINFORCE	$3.50\%\pm 0.56$	$0.32\%\pm 0.08$	$1.26\%\pm 0.25$

Table 1: Real-world physical design tasks. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs. Bold : best.

Method	TF Bind 8	TF Bind 10	D’Kitty	Ant	Avg. Rank
$\mathcal{D}$ (best)	$0.439$	$0.467$	$0.884$	$0.565$	–
Gradient Ascent	$0.854\pm 0.029$	$0.575\pm 0.039$	$0.937\pm 0.014$	$0.749\pm 0.034$	$10.38$
COMs	$0.949\pm 0.018$	$0.598\pm 0.034$	$0.935\pm 0.013$	$0.863\pm 0.021$	$6.75$
BDI	$0.969\pm 0.012$	$0.609\pm 0.026$	$0.931\pm 0.014$	$0.717\pm 0.032$	$7.00$
CbAS	$0.918\pm 0.027$	$0.607\pm 0.032$	$0.908\pm 0.017$	$0.672\pm 0.035$	$11.25$
MINs	$0.923\pm 0.026$	$0.625\pm 0.033$	$0.917\pm 0.017$	$0.806\pm 0.030$	$8.50$

Table 2: Design-Bench tasks. 100th-percentile normalized score

\pm

std over 5 runs. The row

\mathcal{D}

(best) reports the best design present in the offline dataset for each task. Scores above

\mathcal{D}

(best) therefore indicate improvement beyond the best offline example under the evaluation protocol used for that column. Avg. Rank is the mean ordinal rank across all four tasks (

\downarrow

lower is better). Bold : best per column.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	ICF	DC
Hidden dimension $H$	1024	1024
Residual blocks $L$	6	6
Dropout	0.1	0.1
CFG dropout $p_{\mathrm{drop}}$	0.1	0.1
EMA decay $\beta$	0.9999	0.9999
Learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$

Appendix

Table 3: Training hyperparameters for the diffusion proposer.

Hyperparameter	ICF	DC
Guidance scale $w$	2.0	2.0
DDIM inference steps $T$	25	25
Optimization steps $J$	100	40
Learning rate $\eta$	$1\times 10^{-3}$	$2\times 10^{-2}$
Noise trials $N_{z}$	20	20
Weight samples $N_{\lambda}$ per trial	5	5

Appendix

Table 4: Inference-time optimization hyperparameters.

Hyperparameter	ICF	DC
Hidden dimension	1024	1024
Hidden layers	10	6
Residual connections	Every 2 layers	Every 2 layers
Activation	ReLU	ReLU
Dropout (training)	0.1	0.1
Output activation	Sigmoid	Sigmoid

Appendix

Table 5: Training hyperparameters for the surrogate evaluator

S_{\phi}

Hyperparameter	Value
Optimized variable	Initial latent/noise vector $z$ (same dimension as $x$ )
Fixed conditioning target	$c=1.0$
Optimizer	Adam
Learning rate	$1\times 10^{-2}$
Optimization steps	100
DDIM inference steps	50

Appendix

Table 6: D2C inference-time hyperparameters for Design-Bench. These settings are shared across TF Bind 8, TF Bind 10, Ant Morphology, and D'Kitty Morphology unless noted otherwise.

Task	$w{=}0$	$w{=}0.5$	$w{=}1$	$w{=}2$	$w{=}3$	$w{=}5$	$w{=}7$
DC	$2.30\%\pm 0.45$	$6.56\%\pm 0.72$	$7.81\%\pm 0.74$	$8.98\%\pm 0.75$	$8.90\%\pm 0.73$	$8.79\%\pm 0.71$	$8.58\%\pm 0.69$
ICF Top	$0.00\%\pm 0.00$	$5.12\%\pm 0.67$	$8.17\%\pm 0.81$	$10.06\%\pm 0.88$	$3.85\%\pm 0.54$	$1.20\%\pm 0.29$	$0.00\%\pm 0.00$

Appendix

Table 7: Ablation on the guidance scale

w

. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs.

Selection rule	$w{=}1$	$w{=}2$	$w{=}3$	$w{=}4$
Full selection rule	$8.27\%$	$\mathbf{11.20\%}$	$4.14\%$	$3.59\%$
Geometric-only rule	$6.14\%$	$\mathbf{9.44\%}$	$4.01\%$	$3.15\%$

Appendix

Table 8: Surrogate-only selection of the guidance scale

w

on 100 held-out ICF Top samples. Values are the mean surrogate-predicted yield gain (%) of the candidate selected per sample; no oracle evaluation is used. Bold : best per row.

Task	$J{=}10$	$J{=}20$	$J{=}30$	$J{=}40$	$J{=}50$	$J{=}75$	$J{=}100$
DC	$7.90\%\pm 0.71$	$8.66\%\pm 0.72$	$8.92\%\pm 0.69$	$8.98\%\pm 0.75$	$8.97\%\pm 0.81$	$8.97\%\pm 0.73$	$8.94\%\pm 0.73$
ICF Top	$0.56\%\pm 0.18$	$1.64\%\pm 0.31$	$3.46\%\pm 0.49$	$5.38\%\pm 0.62$	$6.82\%\pm 0.74$	$8.94\%\pm 0.85$	$10.06\%\pm 0.88$

Appendix

Table 9: Ablation on the number of optimization steps

J

. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs.

Task	$\eta{=}10^{-1}$	$\eta{=}5{\times}10^{-2}$	$\eta{=}2{\times}10^{-2}$	$\eta{=}10^{-2}$	$\eta{=}10^{-3}$
DC	$8.22\%\pm 0.72$	$8.85\%\pm 0.74$	$8.98\%\pm 0.75$	$8.72\%\pm 0.74$	$7.66\%\pm 0.70$
ICF Top	$2.34\%\pm 0.42$	$5.33\%\pm 0.65$	$9.07\%\pm 0.85$	$9.05\%\pm 0.85$	$10.06\%\pm 0.88$

Appendix

Table 10: Ablation on the optimization learning rate

\eta

. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs.

Task	$T{=}10$	$T{=}25$	$T{=}50$
DC	$8.84\%\pm 0.73$	$8.98\%\pm 0.75$	$8.97\%\pm 0.74$
ICF Top	$9.90\%\pm 0.86$	$10.06\%\pm 0.88$	$10.05\%\pm 0.87$

Appendix

Table 11: Ablation on the number of DDIM inference steps

T

. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs.

Task	All	Number of optimizable inputs
DC	$8.98\%\pm 0.75$	$5.14\%\pm 0.61$ ( $d_{o}{=}1$ )	$5.67\%\pm 0.64$ ( $d_{o}{=}2$ )	$5.89\%\pm 0.65$ ( $d_{o}{=}4$ )	$7.97\%\pm 0.72$ ( $d_{o}{=}6$ )
ICF	$10.06\%\pm 0.88$	$4.02\%\pm 0.56$ ( $d_{o}{=}1$ )	$7.97\%\pm 0.79$ ( $d_{o}{=}4$ )	$8.74\%\pm 0.83$ ( $d_{o}{=}9$ )	$8.97\%\pm 0.85$ ( $d_{o}{=}15$ )

Appendix

Table 12: Ablation on the number of optimizable conditioning inputs

d_{o}

. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs.

Optimized variable	ICF Top	DC
Latent noise $x_{T}$	$7.56\%\pm 0.76$	$3.62\%\pm 0.26$
Conditioning $c_{\mathrm{opt}}$ (D2C)	$\mathbf{10.06\%\pm 0.88}$	$\mathbf{8.98\%\pm 0.75}$

Appendix

Table 13: Conditioning optimization versus latent-noise optimization on the real-world tasks. Values are mean percentage improvement (%) over the baseline design

\pm

std over 5 runs. Bold : best.

Method	ICF Standard	ICF Top	DC
BDI	$27.40\pm 1.28$	$18.24\pm 1.11$	$6.00\pm 0.24$
Brute Force	$32.97\pm 1.36$	$31.26\pm 1.41$	$64.50\pm 2.58$
CbAS	$31.84\pm 1.17$	$18.52\pm 0.83$	$39.23\pm 1.61$
D2C (Ours)	$81.23\pm 2.86$	$75.12\pm 2.78$	$100.00\pm 0$
CMA-ES	$29.97\pm 1.19$	$19.59\pm 0.97$	$42.21\pm 1.59$
COMs	$25.82\pm 0.97$	$9.12\pm 0.24$	$40.13\pm 1.80$

Appendix

Table 14: Constraint-satisfying improvement rate (%) on the real-world tasks. Values are mean constraint-satisfying improvement rate

\pm

std over 5 runs, where the metric is the percentage of evaluation samples for which the final selected design achieves nonzero gain relative to the baseline (equivalently, at least one valid improving candidate is found). Higher is better.

Task	$1\times 3$	$5\times 3$	$10\times 3$	$20\times 1$	$20\times 2$	$20\times 3$	$20\times 5$
ICF Top	$0.86\%\pm 0.24$	$4.59\%\pm 0.58$	$6.50\%\pm 0.72$	$5.72\%\pm 0.69$	$8.99\%\pm 0.85$	$9.64\%\pm 0.86$	$10.06\%\pm 0.88$
DC	$4.21\%\pm 0.57$	$7.82\%\pm 0.74$	$8.66\%\pm 0.69$	$8.41\%\pm 0.64$	$8.83\%\pm 0.72$	$8.91\%\pm 0.74$	$8.98\%\pm 0.75$

Appendix

Table 15: Ablation on the candidate budget

N_{z}\times N_{\lambda}

. Values are mean

\pm

standard deviation over 5 seeds. Configurations are labeled by their

N_{z}\times N_{\lambda}

values.

Method	ICF Top	DC
BDI	2.5	2.9
Brute Force	23.2	35.7
CbAS	14.2	17.5
CMA-ES	35.6	43.7
COMs	18.3	32.9
DDOM	2.3	3.6

Appendix

Table 16: Computation time (seconds per sample) for each method. Lower is better.

Differentiable Optimization

Differentiable optimization makes solvers and constrained decisions trainable with gradients for learning, control, design, and inverse problems.

See all 84 papers

Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
Jun 7, 2026Yilin Zheng, Haowei Wang, Szu Hui Ng +1Bayesian Optimization Black-Box Optimization
National University of Singapore · Singapore · Georgia Institute of Technology +1
ParetoPilot: Zero-Surrogate Offline Multi-Objective Optimization via Infer-Perturb-Guide Diffusion
Jun 3, 2026Ruiqing Sun, Sen Yang, Dawei Feng +3Diffusion Guidance Multi-Objective Optimization
College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China · Bioinformatics Center of AMMS, Beijing 100080, China
D-IMPL: A Diffusion-based Solver for Parameterized BBOs
Sep 19, 2026Yang Hu, Na LiBlack-Box Optimization Learning to Optimize
School of Engineering and Applied Sciences, Harvard University, Boston, MA

Jun 7, 2026J/K move · Enter open · S save

Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models

Yilin Zheng, Haowei Wang, Szu Hui Ng, Enlu Zhou

National University of Singapore · Singapore · Georgia Institute of Technology+1

Bayesian Optimization Black-Box Optimization Conditional Diffusion Models

Open paper AskPDF arXiv

Bayesian optimization (BO) is a widely used approach for black-box optimization that uses a Gaussian process (GP) as a surrogate and guides sequential evaluations via an acquisition function, with the ultimate goal of locating the global optimum

\mathbf{x}^{\star}

. To align with this goal, information-based acquisition functions such as Predictive Entropy Search (PES) model

\mathbf{x}^{\star}

as a random variable and reduce the entropy of its distribution, but approximating this distribution via traditional GP posterior sampling is computationally expensive. To address this limitation, we leverage Conditional Diffusion Models (CDMs) to efficiently approximate the distribution of

\mathbf{x}^{\star}

and develop BO-inherent training strategies for CDMs. Motivated by the structural properties of the CDM-learned distribution, we further develop an acquisition strategy termed Diffusion-based Mode Seeking (DMS) to guide the sequential evaluation. We establish a sub-optimality guarantee for the CDM-learned distribution and demonstrate through extensive experiments that DMS outperforms standard BO baselines.

Abstract

Differentiable Optimization

Explore similar work

Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models

ParetoPilot: Zero-Surrogate Offline Multi-Objective Optimization via Infer-Perturb-Guide Diffusion

D-IMPL: A Diffusion-based Solver for Parameterized BBOs