Characterizing and Mitigating Distributional Bias in Physics-Informed Generative Models

Tanja Bien; Samir Darouich; Lluís Pastor-Pérez; Vinh Tong; Loay Mualem; Mathias Niepert

cs.LGOct 12, 2026

Characterizing and Mitigating Distributional Bias in Physics-Informed Generative Models

Authors: Tanja Bien, Samir Darouich, Lluís Pastor-Pérez, Vinh Tong, Loay Mualem, Mathias Niepert

Organizations: University of Stuttgart

Abstract

Physics-informed generative models augment a data-fitting objective with residual penalties that encourage physical constraints to hold in generated samples. However, improving constraint satisfaction need not improve fidelity to the data distribution. We study this conflict in residual-regularized diffusion models and identify two possible consequences. First, the same prediction error can incur different residual costs near different feasible states, preferring states with lower residual sensitivity (avoidance). Second, this preference can extend to feasible states outside the data support, where the residual also vanishes (leakage). Motivated by this analysis, we propose a data-anchored penalty that weights prediction errors by the residual Jacobian at the clean training sample and normalizes the resulting magnitude. This removes the incentive to relocate to less sensitive states without changing the model architecture or sampling procedure. Experiments on a controlled synthetic system, three PDE systems, and molecular transition-state generation with flow matching demonstrate the benefits of anchoring. Code is available at https://github.com/TanjaBien/anchored_diffusion.

Figures & tables

Figure 1: Illustration of avoidance . The training solution

x_{0}

and an alternative solution

y

both satisfy

R=0

, but the same zero-mean perturbation

\delta

produces a smaller expected residual penalty near the less sensitive solution

y

. Shifting predictions from

x_{0}+\delta

y+\delta

therefore reduces the residual penalty while adding an expected denoising cost of

\|y-x_{0}\|^{2}

. The combined objective prefers this shift when the weighted residual reduction exceeds the added denoising cost. If the favored solution lies outside the data support, this preference results in leakage .

Figure 2: Comparison of the standard and data-anchored objectives for the displacement illustrated in Figure 1 . Curves show the change in expected loss relative to the predictions at

x_{0}

, with the perturbation distribution held fixed. The dashed curve shows the added denoising cost. (a) The standard residual penalty can decrease enough to outweigh this cost, favoring predictions centered at

y

. (b) Anchoring the residual Jacobian at

x_{0}

removes this saving, so displacement can only increase the expected combined loss, whose minimum remains at

x_{0}

Figure 3: PIDM on the shocked circle at $D=10$ . Top, generated samples projected onto the data plane, with shading indicating residual sensitivity. Bottom, their angular density compared with the data distribution (dashed) and the residual sensitivity $\sigma(\theta)$ (grey). The first column uses the flat residual as a control. Right, median radial residual $|\|x\|-1|$ and distance to the data plane $\|x_{3:D}\|$ as functions of the penalty weight.

Figure 4: Trade-offs between distributional fidelity and physical accuracy on Burgers (left), Darcy (middle), and advection (right). Each point shows the MMD and mean absolute residual

\overline{|R|}

for one reported setting. Both axes are logarithmic, and lower left is better. Stars denote DM, orange squares PIDM, and blue circles the anchored objective. Dashed lines connect nondominated settings within each regularized method. A setting is nondominated if no other tested weight of the same method reduces one metric without increasing the other. The line segments serve as visual guides. Rings and labels mark the lowest-MMD setting for each regularized method, breaking MMD ties by lower residual. The labeled

c

values are the coefficients used in each method’s objective.

Burgers
Method	$c$	$\overline{\|R\|}$ $\downarrow$	MMD $\downarrow$	$\sigma$ ratio $\to 1$	Field std. ratio $\to 1$
DM	–	$0.25$	$0.12$	$0.82$	$0.81$
PIDM	None	–	–	–	–
Anchored (ours)	$10^{-2}$	$\mathbf{0.23}$	$\mathbf{0.037}$	$\mathbf{0.91}$	$\mathbf{0.90}$
Darcy
DM	–	$6.0$	$0.054$	$0.84$	$1.1$

Table 1: Results at the weight

c

that minimizes the residual subject to MMD being no higher than the DM baseline. “–” indicates that no tested weight can match the MMD of the DM baseline. Bold marks the best-displayed value within each PDE, with ratios compared by their distance from it. The field standard-deviation ratio uses

u

for Burgers and

p

for Darcy.

	RMSD (Å) $\downarrow$	D-MAE (Å) $\downarrow$	$\Delta E$ (kcal/mol) $\downarrow$	$F_{\text{max}}$ (kcal/mol/Å) $\downarrow$	Hessian MAE (kcal/mol/Å 2 ) $\downarrow$	Valid TS (%) $\uparrow$
FM	0.134	0.060	2.53	33.50	8.40	57.0
FM + residual	0.140	0.066	8.11	67.80	12.56	36.0
FM + anchored (ours)	0.135	0.059	2.36	31.22	8.09	60.0

Table 2: TS generation on Transition1x-2p3p4p. All metrics except the valid TS rate are medians over the generated structures. The Hessian MAE is computed against the GFN2-xTB reference Hessians, and the valid TS rate is the percentage of generated structures whose Hessian has exactly one negative eigenvalue.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Data support and zero set of the residual, drawn in three dimensions for visualization. The residual is satisfied on the whole sphere, and the data are only the circle where the sphere meets the data plane. The experiments use

D=10

Figure 6: The three things a penalized prediction can do, drawn in three dimensions. In black, where the sample should be, at a state where the residual is sensitive. (A) Correct prediction. (B) Leakage, which leaves the data plane and therefore the data support along the zero set, where

|R|=0

. (C) Avoidance, where the prediction moves to a region of lower stiffness, which may or may not lie on the data support.

Figure 7: One data circle, two residuals. Top, the residual in the data plane, where the zero set is the same circle for both. Bottom, the residual cost $2g$ along the data, which spans a factor of $101$ for graded and is constant for flat .

	graded	flat (control)
heights $h_{k}$	$4,12,35,100$	—
widths $w_{k}$	$0.10$	—
$2g$ along $\mathcal{M}$	$2\!\to\!202$ ( $101\times$ )	$2$ ( $1\times$ )
bump HWHM	$4.8^{\circ}$	—

Appendix

Table 3: The two residual profiles of the shocked circle.

	Shocked circle	Advection and Burgers	Darcy
Denoiser	MLP	U-Net	U-Net
Architecture	$5\times 512$ , SiLU	channels $32,64,128,256$	as Burgers
		2 ResNet blocks per level,
		linear attention per level,
		attention in the bottleneck
Input channels	–	$1$ ( $u$ )	$2$ ( $p$ , $K$ )

Appendix

Table 4: Training and sampling settings. The same settings are used for DM, PIDM and the anchored objective.

	DM	PIDM	Anchored
Advection	$526$	$531$ ( $+1.0\%$ )	$539$ ( $+2.5\%$ )
Burgers	$1067$	$1071$ ( $+0.4\%$ )	$1082$ ( $+1.5\%$ )
Darcy	$41$	$57$ ( $+38\%$ )	$92$ ( $+123\%$ )

Appendix

Table 5: Wall-clock time in seconds per

1000

training iterations on one A40 GPU at batch size

8

, for the training step alone. In parentheses the increase over DM.

Figure 8: Anchored on the shocked circle at $D=10$ . Top, samples in the data plane with the residual sensitivity as shading. Bottom, angular density of the samples against the data (dashed) and $\sigma(\theta)$ (grey). The first column uses flat . Right, residual vs. distance to the data plane which measures leakage.

	$c$	$\overline{\|R\|}$ $\downarrow$	WD $\downarrow$	$\sigma$ ratio $\to 1$	$\\|x_{3:D}\\|$ $\downarrow$
DM	–	9.12e-3	8.66e-3	0.978	3.23e-4
PIDM	$10^{-3}$	2.53e-3	2.38e-2	0.582	1.38e-3
	$10^{-2}$	1.90e-3	2.91e-2	0.356	8.60e-4
	$10^{-1}$	1.04e-3	4.15e-2	0.231	1.36e-3
	$1$	5.15e-4	7.80e-2	0.201	3.30e-3
Anchored	$10^{-3}$	7.67e-3	1.17e-2	0.973	1.49e-4

Appendix

Table 6: Shocked circle, graded ,

D=10

\overline{|R|}

measures physical accuracy, WD (circular, on the angle) distributional accuracy, the

\sigma

ratio avoidance and the distance to the data plane leakage.

PIDM
$c$	$\overline{\|R\|}$ $\downarrow$	MMD $\downarrow$	$\sigma$ ratio $\to\!1$	$\mathrm{std}(u)$ ratio $\to\!1$
DM	0.25	0.12	0.82	0.81
$10^{-7}$	0.11	0.57	0.65	0.64
$10^{-6}$	0.13	0.56	0.62	0.62
$10^{-5}$	0.25	0.95	0.46	0.45
$10^{-4}$	0.22	1.2	0.13	0.16

Appendix

Table 7: Burgers.

Figure 9: Burgers fields generated a standard diffusion model vs. PIDM and anchored with increasing penalty weights using the same seed.

Figure 10: Sampling trajectories on Burgers. Each row is one model and each column one step $t$ of the sampler, from pure noise ( $t=99$ ) to the final sample ( $t=0$ ). Every panel shows the model’s prediction $\hat{x}_{0}$ at that step, and the number below it is the mean residual $\overline{|R|}$ of that prediction. All rows start from the same noise and receive the same noise in every step, so the last column equals the corresponding fields in Fig. 9 .

Figure 11: Burgers samples by sensitivity

\sigma

. Top, per-sample residual of DM, the anchored objective (

c=10^{-2}

) and PIDM (

c=10^{-1}

, left). Bottom, density of

\sigma

for the data and each model. PIDM generates only samples far less sensitive than the data, while the anchored samples are closer to the data than those of DM and have a lower residual in every sensitivity range.

Figure 12: Darcy pressure fields generated from the same starting noise by DM, PIDM and the anchored objective at two weights each.

PIDM
$c$	$\overline{\|R\|}$ $\downarrow$	forward error $\downarrow$	MMD $\downarrow$	$\sigma$ ratio $\to\!1$	$\mathrm{std}(p)$ ratio $\to\!1$	$\mathrm{std}(K)$ ratio $\to\!1$
DM	6.0	3.4e-2	5.4e-2	0.84	1.1	0.75
$10^{-7}$	1.2	4.3e-2	7.3e-2	0.80	1.2	0.72
$10^{-6}$	0.96	5.5e-2	4.7e-2	0.85	1.1	0.74
$10^{-5}$	0.28	0.19	4.6e-2	0.83	0.98	0.73
$10^{-4}$	0.33	0.39	0.20	0.91	0.64	0.75

Appendix

Table 8: Darcy. Rows of PIDM and the anchored objective with comparable weights are aligned. The anchored objective reaches a lower residual at comparable MMD.

PIDM				Anchored
$c$	$\overline{\|R\|}$ $\downarrow$	MMD $\downarrow$	$\mathrm{std}(u)$ ratio $\to\!1$	$c$	$\overline{\|R\|}$ $\downarrow$	MMD $\downarrow$	$\mathrm{std}(u)$ ratio $\to\!1$
$10^{-3}$	4.6e-3	0.45	0.37	$2\cdot 10^{3}$	5.3e-3	0.29	0.50
$10^{-2}$	2.2e-3	0.59	0.24	$2\cdot 10^{4}$	4.2e-3	0.37	0.41
$10^{-1}$	1.3e-3	0.84	5.6e-2	$2\cdot 10^{5}$	1.6e-3	0.61	0.17
$1$	9.2e-4	0.87	7.2e-3	$2\cdot 10^{6}$	9.1e-4	0.87	4.1e-3

Appendix

Table 9: Advection. PIDM (left) and the anchored objective (right) with comparable weights in the same row. DM reaches

\overline{|R|}=1.6\text{e-}2

, an MMD of

0.28

and a

\mathrm{std}(u)

ratio of

0.51

Physics-Informed Generative Modeling

Physics-informed generative modeling uses physical laws and data-driven generators to create plausible samples and solve scientific inverse problems.

See all 107 papers

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training
Feb 10, 2026Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin +2Physics-Informed Generative Modeling PDE Surrogate Modeling
Institute of Artificial Intelligence Innovation National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan National Center for Theoretical Sciences National Taiwan University Taipei, Taiwan
Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation
May 28, 2025Yi Zhang, Peng Wang, Difan ZouPhysics-Informed Diffusion Models Diffusion Model Distillation
Institute of Data Science · Department of Computer and Information Science · Institute of Data Science & School of Computing and Data Science
Source Anchoring for Physical Consistency in Flow Matching Models
Sep 27, 2026Giulia Romoli, Filippo Ruffini, Paolo SodaConstrained Generative Modeling Physics-Informed Generative Modeling
Department of Diagnostics and Intervention, Ume˚a University, Sweden · Unit of Artificial Intelligence and Computer Systems, Universit`a Campus Bio-Medico di Roma, Italy

Feb 10, 2026J/K move · Enter open · S save

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training

Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin, Ming-Chih Lai+1

Institute of Artificial Intelligence Innovation National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan National Center for Theoretical Sciences National Taiwan University Taipei, Taiwan

Physics-Informed Generative Modeling PDE Surrogate Modeling Few-Step Diffusion Sampling Physics-Informed ML Consistency Models

Open paper AskPDF arXiv

We propose a physics-informed consistency modeling framework for solving partial differential equations (PDEs) via fast, few-step generative inference. We identify a key stability challenge in physics-constrained consistency training, where PDE residuals can drive the model toward trivial or degenerate solutions, degrading the learned data distribution. To address this, we introduce a structure-preserving two-stage training strategy that decouples distribution learning from physics enforcement by freezing the coefficient decoder during physics-informed fine-tuning. We further propose a two-step residual objective that enforces physical consistency on refined, structurally valid generative trajectories rather than noisy single-step predictions. The resulting framework enables stable, high-fidelity inference for both unconditional generation and forward problems. We demonstrate that forward solutions can be obtained via a projection-based zero-shot inpainting procedure, achieving consistent accuracy of diffusion baselines with orders of magnitude reduction in computational cost.

Abstract

Physics-Informed Generative Modeling

Explore similar work

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training

Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

Source Anchoring for Physical Consistency in Flow Matching Models