Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators

Somyajit Chakraborty; Xizhong Chen

cs.LGSep 30, 2026

Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators

Authors: Somyajit Chakraborty, Xizhong Chen

Organizations: State Key Laboratory of Synergistic Chem-Bio Synthesis, Department of Chemical Engineering, School of Chemistry and Chemical Engineering, Shanghai Jiao Tong University, Shanghai 200240, People’s Republic of China

Abstract

Machine-learning surrogates accelerate physical simulation, but lower prediction error need not coincide with lower error in physically relevant flow statistics. We examine this question for flow around a NACA4418 airfoil using paired computational-fluid-dynamics simulations and experimental particle-image-velocimetry measurements. A mean-preserving input intervention removes velocity fluctuations from selected regions of observed flow histories. Across four neural operators, removing fluctuations from the most energetic 10% of valid observed cells changes forecasts more than equal-area random removal. Because the masks are not matched for removed fluctuation energy, this contrast measures sensitivity, not independent evidence of physical importance. Separately, a CNO has lower velocity-field error but substantially higher two-component fluctuation-energy error than the reference on both analysis subsets. An output attenuation stress test also demonstrates disagreement between benchmark errors and domain-summed fluctuation energy. These single-benchmark results motivate reporting complementary physical diagnostics alongside aggregate prediction scores; they do not establish counterfactual physical correctness.

Figures & tables

Figure 1: Sim2Real forecasting and physics-grounded explanation workflow. Paired CFD/PIV data support model development, while real inference maps 20 observed PIV frames to 20 future frames. History fluctuation energy defines equal-area energetic, random, and low-energy masks; masked fluctuations are replaced by their local temporal mean, and the frozen surrogate response is evaluated. Equal area does not imply equal removed fluctuation energy.

Figure 2: Physics-grounded counterfactual attribution for the reference surrogate. (a) History fluctuation-energy map $k_{\mathrm{hist}}$ for a representative calibration window; the contour marks the highest-energy 10% of valid observed cells. (b) Absolute change in predicted future TKE after removing temporal fluctuations inside that region. (c,d) Trajectory-level energetic-minus-random contrasts for future field response and TKE-map response. Points are equal-weight trajectory aggregates; diamonds and bars denote the mean and 95% trajectory-bootstrap interval.

Figure 3: Cross-model metric ranking. Points are model-minus-reference trajectory-balanced means with 95% bootstrap intervals. The upper-left quadrant denotes lower field error but higher TKE error: CNO lies robustly there on both subsets, while the decoder variant gives a smaller within-lineage example.

Figure 4: Benchmark TKE error and domain-summed energy mismatch rank attenuation differently. (a) Trajectory-level $\Delta e_{\mathrm{TKE}}$ versus $\Delta|1-R_{E,j}|$ . (b) Calibration-mean TKE. (c,d) Signed TKE bias before and after attenuation.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Schema / value
System and domains	NACA4418 airfoil; paired CFD simulation and experimental PIV
Variables	CFD: $u,v,p$ ; PIV: measured $u,v$
Operating conditions	AoA $0^{\circ},5^{\circ},10^{\circ},15^{\circ},20^{\circ}$ ; $Re=2968$ –27975
Released data	100 paired trajectories; approximately 600 time steps per trajectory
Spatial grids	$64\times 128$ released; $32\times 64$ scoring grid after $2\times$ subsampling
Forecast window	20 observed history frames $\rightarrow$ 20 future frames

Appendix

Table 1: Data and evaluation schema.

Figure 5: Temporal evolution of cross-stream velocity $v$ for a representative calibration window. The window is selected using only the reference surrogate as the sample whose Rel- $L_{2}$ is closest to the calibration median, and the same window is shown for every model. Columns correspond to future times $t_{4},t_{8},t_{12},t_{16},t_{20}$ . All panels use the same color scale derived from the experimental PIV target. The comparison shows the experimental target, hybrid reference, FNO, CNO, and Transolver forecasts without model-specific rescaling.

Setting	Selection	Calibration
Reference, 5%	$0.285\,[0.257,0.316]$	$0.284\,[0.259,0.312]$
Reference, 10%	$0.351\,[0.329,0.373]$	$0.346\,[0.326,0.366]$
Reference, 20%	$0.363\,[0.347,0.378]$	$0.358\,[0.343,0.369]$
FNO, 10%	$0.423\,[0.398,0.448]$	$0.421\,[0.398,0.445]$
CNO, 10%	$0.397\,[0.369,0.426]$	$0.392\,[0.369,0.416]$
Transolver, 10%	$0.493\,[0.452,0.534]$	$0.482\,[0.441,0.525]$

Appendix

Table 2: Robustness of the physics-grounded attribution across perturbation size and surrogate architecture. Entries are trajectory-mean energetic-minus-random TKE-response contrasts

A_{\mathrm{TKE}}^{H}-A_{\mathrm{TKE}}^{R}

with 95% trajectory-bootstrap intervals.

Figure 6: Robustness of the physics-grounded attribution. (a) For the hybrid reference surrogate, the energetic-minus-random TKE-response contrast

A_{\mathrm{TKE}}^{H}-A_{\mathrm{TKE}}^{R}

remains positive at

5\%

10\%

, and

20\%

energetic-mask fractions, with closely agreeing selection and calibration estimates. (b) At the canonical

10\%

mask fraction, the same positive contrast is reproduced by the reference, FNO, CNO, and Transolver. Error bars denote 95% trajectory-bootstrap intervals.

Subset	Model	$e_{\mathrm{Rel}\text{-}L_{2}}$	$e_{\mathrm{TKE}}$	MVPE
Selection	Reference	0.120958	0.533389	0.098439
	FNO	0.082586	0.540132	0.069686
	CNO	0.102102	0.653011	0.092459
	Transolver	0.156067	0.840043	0.139438
	Decoder var.	0.119823	0.539500	0.098678
	Attenuation	0.119692	0.528463	0.098439

Appendix

Table 3: Deterministic benchmark errors. Lower values are better. FNO, CNO, and Transolver are separately evaluated released baselines; decoder and attenuation are interventions around the reference lineage.

Split	Model	$\Delta e_{\mathrm{Rel}\text{-}L_{2}}$	$\Delta e_{\mathrm{TKE}}$
Selection	FNO	$-0.0384\,[-0.0493,-0.0277]$	$+0.0067\,[-0.0169,0.0320]$
	CNO	$\mathbf{-0.0189}\,[-0.0265,-0.0117]$	$\mathbf{+0.1196}\,[0.1016,0.1369]$
	Transolver	$+0.0351\,[0.0224,0.0482]$	$+0.3067\,[0.2665,0.3401]$
Calibration	FNO	$-0.0385\,[-0.0495,-0.0279]$	$+0.0071\,[-0.0154,0.0317]$
	CNO	$\mathbf{-0.0191}\,[-0.0267,-0.0120]$	$\mathbf{+0.1232}\,[0.1101,0.1366]$
	Transolver	$+0.0350\,[0.0227,0.0475]$	$+0.3033\,[0.2670,0.3313]$

Appendix

Table 4: Cross-architecture metric differences relative to the reference surrogate. Values are equal-trajectory-weight means with 95% trajectory-bootstrap intervals. Bold rows exhibit a robust ranking inversion: lower field error and higher TKE error.

Explore similar work

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates
Jun 15, 2026Changjian Zhou, Junfeng Fang, Negin Yousefpour +3Neural Operators Parametric Physics-Informed Neural Network
Faculty of Engineering and IT, University of Melbourne · School of Computing, National University of Singapore · Artificial Intelligence Research Institute, IFLYTEK Co., Ltd.
Exactness Matters for Physical Rule Enforcement
May 8, 2026Bum Jun KimReynolds-Averaged Navier-Stokes Exact
The University of Tokyo, Japan
M $^3$ : Reframing Training Measures for Discretized Physical Simulations
May 9, 2026Yuan Mei, Xingyu Song, Xiaowen Song +1Surrogate Models Physics Simulation
1Zhejiang University · *Visiting student at The University of Tokyo from Zhejiang University. · 2The University of Tokyo

Jun 15, 2026, cs.LGJ/K move · Enter open · S save

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Changjian Zhou, Junfeng Fang, Negin Yousefpour, Peng Wu+2

Faculty of Engineering and IT, University of Melbourne · School of Computing, National University of Singapore · Artificial Intelligence Research Institute, IFLYTEK Co., Ltd.

Neural Operators Parametric Physics-Informed Neural Network Fisher Information Matrix Sim-To-Real Gap Projection

Open paper AskPDF arXiv

Neural operator models trained on simulation data often lose accuracy when applied to experimental measurements due to the sim-to-real gap. Standard fine-tuning with limited real data can reduce this gap, but it may also damage the core physics-relevant representations learned during pretraining. Although knowledge-preserving adaptation has been widely investigated in vision or language tasks, it remains unclear whether these methods are suitable for neural operators whose architectures and protected knowledge are fundamentally different. Neural operators need to preserve core-scale physical structures rather than semantic or visual features. We propose PhysGuard, a physics-preserving framework for accurate sim-to-real adaptation of neural operators. Specifically, PhysGuard uses the empirical Fisher Information Matrix computed on simulation data to identify physics-critical parameter directions, then restricts fine-tuning updates to directions that do not interfere with them. A layer-wise Gram-matrix formulation makes this efficient for models with millions of parameters, while an adaptive threshold automatically determines the protected subspace size. A spectral probe experiment shows that the dominant Fisher directions are strongly associated with low-frequency output structures. Experiments on benchmark across four neural operator architectures and different physical systems show that PhysGuard performs strongly on most evaluation metrics compared to baselines. The benefits are most evident under severe domain shift, where it reduces low-frequency error by up to 32% compared to standard fine-tuning while maintaining adaptability. Our code is available at https://github.com/ZhouChaunge/PhysGuard.

Abstract

Explore similar work

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Exactness Matters for Physical Rule Enforcement

M3^33: Reframing Training Measures for Discretized Physical Simulations

M $^3$ : Reframing Training Measures for Discretized Physical Simulations