Edge Accuracy Is Not Enough: Why Dynamics-Learned Structure Fails to Transfer to Inverse Problems

Nicholas Tan Jerome; Fangnian Wang

cs.LGOct 7, 2026

Edge Accuracy Is Not Enough: Why Dynamics-Learned Structure Fails to Transfer to Inverse Problems

Authors: Nicholas Tan Jerome, Fangnian Wang

Organizations: Institute for Data Processing and Electronics (IPE) Karlsruhe Institute of Technology (KIT) Karlsruhe, Germany · Institute for Thermal Energy Technology and Safety (ITES) Karlsruhe Institute of Technology (KIT) Karlsruhe, Germany

Abstract

A natural strategy for inverse problems with scarce labelled data is to transfer relational structure learned from abundant forward-simulation data. We show this strategy fails systematically, even when it satisfies the standard theoretical justification for why structure should help. We prove that approximate structure provides estimation-error benefits whenever the edge error satisfies $Δ< n^2 - kn$ , reducing sample complexity from $O(n^2)$ to $O(kn+Δ)$ . Structure learned via Neural Relational Inference (NRI) from dynamics prediction satisfies this condition, yet on a source-localisation task across 180 CFD-simulated hydrogen-leak scenarios and 180 acoustic scenarios, it degrades performance by 116% and 201% relative to a flexible, task-optimised attention baseline, while a physics-based prior (Green's function) degrades by only 69-72%. Four independent lines of evidence show this is not a tuning failure: NRI improves only 0.5% when given 18x more training data (versus 16.6% for the task-optimised baseline, $p<0.001$ ); performance is insensitive to the NRI edge threshold across a wide range; the dynamics-learned graph overlaps the task-optimal graph on only 6% of edges; and two further dynamics-derived structure estimators (correlation- and mutual-information-based) show no measurable benefit over a structure-free baseline, with the correlation-based estimator performing markedly worse. We formalise this gap as a statement about approximation error that the edge-accuracy condition cannot control, and we provide a lightweight transferability test (Jaccard similarity against a partially-observed target-task graph) that separates successful from failed transfer in all four domain/structure pairs we evaluate, using under an hour of computation and 15-20% of target-domain data; we present this as a heuristic calibrated on few cases, not a validated general threshold.

Figures & tables

Localiser	CFD RMSE (m) $\downarrow$	Acoustic RMSE (m) $\downarrow$
MLP (no structure)	$14.57\pm 0.24$	$4.35\pm 0.07$
Green’s Function	$8.96\pm 0.07$	$1.61\pm 0.05$
Learned Attention	$\mathbf{5.21\pm 0.22}$	$\mathbf{0.95\pm 0.04}$
NRI-Graph	$11.26\pm 0.10$	$2.86\pm 0.05$

Table 1: Structure comparison,

n=15

sensors, mean

\pm

std over 10 seeds. The ranking (Attention

>

Green’s

>

NRI

>

MLP) is preserved across two domains governed by different physics.

$m$	MLP	Green’s	Attention	NRI-Graph
10	$15.94\pm 1.28$	$8.63\pm 0.38$	$6.26\pm 0.54$	$11.32\pm 0.65$
180	$14.57\pm 0.24$	$8.96\pm 0.07$	$5.21\pm 0.22$	$11.26\pm 0.10$
Improvement	$8.6\%$	$-3.9\%$	$16.6\%$	$0.5\%^{***}$

Table 2: Data scaling: RMSE (m) as a function of training scenarios

m

(CFD domain). NRI-Graph’s improvement is an order of magnitude smaller than Attention’s despite identical data growth.

Domain	Structure	RMSE	Degradation	Est. $J$
CFD	Green’s Function	8.96 m	72%	$\approx 0.31$
CFD	NRI Learned Graph	11.26 m	116%	$\approx 0.06$
Acoustic	Green’s Function	1.61 m	69%	$\approx 0.47$
Acoustic	NRI Learned Graph	2.86 m	201%	$\approx 0.19$

Table 3: Transfer outcomes vs. estimated structural similarity to the task-optimal graph. A

J\geq 0.30

threshold separates successful from failed transfer in all four cases.

Method	RMSE (m) $\downarrow$	Sparsity	Est. $J$
MLP (no structure)	$14.57\pm 0.24$	—	0.00
NRI-Graph	$11.26\pm 0.10$	97%	$\approx 0.06$
Correlation-Graph (best $\tau$ )	$18.61\pm 0.01$	5%	$\approx 1.00$
Mutual-Information-Graph (best pct.)	$14.48\pm 0.00$	80%	$\approx 0.37$

Table 4: Two additional dynamics-learned structure baselines (CFD domain,

n=15

, mean

\pm

std over 10 seeds), neither derived from NRI. Correlation-Graph clearly underperforms the unstructured MLP; Mutual-Information-Graph is statistically indistinguishable from it. Neither provides a measurable benefit, ruling out an NRI-architecture-specific explanation for why dynamics-derived structure fails to help.

	CFD RMSE (m)		Acoustic RMSE (m)
$n$	MLP	NRI-Graph	MLP	NRI-Graph
10	14.02	12.37	3.60	3.43
20	15.17	10.53	5.19	2.41
Change	$+8\%$	$-15\%$	$+44\%$	$-30\%$

Table 5: Sensor-count scaling. The unstructured MLP degrades with more sensors in both domains; all structured methods improve, confirming that NRI-Graph’s hypothesis class is genuinely sparsity-constrained even though the constraint is misaligned with the task.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Localiser	Train	Test	Gap	Train	Test	Gap
	CFD			Acoustic
Learned Attention	5.45	5.67	$+4.1\%$	1.05	1.15	$+10.1\%$
Green’s Function	8.88	9.44	$+6.3\%$	1.62	1.61	$-0.7\%$
NRI-Graph	11.48	11.08	$-3.4\%$	3.27	3.20	$-2.0\%$
MLP (no structure)	15.39	16.92	$+10.0\%$	4.78	5.03	$+5.2\%$

Appendix

Table 6: Out-of-distribution generalisation. Green’s Function shows near-perfect acoustic robustness, consistent with its structure being derived directly from the governing physics rather than from data.

$m$	MLP	Green’s	Attention	NRI-Graph
10	$15.94\pm 1.28$	$8.63\pm 0.38$	$6.26\pm 0.54$	$11.32\pm 0.65$
25	$15.54\pm 0.46$	$8.02\pm 0.28$	$5.26\pm 0.39$	$11.32\pm 0.29$
50	$17.84\pm 0.47$	$9.61\pm 0.21$	$5.76\pm 0.21$	$12.10\pm 0.27$
100	$16.68\pm 0.38$	$9.35\pm 0.17$	$5.75\pm 0.16$	$11.58\pm 0.19$
150	$15.08\pm 0.20$	$9.25\pm 0.15$	$5.45\pm 0.17$	$11.61\pm 0.15$
180	$14.57\pm 0.24$	$8.96\pm 0.07$	$5.21\pm 0.22$	$11.26\pm 0.10$

Appendix

Table 7: Full data-scaling curve (CFD domain), RMSE (m) as a function of training scenarios

m

, mean

\pm

std over 10 seeds.

Configuration	Avg. degree	Sparsity	RMSE (m)
$\tau=0.1$	1.1	97%	$11.50\pm 0.07$
$\tau\in[0.2,0.5]$	1.0	97%	$11.49\pm 0.11$
$\tau=0.7$ (disconnected)	0.0	100%	$17.42\pm 0.13$
Top-2 connectivity	2.0	94%	$14.42\pm 0.20$
Top-4 connectivity	4.0	88%	$13.79\pm 0.16$

Appendix

Table 8: NRI-Graph ablation over edge threshold

\tau

and top-

k

connectivity (CFD domain). Performance is flat across

\tau\in[0.2,0.5]

and degrades under denser top-

k

connectivity, ruling out a simple sparsity-tuning explanation.

Explore similar work

Jul 3, 2026cs.LG

Out-of-distribution Neural Inference in Dynamical Ising Models

Neural networks are increasingly used to infer hidden physical structure from dynamical observations, yet it remains unclear whether their out-of-distribution performance reflects transferable physical rule learning. We address this question in a controlled inverse problem: reconstructing interaction graphs of a kinetic Ising model from Glauber magnetization trajectories. Across convolutional, graph, Transformer, and hybrid architectures, we find that data-driven training produces distinct and reproducible statistical strategies under topology and temperature shifts. Edge-population diagnostics reveal that Transformer-based models tend to preserve the link density of the training ensemble, whereas convolutional models can collapse toward sparse- or no-link predictions that appear out-of-distribution stable by exploiting the majority no-link class. Thus, high in-distribution accuracy and apparent out-of-distribution robustness do not necessarily imply a learned dynamics-to-structure rule. Instead, neural reconstruction can be governed by architecture-dependent statistical priors. Our results identify a concrete failure mode of standard data-driven learning in physical inverse problems and motivate rule-guided principles for machine-learning-assisted scientific discovery.

Yuan-Bin Zhu, Shuang Qiao, Shi-Ju Ran

Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing 10048, China

Spatial Photonic Ising Machines Out-Of-Distribution Neural Reconstruction Magnetic Tunnel Junction

Ask PDF

Jun 10, 2026cs.LG

From Uniform to Learned Graph Priors: Diffusion for Structure Discovery

Neural relational inference (NRI) methods discover interaction graphs from trajectories through variational reasoning on discrete potential edges. However, these methods typically rely on oversimplified, factorized graph priors. Such priors, typically nearing uniform distributions, treat edges as independent entities. This systemic misalignment does not match the real-world systems and yields diffuse and indecisive edge posteriors limiting the reliability of structural discovery. To address this, we propose \textit{Diff-prior}, a diffusion-parameterized adaptive prior used to calibrate latent graph distribution rather than generate graphs. Our core insight is to reframe prior integration as a learnable denoising-style calibration that organizes scattered, uncertain edge posteriors into a more reliable overall structure which can be trained by the diffusion model. Diff-prior learns an adaptive structure prior that performs structured calibration on the edge posteriors during inference, guiding it towards a distribution closer to the underlying structure. The diff-prior operates before structural sampling and acts as a denoising calibrator directly on the encoder edge distribution, which provides a generic training paradigm over structured variables. Experiments on standard benchmarks validated our framework, and the results indicate that Diff-prior improves the performance of structure inference and generates more decisive edge posteriors across multiple NRI-family architectures. The code is available on https://github.com/Hardy158118/Diffprior.

Qi Shao, Hao Guo, Jiawen Chen +2

School of Mathematics, Southeast University Nanjing, Jiangsu, China

Prior Knowledge Integration

Ask PDF

Jun 5, 2026cs.LG

Architecture Shapes Transfer Specificity in Implicit Neural Representations

Transfer in coordinate networks is often measured by warm-start gain, but whether that gain reflects source-specific structure or generic weight reuse is less clear. We study this question across three implicit neural representation (INR) families, SIREN, ReLU MLPs, and Fourier-feature MLPs, using controlled analytic tests, a 2D lid-driven-cavity Navier--Stokes benchmark, and 1D PDE reference-solution suites for heat, viscous Burgers, and focusing cubic NLS. The analytic tests use independent-seed random controls, while the PDE benchmarks use alternate same-family source controls and auxiliary ablations. Across settings, transfer magnitude and transfer specificity separate clearly. In a 10-seed controlled 1D geometric test, Fourier Features show the largest structured transfer (

33.1\times

), followed by SIREN (

23.0\times

) and ReLU (

10.7\times

), but ReLU is far more selective: random-control transfer is

0.41\times

for ReLU versus

14.24\times

for SIREN. On a controlled two-parameter 1D family, the ranking changes: ReLU gives the clearest structured-versus-control separation at default settings, whereas Fourier Features improve only after bandwidth retuning. In Navier--Stokes and the broader 1D PDE suite, no single architecture dominates every equation, yet the same pattern remains: SIREN often reuses weights broadly, whereas ReLU and, in some equations, Fourier Features are more source-selective. Static diagnostics remain weak, and the heuristic scaling law

A_{\text{transfer}} \propto 1/Δt^2

is rejected in the implemented 1D audit. These results position transfer specificity as a useful diagnostic for coordinate networks and suggest that architecture selection in scientific machine learning should be evaluated under explicit control conditions, not by transfer magnitude alone.

D Yang Eng

Multilayer Perceptrons Reynolds-Averaged Navier-Stokes Optical Networks Warm Starts

Ask PDF