Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptation

Woojin Cho; Junghwan Park

cs.LGSep 30, 2026

Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptation

Authors: Woojin Cho, Junghwan Park

Organizations: TelePIX

Abstract

Neural operators increasingly benefit from pretraining on numerical simulations, yet adapting them for real-world prediction remains challenging. We introduce the Retain-and-Repair Neural Operator (R $^2$ NO), a framework for adapting simulation-pretrained operators to real-world data while retaining useful pretrained structure. The pretrained operator is first finetuned on real data and then frozen to provide a source prediction, and a shared repair module learns a sequence of refinements from the same observations. Using orthogonal Fourier projections, a spectral ensemble fits a small ridge regression within each cell of the Fourier domain and combines the refinements by weights fitted on a held-out split of the real data. The cells are defined jointly by radial ranges, angular sectors, and measured channels, allowing refinement depth to vary with frequency magnitude, with orientation, and across channels. Including the source prediction as a candidate makes retention available in every cell, and independently trained repair modules enter the same combination as additional candidates. On all RealPDEBench systems and six backbones, R $^2$ NO consistently outperforms full finetuning and iterative refinement. The framework treats adaptation depth as a cell-specific choice learned from real data.

Figures & tables

Panel of Figure 1 — Figure 1: Different Fourier regions favor different refinement depths. (a) A real Cylinder observation in physical space. (b) Five radial Fourier regions, ordered from low to high frequency. (c) Region-wise fRMSE change (%) relative to $\ell=0$ along an IRNO trajectory trained on FNO; markers indicate minima over the evaluated depths, and lower is better. Setup in Appendix C.1 .

Figure 2: Overall pipeline of $\text{R}^{2}\text{NO}$ . The retain-and-repair loop applies the shared repair module $L$ times to the frozen source prediction $h_{0}$ and keeps every iterate. The spectral ensemble then fits, for each Fourier cell, how much of the source prediction and of each repair to use.

		Cylinder		Controlled Cylinder		FSI		Foil		Combustion
Backbone	Adaptation	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$
U-Net	Full finetuning	6.315	9.731	0.786	0.924	0.841	0.714	0.936	0.825	2.134	2.487
	IRNO ( $\ell{=}12$ )	5.982	9.198	0.788	0.929	0.876	0.739	0.935	0.820	2.159	2.526
	IRNO ( $\ell^{\star}$ )	5.970	9.172	0.778	0.912	0.838	0.700	0.922	0.819	2.133	2.489
	$\text{R}^{2}\text{NO}$ ( $M{=}1$ )	3.510	5.181	0.776	0.907	0.836	0.697	0.911	0.797	2.121	2.466
	$\text{R}^{2}\text{NO}$ ( $M{=}3$ )	2.775	3.912	0.776	0.905	0.836	0.698	0.905	0.784	2.119	2.462

Table 1: Real-world adaptation on RealPDEBench. All adapted methods start from the same finetuned checkpoint within each backbone, IRNO is reported at its final depth and at the depth

\ell^{\star}

selected on the fitting split, and

M

counts the repair modules of

\text{R}^{2}\text{NO}

. Lower is better, and the best value of each column within a backbone is set in bold, with ties marked jointly.

		Cylinder		Controlled Cylinder		FSI		Foil		Combustion
Backbone	Combination	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$
U-Net	Mean of final iterates	5.841	9.018	0.783	0.920	0.867	0.737	0.927	0.820	2.141	2.497
	Mean of all iterates	5.973	9.211	0.778	0.912	0.841	0.702	0.922	0.820	2.134	2.490
	Global combination	5.213	8.110	0.778	0.912	0.837	0.699	0.921	0.818	2.131	2.497
	$\text{R}^{2}\text{NO}$	2.775	3.912	0.776	0.905	0.836	0.698	0.905	0.784	2.119	2.462
FNO	Mean of final iterates	4.868	7.843	0.811	0.952	1.017	0.991	1.135	1.152	2.248	2.700

Table 2: Ablation of the combination rule at a matched repair budget. All rows use the same three repair modules rolled out to

L=12

and differ only in how their iterates are combined. Lower is better, and the best value of each column within a backbone is set in bold.

Figure 3: Number of columns. Each curve is normalized by its value at thirty-seven columns, and every configuration consumes thirty-six evaluations of

\Phi

Figure 4: Ridge parameter. RMSE averaged over the five systems. Dotted lines mark the source prediction, and stars mark the values selected on the fitting split, with the number of systems selecting each.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Adaptation	Repair budget	Cylinder	Controlled Cylinder	FSI	Foil	Combustion
U-Net	Full finetuning	0	7.284	5.435	5.789	1.447	54.025
	IRNO	12	6.926	5.421	6.053	1.454	55.257
	$\text{R}^{2}\text{NO}$ ( $M{=}1$ )	12	7.652	5.317	5.753	1.414	53.840
	$\text{R}^{2}\text{NO}$ ( $M{=}3$ )	36	7.103	5.314	5.751	1.412	53.789
DeepONet	Full finetuning	0	15.031	22.837	23.675	3.751	57.215
	IRNO	12	8.468	6.678	7.640	2.869	55.707

Appendix

Table 3: Relative

L_{2}

error (

\times 10^{-2}

, lower is better) on RealPDEBench for the settings of Table 1 . The repair budget counts evaluations of

\Phi

, and the best value of each column within a backbone is set in bold.

		Cylinder		Controlled Cylinder		FSI		Foil		Combustion
Backbone	Partition	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$
U-Net	Single cell	5.213	8.110	0.778	0.912	0.837	0.699	0.921	0.818	2.131	2.497
	No radial axis	3.606	5.709	0.775	0.905	0.836	0.697	0.908	0.794	2.121	2.468
	No angular axis	3.154	4.473	0.776	0.904	0.836	0.697	0.910	0.793	2.122	2.468
	No channel axis	3.412	5.557	0.776	0.908	0.836	0.697	0.910	0.798	2.119	2.462
	Full partition	2.775	3.912	0.776	0.905	0.836	0.698	0.905	0.784	2.119	2.462

Appendix

Table 4: Removing one axis of the partition at a matched repair budget. All rows use three repair modules rolled out to

L=12

and fitted weights, and differ only in the partition. The RMSE and the fRMSE are reported, lower being better, and the best value of each column within a backbone is set in bold, with ties at the reported precision marked jointly.

		Cylinder		Controlled Cylinder		FSI		Foil		Combustion
Backbone	Base column	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$
U-Net	Without	2.805	3.953	0.776	0.904	0.836	0.697	0.911	0.788	2.128	2.480
U-Net	With	2.775	3.912	0.776	0.905	0.836	0.698	0.905	0.784	2.119	2.462
FNO	Without	3.534	4.984	0.806	0.934	0.988	0.927	1.091	1.021	2.222	2.678
FNO	With	3.432	4.919	0.804	0.934	0.985	0.927	1.061	1.019	2.201	2.659
DPOT-S	Without	2.938	3.765	0.790	0.914	0.862	0.726	1.024	1.083	2.068	2.323

Appendix

Table 5: Effect of the base column at the operating point. Both rows use three repair modules rolled out to

L=12

, fitted weights, and the full partition. The RMSE and the fRMSE are reported, lower being better, and the better value of each pair is set in bold, with ties at the reported precision marked jointly.

		Cylinder		Controlled Cylinder		FSI		Foil		Combustion
Backbone	Adaptation	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$	RMSE $(\times 10^{-2})$	fRMSE $(\times 10^{-3})$
FNO	Pretrained (source)	7.389	11.481	2.845	5.934	4.263	5.899	2.738	3.733	3.630	5.147
	Full finetuning	5.449	8.661	0.943	1.077	1.266	1.162	1.199	1.241	2.246	2.726
	IRNO ( $\ell{=}12$ )	3.194 †	4.575 †	0.890 †	1.124	0.981 †	0.927 †	1.546	1.794	2.654	3.482
	$\text{R}^{2}\text{NO}$ ( $M{=}1$ )	2.787 †	3.545 †	0.873 †	1.053 †	0.935 †	0.818 †	1.177 †	1.309	2.521	3.301
	$\text{R}^{2}\text{NO}$ ( $M{=}3$ )	2.456 †	2.981 †	0.843 †	0.997 †	0.912 †	0.773 †	1.114 †	1.151 †	2.347	2.932

Appendix

Table 6: Adaptation from the frozen simulation-pretrained checkpoint. A dagger marks a value lower than full finetuning of the same backbone. Lower is better, and the best value of each column within a backbone is set in bold.

Figure 5: Predictions of DPOT-S on one Cylinder test window at five forecast frames. Rows show the measured target, full finetuning, IRNO, and $\text{R}^{2}\text{NO}$ ; all panels share one color scale.

Figure 6: Predictions of DPOT-S on one Controlled Cylinder test window at five forecast frames. Rows show the measured target, full finetuning, IRNO, and $\text{R}^{2}\text{NO}$ ; all panels share one color scale.

Figure 7: Predictions of DPOT-S on one FSI test window at five forecast frames. Rows show the measured target, full finetuning, IRNO, and $\text{R}^{2}\text{NO}$ ; all panels share one color scale.

Figure 8: Predictions of DPOT-S on one Foil test window at five forecast frames. Rows show the measured target, full finetuning, IRNO, and $\text{R}^{2}\text{NO}$ ; all panels share one color scale.

Figure 9: Predictions of DPOT-S on one Combustion test window at five forecast frames. Rows show the measured target, full finetuning, IRNO, and $\text{R}^{2}\text{NO}$ ; all panels share one color scale.

Explore similar work

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates
Jun 15, 2026Changjian Zhou, Junfeng Fang, Negin Yousefpour +3Neural Operators Parametric Physics-Informed Neural Network
Faculty of Engineering and IT, University of Melbourne · School of Computing, National University of Singapore · Artificial Intelligence Research Institute, IFLYTEK Co., Ltd.
Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation
May 21, 2026Xiaotian Liu, Shuyuan Shang, Xiaopeng Wang +2Classical Numerical Solver Iterative Refinement
1Dartmouth College · 2CUHK Shenzhen · 3Lawrence Berkeley National Lab
Learnable composition for neural operators
Sep 2, 2026Zituo Chen, Baiming Zhang, Sili DengNeural Operators Neural Fields
MIT MechE · University of Toronto

Jun 15, 2026, cs.LGJ/K move · Enter open · S save

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Changjian Zhou, Junfeng Fang, Negin Yousefpour, Peng Wu+2

Faculty of Engineering and IT, University of Melbourne · School of Computing, National University of Singapore · Artificial Intelligence Research Institute, IFLYTEK Co., Ltd.

Neural Operators Parametric Physics-Informed Neural Network Fisher Information Matrix Sim-To-Real Gap Projection

Open paper AskPDF arXiv

Neural operator models trained on simulation data often lose accuracy when applied to experimental measurements due to the sim-to-real gap. Standard fine-tuning with limited real data can reduce this gap, but it may also damage the core physics-relevant representations learned during pretraining. Although knowledge-preserving adaptation has been widely investigated in vision or language tasks, it remains unclear whether these methods are suitable for neural operators whose architectures and protected knowledge are fundamentally different. Neural operators need to preserve core-scale physical structures rather than semantic or visual features. We propose PhysGuard, a physics-preserving framework for accurate sim-to-real adaptation of neural operators. Specifically, PhysGuard uses the empirical Fisher Information Matrix computed on simulation data to identify physics-critical parameter directions, then restricts fine-tuning updates to directions that do not interfere with them. A layer-wise Gram-matrix formulation makes this efficient for models with millions of parameters, while an adaptive threshold automatically determines the protected subspace size. A spectral probe experiment shows that the dominant Fisher directions are strongly associated with low-frequency output structures. Experiments on benchmark across four neural operator architectures and different physical systems show that PhysGuard performs strongly on most evaluation metrics compared to baselines. The benefits are most evident under severe domain shift, where it reduces low-frequency error by up to 32% compared to standard fine-tuning while maintaining adaptability. Our code is available at https://github.com/ZhouChaunge/PhysGuard.

Abstract

Explore similar work

PhysGuard: Fisher-Guided Gradient Projection for Sim-to-Real Neural PDE Surrogates

Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation

Learnable composition for neural operators