In-Context Learning for Multi-Fidelity Multiscale Model Closure and Cross-Operator Out-Of-Distribution Generalization

Xinyuan Li; Matthew Zahr; Panos Stinis; Zecheng Zhang

math.NAOct 28, 2026

In-Context Learning for Multi-Fidelity Multiscale Model Closure and Cross-Operator Out-Of-Distribution Generalization

Authors: Xinyuan Li, Matthew Zahr, Panos Stinis, Zecheng Zhang

Organizations: Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, IN 46556, United States. · Advanced Computing, Mathematics, and Data Division, Pacific Northwest National Laboratory, Richland, WA 99354, USA.

Abstract

Multi-fidelity learning uses low-fidelity information (coarse-scale PDE models) together with limited high-fidelity data (fine-scale solutions) to obtain high-accuracy PDE solutions. It is a data-driven approach to downscale reduced-order models to solve PDEs efficiently and accurately. Standard multi-fidelity learning focuses on one operator or PDE; given a new PDE, retraining using additional coarse-fine solution pairs may be required, which is costly. In this work, we propose an in-context multi-fidelity learning framework for learning multiple multi-fidelity operators with one shared model. The model is trained using paired low- and high-fidelity examples as demonstrations from multiple operators, together with a new coarse-scale solution query, to predict the corresponding fine-scale solution. Once trained, given a new PDE operator, a few (fewer than five) coarse-fine demonstration pairs, and a low-cost coarse query, the model predicts the corresponding fine-scale solution without fine-tuning. When the target operator appears in the training dataset but has a different prediction goal, such as predicting beyond the training time interval, no additional demonstrations are needed. Instead, the model uses training coarse-fine pairs from that PDE and information from other training operators. Testing then only requires generating a low-cost coarse-scale solution query, without fine-tuning. Notably, our smallest model contains only 255,364 trainable parameters, corresponding to approximately 456 parameters per training operator for 560 operators.

Figures & tables

Figure 1: Overview of the proposed in-context multi-fidelity learning framework. Paired coarse–fine demonstrations from a target operator and a coarse query are arranged into an ordered sequence. A Transformer predicts the missing fine-solution row corresponding to the query through a one-row shift, while the same shared trainable parameters are used across the operator family.

Flux	Regime	Test cases	$\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$R^{2}$
Overall	—	1024	$\mathbf{39.33\pm 26.41}$	$\mathbf{3.66\pm 2.15}$	$\mathbf{0.9945\pm 0.0044}$
$u^{2}/2$	Viscous	111	$28.77\pm 19.61$	$1.97\pm 0.98$	$0.9945\pm 0.0047$
$u^{2}/2$	Inviscid	120	$14.25\pm 4.73$	$4.66\pm 1.63$	$0.9896\pm 0.0043$
$u^{3}/3$	Viscous	132	$48.09\pm 20.74$	$2.16\pm 0.95$	$0.9953\pm 0.0048$
$u^{3}/3$	Inviscid	114	$13.52\pm 2.11$	$5.66\pm 1.14$	$0.9914\pm 0.0031$
$\sin(u)$	Viscous	139	$51.58\pm 17.90$	$2.19\pm 0.87$	$0.9968\pm 0.0025$

Table 1: Unseen-operator prediction across the eight scalar conservation-law families using five same-operator coarse–fine demonstrations. The table reports the coarse–fine relative

L^{2}

discrepancy, the ICL prediction error, and

R^{2}

Figure 2: Representative predictions for unseen testing scalar conservation-law operators. The left and right columns show viscous and inviscid linear-flux examples, respectively. The top row shows the query initial condition in black and the demonstration initial conditions as faded curves. The bottom row shows the solutions at

t=2

. Faded curves show the coarse–fine demonstrations. The blue dashed curve shows the coarse query, the orange curve shows the fine target, and the green curve shows the ICL prediction. The relative

L^{2}

prediction error is reported above each solution panel.

Figure 3: Context scaling and intervention results for the scalar conservation laws. Panel (a) shows the family-wise mean prediction error normalized by its value with one demonstration. Panel (b) compares the mean prediction errors under correct, wrong-operator, and wrong-

\mathbf{v}_{f}

contexts. The wrong-operator context replaces all demonstration pairs with pairs from another unseen operator in the same PDE family. The wrong-

\mathbf{v}_{f}

context retains the demonstrated coarse solutions and replaces their fine counterparts. All evaluations use the same trained model and fixed test queries without parameter updates.

Setting	$q_{1}$ range	Test cases	$\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$R^{2}$
Base range	$[0.50,2.00]$	1024	$39.33\pm 26.41$	$3.66\pm 2.15$	$0.9945\pm 0.0044$
Lower OOD	$[0.45,0.50)$	498	$45.52\pm 32.41$	$5.01\pm 2.79$	$0.9929\pm 0.0054$
Upper OOD	$(2.00,2.20]$	526	$31.95\pm 22.92$	$3.80\pm 2.34$	$0.9910\pm 0.0090$
Overall OOD	$[0.45,0.50)\cup(2.00,2.20]$	1024	$\mathbf{38.55\pm 28.75}$	$\mathbf{4.39\pm 2.64}$	$\mathbf{0.9919\pm 0.0075}$

Table 2: Operator-parameter OOD prediction for

q_{1}

. The base-range results correspond to unseen operators with

q_{1}\in[0.50,2.00]

, while the lower- and upper-OOD results use operators outside this range. Each prediction uses five same-operator coarse–fine demonstrations. The table reports the coarse–fine relative

L^{2}

discrepancy, the ICL prediction error, and

R^{2}

, with results shown as mean

\pm

standard deviation.

	Training		Evaluation
	Group A	Group B	Group A	Group B
Base amplitude $A\in[0.80,1.50]$	$\checkmark$	$\checkmark$	–	–
Near-OOD amplitude $A\in(1.50,1.57]$	$\checkmark$	–	–	$\checkmark$
Farther-OOD amplitude $A\in(1.57,1.65]$	$\checkmark$	–	–	$\checkmark$

Table 3: Training and evaluation setup for the initial-condition amplitude OOD experiment. Group A is trained on base-, near-OOD, and farther-OOD amplitudes, while Group B is trained only on the base-amplitude range. Group B is then evaluated on the near- and farther-OOD amplitude ranges using its base-amplitude coarse–fine pairs as demonstrations.

Figure 4: Representative initial conditions for the amplitude-OOD experiment. The left and right panels show narrow and wide profiles, respectively. Each profile is shown at base, near-OOD, and farther-OOD amplitudes in blue, orange, and green, respectively.

Query regime	$A$ range	Test cases	Relative $L^{2}$ error (%)			$R^{2}$
			LF baseline	Base-amplitude-only model	Mixed-amplitude model	Base-amplitude-only model	Mixed-amplitude model
			$\displaystyle\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$
Near OOD	$(1.50,1.57]$	80,000	$31.66\pm 23.78$	$3.88\pm 2.37$	$\mathbf{3.57\pm 2.21}$	$0.9919\pm 0.0075$	$\mathbf{0.9932\pm 0.0058}$
Farther OOD	$(1.57,1.65]$	80,000	$30.95\pm 23.76$	$4.18\pm 2.74$	$\mathbf{3.65\pm 2.30}$	$0.9901\pm 0.0099$	$\mathbf{0.9925\pm 0.0067}$

Table 4: Group B amplitude-OOD prediction results. Each row contains 80,000 queries from 800 Group B operators. The base-amplitude-only model is trained with base-amplitude data from both groups, while the mixed-amplitude model additionally uses near- and farther-OOD amplitude data from Group A. Both models use the same previously available Group B base-amplitude demonstrations when predicting the Group B OOD queries.

Setting	Test cases	$\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$R^{2}$
Unseen operators	960	$29.23\pm 35.25$	$4.39\pm 5.96$	$0.994$

Table 5: Prediction accuracy on 60 unseen testing KS operators using five same-operator, same-trajectory coarse–fine demonstrations. The coarse query, fine target, and ICL prediction correspond to the same query time.

R^{2}

is reported as the mean over the 960 fixed test cases.

Figure 5: Representative predictions for unseen testing KS operators. The panels show low-, average-, and high-error examples. Faded curves show the coarse–fine demonstrations. The blue dashed curve shows the coarse query, the orange curve shows the fine target, and the green curve shows the ICL prediction. The relative

L^{2}

prediction error is reported above each panel.

Figure 6: Context scaling and intervention results for KS. Panel (a) shows the mean prediction error normalized by its value with one demonstration. Panel (b) compares the mean prediction errors under correct, wrong-operator, and wrong-

\mathbf{v}_{f}

contexts. The wrong-operator context replaces all demonstration pairs with pairs from another unseen operator. The wrong-

\mathbf{v}_{f}

context retains the demonstrated coarse solutions and replaces their fine counterparts. All evaluations use the same trained model and fixed test queries without parameter updates.

Setting	$\beta$ range	Test cases	$\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$R^{2}$
Base range	$[0.070,0.085]$	960	$29.23\pm 35.25$	$4.39\pm 5.96$	$0.9942\pm 0.0376$
OOD	$(0.085,0.100]$	512	$17.70\pm 25.52$	$\mathbf{4.63\pm 4.72}$	$\mathbf{0.9953\pm 0.0208}$

Table 6: Operator-parameter OOD prediction for the KS equation. The base-range results correspond to unseen operators with

\beta\in[0.070,0.085]

, while the OOD results use unseen operators with

\beta\in(0.085,0.100]

. Each OOD prediction uses five same-operator, same-trajectory coarse–fine demonstrations.

	Training		Evaluation
	Group A	Group B	Group A	Group B
Base time $t\in[2.505,3.000]$	$\checkmark$	$\checkmark$	–	–
Temporal OOD $t\in[3.002,3.100]$ or $t\in[3.005,3.250]$	$\checkmark$	–	–	$\checkmark$

Table 7: Data allocation for each mixed-time KS temporal OOD experiment. Group A is trained on both base-time and temporal OOD data, while Group B is trained only on base-time data. Group B is then evaluated at temporal OOD times using its base-time coarse–fine demonstrations as context.

Query regime	$t$ range	Test cases	Relative $L^{2}$ error (%)			$R^{2}$
			LF baseline	Base-time-only model	Mixed-time model	Base-time-only model	Mixed-time model
			$\displaystyle\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$
Temporal OOD	$[3.002,3.100]$	100,000	$28.77\pm 34.02$	$2.90\pm 2.90$	$\mathbf{2.70\pm 2.67}$	$0.9982$	$\mathbf{0.9984}$
Temporal OOD	$[3.005,3.250]$	100,000	$30.52\pm 35.34$	$6.62\pm 9.02$	$\mathbf{5.08\pm 7.02}$	$0.9865$	$\mathbf{0.9919}$

Table 8: Cross-operator temporal OOD prediction on 100 Group B KS operators for two evaluation intervals. Each row contains 100,000 paired test cases from 20 trajectories and 50 OOD query times per operator. The base-time-only model is trained on base-time data from Groups A and B, while the mixed-time model additionally uses temporal OOD data from Group A. Group B OOD states are not used during training. Each prediction uses five same-operator, same-trajectory base-time coarse–fine demonstrations. Relative

L^{2}

errors are reported as mean

\pm

standard deviation, and

R^{2}

is reported as the mean.

Setting	Test cases	$\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)	$R^{2}$
Unseen operators	960	$20.18\pm 6.77$	$\mathbf{3.33\pm 3.52}$	$0.9977\pm 0.0070$

Table 9: Prediction accuracy on 60 unseen testing KdV operators using five same-operator coarse–fine demonstrations. Here

\mathbf{v}_{c}

denotes the coarse query at source time

t_{q}

, while

\mathbf{v}_{f}

and

\widehat{\mathbf{v}}_{f}

denote the fine target and ICL prediction at

t_{q}+0.5

, respectively.

Figure 7: Representative predictions for unseen testing KdV operators. The panels show low-, average-, and high-error examples. Faded curves show the coarse–fine demonstrations. The blue dashed curve shows the coarse query, the orange curve shows the fine target, and the green curve shows the ICL prediction. The relative

L^{2}

prediction error is reported above each panel. The coarse query is evaluated at

t_{q}

, while the fine target and prediction correspond to

t_{q}+0.5

Figure 8: Context scaling and intervention results for KdV. Panel (a) shows the mean prediction error normalized by its value with one demonstration. Panel (b) compares the mean prediction errors under correct, wrong-operator, and wrong-

\mathbf{v}_{f}

contexts. The wrong-operator context replaces all demonstration pairs with pairs from another unseen operator. The wrong-

\mathbf{v}_{f}

context retains the demonstrated coarse solutions and replaces their fine counterparts. All evaluations use the same trained model and fixed test queries without parameter updates.

Method	Test cases	$\displaystyle\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)			MSE $(\mathrm{m}/\mathrm{yr})^{2}$
		Mean	Median	P90
LF baseline	50	27.35	27.60	29.84	14.302
Method	Test cases	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)			MSE $(\mathrm{m}/\mathrm{yr})^{2}$
		Mean	Median	P90
Non-composite MF	50	7.70	7.03	12.15	2.171
Composite MF	50	7.14	6.84	10.95	1.787

Table 10: Multi-fidelity prediction on 50 ice-sheet test cases at

t=50

that are not used for training. All learned methods use the same 20 high-fidelity velocity training samples. ICL and the non-composite model use 20 paired low-fidelity samples, while the composite model uses 100 low-fidelity samples. The upper block reports the coarse–fine discrepancy of the low-fidelity baseline, while the lower block reports the prediction errors of the learned multi-fidelity models. Both blocks report the mean, median, and 90th-percentile relative

L^{2}

error together with the MSE.

Evaluation	Test cases	$\displaystyle\frac{\\|\mathbf{v}_{c}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)
		Mean	Median	P90
LF baseline	1350	30.62	30.85	33.27
Evaluation	Test cases	$\displaystyle\frac{\\|\widehat{\mathbf{v}}_{f}-\mathbf{v}_{f}\\|_{2}}{\\|\mathbf{v}_{f}\\|_{2}}$ (%)
		Mean	Median	P90
ICL, correct context	1350	1.15	0.95	1.86
ICL, wrong-operator context	1350	1.49	1.06	2.63

Table 11: Unseen-operator prediction on 1,350 test cases from 30 unseen testing ice-sheet operators using five demonstrations. The upper block reports the coarse–fine discrepancy of the low-fidelity baseline, while the lower block reports the ICL prediction error with the correct and wrong-operator contexts. In the wrong-operator context, the query is unchanged while all five demonstrations are replaced by coarse–fine pairs from another unseen testing operator. Both blocks report the mean, median, and 90th-percentile relative

L^{2}

error.

Figure 9: Representative predictions for unseen testing two-dimensional ice-sheet operators. The panels show low-, average-, and high-error examples. The coarse query, fine target, and ICL prediction are overlaid as blue, orange, and green velocity vectors, respectively. The relative

L^{2}

prediction error is reported above each panel.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Application	Auxiliary-row quantities	$n_{a}$	$L_{D}$	Max. length
Scalar conservation laws	$[x]$	1	$3D+2$	17
KS	$[x,t]$	2	$4D+3$	23
KdV	$[x]$	1	$3D+2$	17
Ice sheet, reference	$[XY,\beta,H]$	3	$5D+4$	29
Ice sheet, multi-operator	$[XY,t]$	2	$4D+3$	23

Appendix

Table 12: Application-specific input-sequence structure. The maximum length corresponds to

D=5

demonstrations. For KdV, the demonstrated fine row is evaluated at the shifted target time

t+\Delta t

Multifidelity Modeling

Multifidelity modeling combines inexpensive approximate models with costly accurate simulations or measurements to improve prediction and decision-making.

See all 36 papers

Aug 25, 2026

The Frame Kernel Method for Multiscale Operator Learning

We present a natively multiscale operator learning method for the surrogate modeling of (numerical solvers for) multiscale partial differential equations (PDEs). The primary novelty of our method lies in a novel multiscale kernel frame function approximation technique. Leveraging this new kernel frame technique, we cast the operator learning problem as one of learning frame coefficients of output functions as a function of frame coefficients of input functions. The generalization step then automatically allows for a multiscale decomposition of the output functions. Our method is applicable to both tensor-product grids and point clouds. We present interpolation proofs, error estimates, and numerical convergence rates for our frame approximation. We the demonstrate the applicability of our method for the surrogate modeling of inherently multiscale PDEs. The new multiscale frame kernel method is significantly more accurate than popular neural operators on challenging problems from the literature, while simultaneously admitting an a posteriori multiscale decomposition upon generalization.

Branden Frieden, Ryan Whitehead, M. Keith Ballard +2

Kahlert School of Computing, University of Utah, Salt Lake City, UT 84112, USA. · U.S. Air Force Research Laboratory, Wright-Patterson AFB, OH 45433, USA.

PDE Surrogate Modeling Kernel Methods PDE Operator Learning Operator Learning

Ask PDF

Mar 2, 2026

Tackling multiphysics problems via finite element-guided physics-informed operator learning

This work presents a finite element-guided physics-informed operator learning framework for multiphysics problems with coupled partial differential equations (PDEs) on arbitrary domains. The proposed framework learns an operator from the input space to the solution space with a weighted residual formulation based on the finite element method, enabling discretization-independent prediction beyond the training resolution without relying on labeled simulation data. The present framework for multiphysics problems is implemented in Folax, a JAX-based operator learning platform, and is verified on nonlinear coupled thermo-mechanical problems. Two- and three-dimensional representative volume elements with varying heterogeneous microstructures, and a close-to-reality industrial casting example under varying boundary conditions are investigated as the example problems. We investigate the potential of several neural operators combined with the proposed finite element-guided approach, including Fourier neural operators (FNOs), deep operator networks (DeepONets), and a newly proposed implicit finite operator learning (iFOL) approach based on conditional neural fields. The results demonstrate that FNOs yield highly accurate solution operators on regular domains, where the global features can be efficiently learned in the spectral domain, and iFOL offers efficient parametric operator learning capabilities for complex and irregular geometries. Furthermore, studies on training strategies, network decomposition, and training sample quality reveal that a monolithic training strategy using a single network is sufficient for accurate predictions, while training sample quality strongly influences performance. Overall, the present approach highlights the potential of physics-informed operator learning with a finite element-based loss as a unified and scalable approach for coupled multiphysics simulations.

Yusuke Yamazaki, Reza Najian Asl, Markus Apel +2

Graduate School of Science and Technology, Keio University, Hiyoshi3-14-1,Kohoku-ku, Yokohama 223-8522, Japan · Chair of Structural Analysis, Technical University of Munich, Arcisstraße 21, 80333 Munich, Germany · ACCESS e.V., Intzestr. 5, 52072 Aachen, Germany +1

Multiphysics Simulation Physics-Informed Neural Operators Physics-Based Simulation Deep Operator Networks Fourier Neural Operator Implicit Neural Representations

Ask PDF

Aug 12, 2026

Kernel Methods for Learning Operators with Multiple Inputs and Outputs

Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation, representation, learning, and reconstruction. We develop this framework for multi-input, multi-output operator learning, where operators map between products of potentially distinct function spaces. Our approximation theory shows that, although the number of inputs and outputs can increase, the convergence rate is governed by the most challenging constituent approximation problem rather than the overall problem dimension. The framework leads to practical kernel methods with closed-form training and inference, combining mathematical tractability with computational efficiency. We further specialize the approach to multiple operator learning by introducing KernelMO, a family of kernel methods with complementary operator-valued and product-space formulations. Across five families of parametric partial differential equations, the proposed methods achieve competitive or state-of-the-art predictive accuracy while reducing training and inference costs relative to neural operator architectures and deep learning based models, offering an efficient and lightweight alternative.

Adrien Weihs, Chunyang Liao, Jingmin Sun +1

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA. · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701, USA. · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA.

Kernel Methods PDE Operator Learning Operator Learning

Ask PDF

Abstract

Multifidelity Modeling

Explore similar work

The Frame Kernel Method for Multiscale Operator Learning

Tackling multiphysics problems via finite element-guided physics-informed operator learning

Kernel Methods for Learning Operators with Multiple Inputs and Outputs