Compute-optimal data scaling for neural surrogates via multi-fidelity training

Paul Setinek; Fabian Paischer; Nils Thuerey; Johannes Brandstetter; Felix Koehler

cs.LGOct 12, 2026

Compute-optimal data scaling for neural surrogates via multi-fidelity training

Authors: Paul Setinek, Fabian Paischer, Nils Thuerey, Johannes Brandstetter, Felix Koehler

Organizations: Institute for Machine Learning, JKU Linz · Mistral AI · Technical University of Munich · Munich Center for Machine Learning (MCML)

Abstract

Neural surrogates for Partial Differential Equations (PDEs) are usually trained on synthetic data from numerical simulators. As models grow and scaling laws demand larger datasets, the cost of data generation often outweighs the training cost and becomes the bottleneck in the scaling quest. Unlike data in other domains, however, each PDE sample has a tunable cost that depends on its numerical fidelity, with the resulting cost-accuracy trade-off governed by known convergence rates. We exploit this property by formalizing neural surrogate training as a function of both data quantity and fidelity under a fixed data generation compute budget, treating fidelity as a fine-grained discrete spectrum rather than a binary distinction. Building on this, we show that multi-fidelity training enables Pareto-optimal error scaling against data generation compute. By using simulations at lower fidelities as an additional training signal, training on the optimal fidelity mixture matches the accuracy of high-fidelity-only training, with up to $6.3\times$ less data generation compute across our settings. We validate these findings across six PDE problems spanning structured grids and unstructured meshes, two distinct error sources (discretization and iterative-solver error), and four surrogate architectures, totaling over 7,000 training runs.

Figures & tables

Figure 1: Overview of our multi-fidelity data scaling approach. We split the data axis into generation budget and fidelity mixture, parametrized by $\lambda$ . Bottom: example discretization fidelity mixtures. Right: Across diverse settings, multi-fidelity training at optimal mixtures (green) reaches the error of high-fidelity-only training (orange, dashed) with up to $6.3\times$ less data generation compute.

Panel of Figure 2 — Figure 2: Data generation pipeline for neural surrogates, highlighting the primary sources of numerical error. Discretization error and solver error (blue), which exhibit known convergence behaviors and can be reduced at the expense of additional compute.

Figure 3: Overview of the multi-fidelity sampling approach. Left: Probability mass function with higher $\lambda$ biasing towards higher fidelities $\hat{\xi}$ . Right: Example dataset compositions for the linear elasticity solver error experiment.

Figure 4: Spectrum vs. binary approach for a Transolver model on the hyperelasticity dataset (discretization error) at

\tilde{B}=200

Figure 5: The multi-fidelity surrogate respects the conditioning signal. We show a Transolver model on the hyperelasticity dataset (solver error) trained at

\tilde{B}=1{,}600

and

\lambda^{\star}=-3.59

Application domain	Physics / Equation	Data format	Error source		Architectures
Application domain	Physics / Equation	Data format	Discretization	Solver	Architectures
Mathematical / PDEs	Poisson equation	1D, 2D, 3D grids	✓	✓	FNO, U-Net
Structural Mechanics	Linear elasticity	2D meshes	✓	✓	AB-UPT, Transolver
Structural Mechanics	Hyperelasticity (nonlinear)	2D meshes	✓	✓	AB-UPT, Transolver
Fluid Dynamics	RANS (airfoils)	2D meshes	✓	– ⋆	AB-UPT, Transolver

Table 1: Overview of PDE , data representations, sources of error and architectures.

Figure 6: Poisson 3D experiment results for the U-Net model . Each group shows iso-budget curves over the mixture hyperparameter $\lambda$ (left) and the compute-optimal Pareto front using $\lambda^{\star}(B)$ at each budget (right). Multi-fidelity training enables Pareto-optimal scaling for both sources of error. Results are averaged across seeds with shaded bands indicating $\pm 1$ standard deviation.

Figure 7: Hyperelasticity experiment results for the Transolver model . Multi-fidelity training also enables data savings in these cases on unstructured meshes with nonlinear PDE . Solver error savings are higher (larger distance between the two scaling curves) due to operating at a consistent resolution.

Figure 8: Airfoil experiment results on the discretization error for the AB-UPT model . Multi-fidelity training cuts simulation time by $1,820$ core-hours at a matched volumetric error of $2\%$ . The savings also extend to derived engineering quantities like the lift coefficient (right plot).

Dataset	Solver error			Discretization error
Dataset	1 st	2 nd	3 rd	1 st	2 nd	3 rd
Linear elasticity	4.0	5.2	4.4	6.3	4.5	4.0
Hyperelasticity	2.3	3.4	3.9	9.0	10.0	4.3
Airfoil	–	–	–	0.0	1.7	–

Table 2: Mean held-out regret (%) of the extrapolation recipe per dataset and fidelity axis (mesh experiments) across the first three held-out budgets (ascending, means taken over architectures).

Figure 9: Cost-savings ratio

S(\mathcal{E}^{\star})

of multi-fidelity training relative to high-fidelity training. Relative

L_{2}

error thresholds:

\mathcal{E}^{\star}=10\%

for Poisson, linear elasticity, and hyperelasticity, and

\mathcal{E}^{\star}=2\%

for the airfoil experiment (volume fields). Multi-fidelity training yields cost savings in every one of the 20 setups, up to

6.3\times

with a mean of

{\sim}3.1\times

. Bar heights are bootstrap medians and whiskers denote 95% confidence intervals, all excluding

1\times

. We describe the postprocessing and bootstrapping procedure in Section J.5 .

Appendix figures & tables50 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10: Schematic arrangement of the fidelity menu $\Xi$ on the fidelity axis. The admissible support (solid segment, five illustrative dots) sits between the two excluded tails $\Xi_{\mathrm{bottom}}$ (high-error, sub-crude: the fidelity tag carries no signal) and $\Xi_{\mathrm{top}}$ (low-error, over-solved relative to the surrogate floor). The machine-precision fidelity $\xi_{\epsilon}$ defines the far end of $\Xi_{\mathrm{top}}$ . Concrete error values on the two endpoints are the typical targets we use in practice ( $e(\xi_{\min})\approx 20\%$ and $e(\xi_{\max})\approx 0.5\%$ ).

Figure 11: Discretization error convergence for the Poisson problem on a uniform Cartesian grid in 1D, 2D, and 3D. Asymptotically, each configuration reaches the polynomially quadratic decay in the optimality error.

Figure 12: Multigrid V-cycle iterative truncation error for 1D, 2D, and 3D. Each plot uses a fixed resolution of

64

per dimension (i.e.,

64^{D}

total degrees of freedom). Different colors indicate different numbers of active modes in the right-hand side distribution.

Panel of Figure 14 — Figure 14: Discretization convergence for the linear elasticity experiment.

Figure 15: Empirical von Mises stress error convergence with iteration count

k

for the linear elasticity solver error experiment. We show the mean and

\pm 1

std across 90,000 simulations.

Figure 16: Visualization of the von Mises stress field at the lowest and highest solver fidelities for an example simulation.

Parameter	Symbol	Range
Traction magnitude	$T$	$1$ – $50$ kPa
Traction angle	$\alpha$	$-\pi/6$ – $\pi/6$
Number of holes	–	$5$ – $20$
Hole radii	$r_{i}$	$0.02$ – $0.15$

Appendix

Table 4: Sampling ranges for boundary conditions and geometry parameters in the hyperelasticity experiment.

Panel of Figure 17 — Figure 17: Discretization convergence for the hyperelasticity experiment.

Figure 18: Empirical von Mises stress error convergence with iteration count

k

for the hyperelasticity solver-error experiment. We show the mean and

\pm 1

std across 90,000 simulations.

Figure 19: Visualization of the von Mises stress field at the lowest and highest solver fidelities for an example simulation.

Parameter	Family	Range
Reynolds number	–	$2$ – $6\times 10^{6}$
Angle of attack	–	$-5^{\circ}$ – $15^{\circ}$
Max camber ( $M$ )	4-digit	$\{0,1,\ldots,7\}$
Camber position ( $P$ )	4-digit	$\{0\}\cup\{2,3,\ldots,7\}$
Max thickness ( $XX$ )	4-digit	$\{5,6,\ldots,20\}$
Design lift coeff. ( $L$ )	5-digit	$\{0,1,2,3,4\}$

Appendix

Table 5: Sampling ranges for flow conditions and airfoil geometry parameters in the airfoil experiment. Flow conditions (Reynolds, angle of attack) are sampled uniformly from continuous ranges; airfoil shape parameters are sampled uniformly and rounded to integers, following the NACA 4- and 5-digit families. For 4-digit airfoils, the camber position is set to

0

(uncambered) when the sampled value is below

1.5

, eliminating geometries with maximum camber too close to the leading edge, where NACA 4-digit parametrizations become numerically problematic.

Panel of Figure 20 — Figure 20: Discretization error convergence for the airfoil experiment. We show the median (line) and 10th-90th percentiles (shaded band) across 1,627 simulations.

Axis	$\bm{D}$	Menu $\Xi$	Cost $c(\xi)$	$e(\xi_{\min})$	$e(\xi_{\max})$	$\tilde{B}$ range
truncation	1	$\{1,2,3\}$	$K$	$\approx 20\%$	$\approx 1\%$	$10$ – $10^{4}$
truncation	2	$\{1,2,3,4,5,6,7\}$	$K$	$\approx 25\%$	$\approx 0.5\%$	$10$ – $10^{4}$
truncation	3	$\{2,3,4,5,6,7,8\}$	$K$	$\approx 13\%$	$\approx 0.4\%$	$10$ – $10^{4}$
discretization	1	$\{20,28,36,48\}$	$N^{1}$	$\approx 25\%$	$\approx 0.5\%$	$10$ – $10^{4}$
discretization	2	$\{20,32,48,64,96\}$	$N^{2}$	$\approx 30\%$	$\approx 0.5\%$	$10$ – $10^{4}$
discretization	3	$\{16,24,32,48,64,96\}$	$N^{3}$	$\approx 20\%$	$\approx 0.5\%$	$10$ – $10^{4}$

Appendix

Table 6: Fidelity menus used for the grid-based Poisson experiments. Cost

c(\xi)

is expressed in V-cycles (

K

) for truncation or native grid points (

N^{D}

) for discretization.

\tilde{B}=B/c(\xi_{\max})

is the budget expressed in units of one maximum-fidelity sample. All menus use

N_{\star}=64

as the target resolution at which the surrogate is trained and evaluated.

Group	Symbol / flag	Value
Shared backbone / conditioning
Target resolution	$N_{\star}$	$64$ (per axis, all $D$ )
Conditioning dim	$d_{\mathrm{cond}}$	$256$
Sinusoidal frequencies	–	$16$ , log-spaced in $[0.1,10]$
Conditioning MLP	–	2 hidden layers, SiLU
FiLM-U-Net

Appendix

Table 7: Architecture, optimization, and data hyperparameters shared across every grid-based Poisson run. Fidelity menus per

(D,\text{axis})

are listed in Table 6 .

Axis	Menu $\Xi$ ( $\|\Xi\|$ )	Cost $c(\xi)$	$e(\xi_{\min})$	$e(\xi_{\max})$	$\tilde{B}$ range
solver	$\{25{,}000,\,\ldots,\,262{,}144\}$ ( $11$ )	$K$	$\approx 15\%$	$\approx 0.5\%$	$100$ – $6{,}400$
discretization	$\{212,\,\ldots,\,29{,}408\}$ ( $21$ )	$N$	$\approx 25\%$	$\approx 1\%$	$100$ – $3{,}200$

Appendix

Table 8: Fidelity menus used for the linear elasticity experiments. Per-endpoint errors

e(\xi_{\min}),e(\xi_{\max})

are motivated by the target anchors from the menu-selection rule. Cost

c(\xi)

is expressed in solver iterations (

K

) for the solver axis or mesh nodes (

N

) for the discretization axis.

\tilde{B}=B/c(\xi_{\max})

is the budget expressed in units of one maximum-fidelity sample.

Axis	Menu $\Xi$ ( $\|\Xi\|$ )	Cost $c(\xi)$	$e(\xi_{\min})$	$e(\xi_{\max})$	$\tilde{B}$ range
solver	$\{20,\,\ldots,\,190\}$ ( $16$ )	$K$	$\approx 30\%$	$\approx 0.5\%$	$100$ – $6{,}400$
discretization	$\{212,\,\ldots,\,29{,}408\}$ ( $21$ )	$N$	$\approx 25\%$	$\approx 1\%$	$100$ – $3{,}200$

Appendix

Table 9: Fidelity menus used for the hyperelasticity experiments. The discretization menu is identical to that of linear elasticity ( Table 8 ). Cost

c(\xi)

is expressed in modified Newton iterations (

K

) for the solver axis or mesh nodes (

N

) for the discretization axis.

Axis	Menu $\Xi$ ( $\|\Xi\|$ )	Cost $c(\xi)$	$e(\xi_{\min})$	$e(\xi_{\max})$	$\tilde{B}$ range
discretization	$\{7{,}600,\,\ldots,\,180{,}000\}$ ( $11$ )	CPU core-hours	$\approx 10\%$	$\approx 0.5\%$	$50$ – $800$

Appendix

Table 10: Fidelity menu used for the airfoil experiments. The cost oracle is the empirically measured wall-clock simulation time, in CPU core-hours; we report the mean across samples at each fidelity level.

e(\xi_{\min})

and

e(\xi_{\max})

are the volume-field-averaged relative

L_{2}

errors at the menu endpoints.

Transolver
Group	Value
Layers	$8$
Hidden dim	$128$
Slice dim	$128$
Attention heads	$4$
MLP expansion	$4$

Appendix

Table 11: Architecture, optimization, and data hyperparameters shared across the linear elasticity and hyperelasticity experiments. The same configuration applies to both discretization error and solver error experiments.

Transolver (large)
Group	Value
Layers	$12$
Hidden dim	$256$
Slice dim	$256$
Attention heads	$8$
MLP expansion	$4$

Appendix

Table 12: Architecture, optimization, and data hyperparameters for the airfoil experiments.

Cluster	Hardware	Dim	Cells	Seeds	Runs	h / run	GPU-h
FAU	A100 80 GB	3D	42	1	42	1–3	42–126
LRZ	A100 80 GB (1g.10gb)	1D	648	6	3,888	0.04–0.1	156–389
LRZ	A100 80 GB (1g.10gb)	2D	$\approx$ 218	6	1,310	0.3–0.6	393–786
Leonardo	A100 64 GB (slower PCIe)	3D	$\approx$ 70	3	210	5.5–6	1,155–1,260
Total			$\approx$ 978		5,450		$\approx$ 1,746–2,561
Full-A100							$\approx$ 1,275–1,554

Appendix

Table 13: Compute resources used to produce all experiments related to the Poisson equation in 1D, 2D, and 3D (5,450 distinct training runs across the

(\text{axis},\text{dim},\text{arch},\lambda,B,\text{seed})

grid). The training recipe is fixed across all cells, so per-cell wallclock depends only on (dim, arch, cluster). A 1g.10gb MIG slice uses one seventh of a A100 resource. Cluster systems used : (i) FAU: Alex system at Friedrich-Alexander Universtität Nürnberg, (ii) LRZ: Leibniz Rechenzentrum AI resources MCML partition, (iii) Leonardo: Leonardo (CINECA) Boost partition

Figure 21: Poisson experiments of multi-fidelity training with discretization error. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of test error. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Figure 22: Poisson experiments of multi-fidelity training with truncation error. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of test error. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Dim	Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Dim	Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
1D	U-Net	10	-20.00	30	10	+200.0	1.113	1.325	-16.0
		30	-20.00	90	30	+200.0	0.803	1.123	-28.5
		100	-7.00	292	100	+192.0	0.295	0.770	-61.6
		300	-3.50	751	300	+150.3	0.081	0.214	-62.2
		1,000	-1.00	1,781	1,000	+78.1	0.028	0.036	-20.4

Appendix

Table 14: Summary of the Poisson experiment results (truncation/solver error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline (

\lambda\to\infty

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

Dim	Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Dim	Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
1D	U-Net	10	7.00	10	10	+0.0	1.321	1.454	-9.2
		30	-5.00	65	30	+116.7	0.752	1.232	-39.0
		100	-15.00	239	100	+139.0	0.263	0.685	-61.5
		300	-2.00	539	300	+79.7	0.107	0.204	-47.7
		1,000	20.00	1,000	1,000	+0.0	0.034	0.034	+0.0

Appendix

Table 15: Summary of the Poisson experiment results (discretization error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline (

\lambda\to\infty

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

Axis	Arch	Cluster	Hardware	Runs	h / run	GPU-h
solver	Transolver	local	A100 80 GB	147	$\sim$ 2.3	$\sim$ 340
	Transolver	local	A100 40 GB	21	$\sim$ 2.7	$\sim$ 58
	AB-UPT	local	H100 80 GB	168	$\sim$ 3.0	$\sim$ 506
discretization	Transolver	Vega	A100 40 GB	147	$\sim$ 11.0	$\sim$ 1,614
	AB-UPT	Vega	A100 40 GB	115	$\sim$ 6.6	$\sim$ 760
	AB-UPT	MeluXina	A100 40 GB	32	$\sim$ 6.3	$\sim$ 201

Appendix

Table 16: Compute resources used to produce all hyperelasticity experiments, reported as actual measured wall-clock GPU-hours per (architecture, cluster) with the mean per-run time. Partially, runs of one sweep were distributed across sites. The ablations rows cover the figure ablations on the hyperelasticity solver axis (no-conditioning,

\ell_{\min}

, binary-vs-spectrum). Cluster systems used: (i) local: in-house GPU servers (NVIDIA H100 80 GB, A100 80 GB, A100 40 GB), (ii) Karolina: IT4Innovations (A100 40 GB), (iii) Vega: IZUM (A100 40 GB), (iv) MeluXina: LuxProvide (A100 40 GB).

Figure 23: Hyperelasticity experiments. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of test error. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	100	-6.25	791	100	+691.0	0.114	0.407	-72.0
	200	-6.25	1,582	200	+691.0	0.073	0.247	-70.5
	400	-1.95	2,104	400	+426.0	0.048	0.135	-64.3
	800	-3.59	5,303	800	+562.9	0.035	0.077	-55.1
	1,600	-1.91	8,425	1,600	+426.6	0.031	0.047	-34.3

Appendix

Table 17: Summary of the hyperelasticity experiment results (solver error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	100	-2.81	4,049	104	+3793.3	0.187	0.300	-37.8
	200	-1.95	5,416	206	+2529.1	0.114	0.159	-28.1
	400	$\geq 15$	583	411	+41.8	0.061	0.065	-6.2
	800	$\geq 15$	1,140	810	+40.7	0.036	0.038	-7.1
	1,600	$\geq 15$	2,279	1,605	+42.0	0.030	0.028	+8.2

Appendix

Table 18: Summary of the hyperelasticity experiment results (discretization error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Axis	Arch	Cluster	Hardware	Runs	h / run	GPU-h
solver	Transolver	local	A100 80 GB	147	$\sim$ 2.1	$\sim$ 312
	Transolver	local	A100 40 GB	21	$\sim$ 2.8	$\sim$ 59
	AB-UPT	local	H100 80 GB	168	$\sim$ 2.9	$\sim$ 491
discretization	Transolver	local	A100 80 GB	45	$\sim$ 7.9	$\sim$ 354
	Transolver	local	A100 40 GB	56	$\sim$ 14.7	$\sim$ 822
	Transolver	Vega	A100 40 GB	46	$\sim$ 13.9	$\sim$ 639

Appendix

Table 19: Compute resources used to produce all linear elasticity experiments, reported as actual measured wall-clock GPU-hours per (architecture, cluster) with the mean per-run time. Partially, runs of one sweep were distributed across sites. Cluster systems used: (i) local: in-house GPU servers (NVIDIA H100 80 GB, A100 80 GB, A100 40 GB), (ii) Karolina: IT4Innovations (A100 40 GB), (iii) Vega: IZUM (A100 40 GB), (iv) MeluXina: LuxProvide (A100 40 GB).

Figure 24: Linear elasticity experiments. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of test error. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	100	-4.06	879	100	+779.0	0.124	0.686	-82.0
	200	-2.34	1,455	200	+627.5	0.081	0.494	-83.6
	400	-4.06	3,542	400	+785.5	0.050	0.255	-80.3
	800	-0.98	4,598	800	+474.7	0.039	0.111	-64.7
	1,600	-2.34	11,663	1,600	+628.9	0.035	0.057	-37.6

Appendix

Table 20: Summary of the linear elasticity experiment results (solver error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	100	-4.92	8,000	104	+7592.3	0.291	0.776	-62.4
	200	-1.93	5,399	206	+2520.9	0.200	0.276	-27.8
	400	$\geq 15$	565	411	+37.5	0.089	0.124	-28.4
	800	$\geq 15$	1,114	810	+37.5	0.050	0.050	+0.9
	1,600	$\geq 15$	2,229	1,605	+38.9	0.043	0.041	+4.1

Appendix

Table 21: Summary of the linear elasticity experiment results (discretization error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Arch	Cluster	Hardware	Runs	h / run	GPU-h
Transolver	local	A100 80 GB	42	$\sim$ 8.9	$\sim$ 374
Transolver	Karolina	A100 40 GB	78	$\sim$ 13.1	$\sim$ 1,023
AB-UPT	local	H100 80 GB	21	$\sim$ 12.8	$\sim$ 269
AB-UPT	Karolina	A100 40 GB	99	$\sim$ 19.9	$\sim$ 1,974
Total			240		$\sim$ 3,640

Appendix

Table 22: Compute resources used to produce all airfoil experiments, reported as actual measured wall-clock GPU-hours per (architecture, cluster) with the mean per-run time. Partially, runs of one sweep were distributed across sites. Fifteen duplicated AB-UPT HF-only runs on Vega (

\sim

328 GPU-h), superseded by the Karolina re-runs, are excluded. Cluster systems used: (i) local: in-house GPU servers (NVIDIA H100 80 GB, A100 80 GB, A100 40 GB), (ii) Karolina: IT4Innovations (A100 40 GB), (iii) Vega: IZUM (A100 40 GB), (iv) MeluXina: LuxProvide (A100 40 GB).

Figure 25: Airfoil experiments. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of volume and surface test error. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Figure 26: Airfoil experiments. Top: Iso-budget curves over mixture hyper-parameter $\lambda$ in terms of test error of aerodynamic coefficients. Minimum loss for each budget is highlighted with $\star$ . Bottom: Compute-optimal Scaling behavior using $\lambda^{*}$ at each budget.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	50	-1.46	316	52	+507.7	0.040	0.133	-69.5
	100	-1.33	634	101	+527.7	0.025	0.073	-65.2
	200	1.84	584	200	+192.0	0.017	0.036	-54.1
	400	3.05	872	397	+119.6	0.013	0.018	-29.3
	800	6.41	1,236	798	+54.9	0.011	0.013	-12.0

Appendix

Table 23: Summary of the airfoil experiment results (volume field error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	50	-1.46	316	52	+507.7	0.062	0.215	-71.1
	100	-1.33	634	101	+527.7	0.036	0.119	-69.9
	200	0.59	764	200	+282.0	0.022	0.054	-60.3
	400	1.68	1,155	397	+190.9	0.014	0.023	-37.2
	800	6.41	1,236	798	+54.9	0.011	0.012	-7.7

Appendix

Table 24: Summary of the airfoil experiment results (surface field error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	50	-2.56	385	52	+640.4	0.050	0.144	-65.6
	100	-1.33	634	101	+527.7	0.025	0.072	-64.6
	200	0.59	764	200	+282.0	0.018	0.040	-54.9
	400	2.36	1,013	397	+155.2	0.014	0.024	-41.4
	800	10.62	1,024	798	+28.3	0.012	0.014	-16.6

Appendix

Table 25: Summary of the airfoil experiment results (lift coefficient

C_{l}

error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. error
Arch	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
AB-UPT	50	-1.46	316	52	+507.7	0.181	0.629	-71.2
	100	-0.23	503	101	+398.0	0.105	0.380	-72.3
	200	1.84	584	200	+192.0	0.069	0.176	-60.8
	400	1.68	1,155	397	+190.9	0.052	0.086	-40.0
	800	$\geq 15$	880	798	+10.3	0.045	0.052	-12.6

Appendix

Table 26: Summary of the airfoil experiment results (drag coefficient

C_{d}

error). Multi-fidelity (MF) uses the optimal

\lambda^{*}

; High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples;

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Dataset	Arch	Calibration $\bm{\tilde{B}}$	Regret at holdout $\bm{\tilde{B}}$ (%)
Dataset	Arch	Calibration $\bm{\tilde{B}}$	1 st	2 nd	3 rd	4 th
Poisson 1D	FNO	10, 30, 100	13.6	10.7	8.1	5.9
Poisson 1D	U-Net	100, 233, 520	8.7	4.1	2.4	3.3
Poisson 2D	FNO	100, 300, 1000	0.1	0.3	–	–
Poisson 2D	U-Net	100, 300, 1000	12.9	3.1	–	–
Poisson 3D	U-Net	100, 300, 1000	0.0	3.1	–	–

Appendix

Table 27: Per setting recipe results on the solver error axis: regret at each holdout budget (ascending).

Dataset	Arch	Calibration $\bm{\tilde{B}}$	Regret at held-out $\bm{\tilde{B}}$ (%)
Dataset	Arch	Calibration $\bm{\tilde{B}}$	1 st	2 nd	3 rd	4 th
Poisson 1D	FNO	10, 30, 100	48.9	9.2	0.0	0.0
Poisson 1D	U-Net	30, 100, 300	27.7	28.9	0.0	–
Poisson 2D	FNO	100, 300, 1000	1.2	9.1	–	–
Poisson 2D	U-Net	100, 300, 1000	70.3	99.2	–	–
Poisson 3D	U-Net	10, 100, 300	0.0	22.8	55.6	–

Appendix

Table 28: Per-setting recipe results on the discretization-error axis: regret at each held-out budget (ladder rungs above the calibration window, in ascending order).

Panel of Figure 27 — Figure 27: Spectrum vs. binary fidelity formulation at fixed data generation budgets $\tilde{B}$ for a Transolver model on the the hyperelasticity dataset (solver error).

Panel of Figure 28 — Figure 28: Influence of training budget and fidelity mixture on interpolation smoothness. Showing Transolver models trained on the optimal fidelity mixtures $\lambda^{\star}$ at different budgets $\tilde{B}$ on the hyperelasticity dataset (solver error axis). (b) repeats Fig. 5 for comparison.

Panel of Figure 29 — Figure 29: Influence of conditioning resolution on interpolation smoothness. Showing Transolver models trained with $\tilde{B}=1{,}600$ and $\lambda^{\star}=-3.59$ across different encoding resolutions $\ell_{\min}$ .

Prolongation	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Prolongation	$\bm{\tilde{B}}$	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
Multi-linear	10	5.00	11	10	+10.0	1.422	1.454	-2.2
	100	-20.00	240	100	+140.0	0.273	0.685	-60.1
	300	-1.00	488	300	+62.7	0.114	0.204	-44.3
	1,000	$\geq 15$	1,000	1,000	+0.0	0.034	0.034	+0.0
	3,000	$\geq 15$	3,000	3,000	+0.0	0.018	0.018	+0.0

Appendix

Table 29: Interpolation ablation on the 1D U-Net Poisson experiment (discretization error). We compare the multi-linear prolongation of the main experiments with prolongation via the discrete sine transform (DST). Both branches use the same mixtures

\lambda\in\{-20,-5,-1,0,1,5,20\}

and 6 seeds, so the multi-linear values can differ slightly from Table 15 , which uses a finer

\lambda

grid. Multi-fidelity (MF) uses the optimal

\lambda^{*}

. High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples.

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline. Optima in the saturated high-fidelity regime are reported as

\lambda^{*}\geq 15

(HF-equivalent).

Base ch.	Params	NN floor	$\bm{\tilde{B}_{10\%}}$ , MF $\bm{(\lambda^{*})}$	$\bm{\tilde{B}_{10\%}}$ , HF-only	Saving @ 10%
16	0.69M	4.0%	260	715	2.8 $\times$ [2.5, 3.0]
32	2.74M	2.5%	236	745	3.2 $\times$ [2.8, 3.5]
64 (main)	10.92M	1.8%	224	758	3.4 $\times$ [3.2, 3.6]
128	43.65M	1.4%	207	654	3.2 $\times$ [2.7, 3.5]

Appendix

Table 30: Varying the parameter size of the surrogate architecture via the base channels of the U-Net (2D Poisson, solver error). NN floor is the HF-only error at the highest budget.

\tilde{B}_{10\%}

is the budget at which the MF (

\lambda^{*}

) and the HF-only scaling curves reach a relative

L_{2}

error of 10%, and the saving is their ratio, as in Fig. 9 (95% seed-bootstrap confidence interval in brackets). All widths use the mixtures

\lambda\in\{-20,-5,-1,0,1,5,20\}

and 6 seeds. With the finer

\lambda

grid of the main experiment, the saving of the 64-channel model is 3.7

\times

Steps	$\bm{\lambda^{*}}$	$\mathbf{M_{\textbf{train}}}$			Rel. $\mathbf{L_{2}}$ error
Steps	$\bm{\lambda^{*}}$	MF	HF	$\bm{\Delta}$ (%)	MF	HF	$\bm{\Delta}$ (%)
1,000	-1.00	534	300	+78.0	0.181	0.246	-26.6
2,500	-1.00	534	300	+78.0	0.153	0.239	-36.0
5,000	-1.00	534	300	+78.0	0.139	0.227	-38.7
10,000	-5.00	826	300	+175.3	0.124	0.222	-44.2
20,000 (main)	-5.00	826	300	+175.3	0.094	0.214	-56.2

Appendix

Table 31: Training duration ablation for the 1D U-Net Poisson experiment (solver error) at

\tilde{B}=300

. All step counts use the mixtures

\lambda\in\{-20,-5,-1,0,1,5,20\}

and 6 seeds, so the 20,000-step row can differ from Table 14 , which uses a finer

\lambda

grid. Multi-fidelity (MF) uses the optimal

\lambda^{*}

. High-fidelity (HF) is the high-fidelity-only baseline.

M_{\mathrm{train}}

denotes the number of training samples.

\Delta(\%)

denotes the relative change in number of training samples and error compared to the high-fidelity baseline.

$\bm{\tilde{B}_{c}}$	$\bm{K_{\max}=3}$	$\bm{K_{\max}=5}$	$\bm{K_{\max}=6}$	$\bm{K_{\max}=7}$ (main)	$\bm{K_{\max}=9}$
100	14.94 (-62.4)	15.97 (-68.7)	18.00 (-68.3)	20.64 (-62.5)	18.98 (-72.5)
300	7.78 (-22.8)	6.90 (-53.7)	7.36 (-64.2)	7.75 (-70.9)	8.65 (-78.2)
1,000	4.67 (-15.0)	3.21 (-46.3)	3.11 (-53.8)	3.19 (-57.5)	3.43 (-60.1)
3,000	4.14 (-0.4)	2.12 (-14.6)	2.07 (-25.7)	2.12 (-32.5)	2.11 (-45.3)
10,000	4.06 (-0.8)	1.84 (+0.0)	1.76 (-1.6)	1.77 (+0.0)	1.77 (-3.9)

Appendix

Table 32: Endpoint ablation for the 2D Poisson U-Net experiment (solver error). Each entry reports the mean relative

L_{2}

error (%) at

\lambda^{*}

and, in parentheses, the relative change

\Delta(\%)

in error compared to HF-only training within the same menu. Budgets

\tilde{B}_{c}

are normalized by a fixed cost anchor (

K=7

), and all menus are evaluated against the same DST-exact reference. All menus use the mixtures

\lambda\in\{-20,-5,-1,0,1,5,20\}

and 6 seeds, so the

K_{\max}=7

column differs from Table 14 , which uses a finer

\lambda

grid and the menu top as reference.

	Error at $\bm{\lambda^{*}}$ (%)			$\bm{\Delta}$ (%)
$\bm{\tilde{B}}$	menu top	conv. ref	rel. diff.	menu top	conv. ref
100	20.56	20.64	+0.42%	-62.7	-62.5
300	7.73	7.75	+0.25%	-71.0	-70.9
1,000	3.18	3.19	+0.25%	-57.6	-57.5
3,000	2.12	2.12	+0.08%	-32.4	-32.5
10,000	1.77	1.77	+0.51%	+0.0	+0.0

Appendix

Table 33: Results for evaluating the surrogates not only against their best in-support label at

\xi_{\max}

, but also against the converged reference

\xi\to\infty

for the 2D U-Net Poisson experiment on the solver error axis (

K_{\max}=7

runs of Table 32 ). Errors are mean relative

L_{2}

errors at

\lambda^{*}

over 6 seeds; rel. diff. is computed from unrounded values.

\Delta(\%)

denotes the relative change in error at

\lambda^{*}

compared to the high-fidelity baseline within the menu.

Explore similar work

Operator Boosting Produces Pareto-Efficient PDE Surrogates
Jun 16, 2026Lennon J. ShikhmanPDE Surrogate Modeling Deep Operator Networks
Georgia Institute of Technology College of Computing Atlanta, Georgia, USA
Performance of Neural and Polynomial Operator Surrogates
Apr 1, 2026Josephine Westermann, Benno Huber, Thomas O'Leary-Roseberry +1Surrogate Modeling PDE Surrogate Modeling
Heidelberg University · The Ohio State University
The Symbol of the Surrogate: Measuring Numerical Provenance in Neural PDE Solvers
Oct 7, 2026Ridham PatelPDE Surrogate Modeling
Computer Science & Engineering Indian Institute of Technology Gandhinagar Gandhinagar, India

Jun 16, 2026J/K move · Enter open · S save

Operator Boosting Produces Pareto-Efficient PDE Surrogates

Lennon J. Shikhman

Georgia Institute of Technology College of Computing Atlanta, Georgia, USA

PDE Surrogate Modeling Deep Operator Networks Neural Network Compression Neural Operators

Open paper AskPDF arXiv

Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows. This work introduces Operator Boosting, a stagewise residual-learning framework for constructing compact neural-operator surrogates directly, rather than training a large model and compressing it afterward. Starting from the empirical mean predictor in normalized output coordinates, the method trains a sequence of tiny same-family neural operators on residual fields and incorporates each correction through validation-selected shrinkage. We instantiate the framework with Fourier neural operators (FNOs), DeepONets, and convolutional neural operators (CNOs), and compare boosted tiny stacks against full-size monolithic baselines across one-, two-, and three-dimensional PDE benchmarks from PDEBench, APEBench, and The Well. Across 30 dataset-architecture pairs, 21 show positive mean accuracy gains and 17 have positive confidence intervals, while all boosted stacks reduce trainable parameter count by approximately 72-95%. Best-model comparisons show empirical Pareto improvements on 7 of 10 completed PDE benchmarks, including two-dimensional Navier-Stokes, shallow-water dynamics, Darcy flow, one-dimensional transport and reaction systems, and three-dimensional compressible Navier-Stokes. These results show that Operator Boosting often improves the empirical accuracy-parameter Pareto frontier of neural PDE surrogates, while also exposing PDE- and architecture-dependent regimes where residual boosting fails to offset compression.

Abstract

Explore similar work

Operator Boosting Produces Pareto-Efficient PDE Surrogates

Performance of Neural and Polynomial Operator Surrogates

The Symbol of the Surrogate: Measuring Numerical Provenance in Neural PDE Solvers