Transferability of Learned States in Neural PDE Solvers

Shunye Wang; Haochen Wen; Shuo Li Liu; Xuanyi Wang; Lihao Liu; Zhongying Deng

cs.LGOct 7, 2026

Transferability of Learned States in Neural PDE Solvers

Authors: Shunye Wang, Haochen Wen, Shuo Li Liu, Xuanyi Wang, Lihao Liu, Zhongying Deng

Organizations: University of Chicago · University of Cambridge · Princeton University · ByteDance Inc. · Amazon

Abstract

Assessing useful reuse in neural PDE solvers is challenging: final accuracy can reflect source learning and target-time computation. Our reuse contract separates solution accuracy, learning contribution, and numerical utility through paired state comparisons, matched target information and budgets, and cost accounting. A literature audit extracts 18 version-specific protocol records from 12 papers, documenting retained states, target-time resources, and reported controls. For a fixed linear system and residual tolerance, we construct two initial guesses with identical solution-error, energy-error, and residual norms, reaching the same solution with different conjugate-gradient (CG) iteration counts. Across 240 source-training trajectories, two linear PDE families, Fourier neural operators and convolutional networks, a fixed predictor's benefit reverses across correction algorithms. Among pairs with both relative prediction errors less than or equal to 5 percent on 64 in-distribution tasks (63 by 63 interior grids), reductions in all three norms accompany more CG iterations, at mean taskwise rates of 23.5 percent and 23.9 percent in two libraries. Work-based selection saves 2.50-3.33 CG iterations on held-out in-distribution tasks; matched adaptation demonstrates finite-budget pretraining value. Independent batches confirm a 0.73 percent complete online saving for one physics-trained Fourier neural operator against zero-initialized Poisson-preconditioned CG. Reuse requires matched state comparisons and downstream computational evidence.

Figures & tables

Comparison	Test units and retained states	Target information and outcome	Evidence status
Corrector reversal	16 joint tasks; 2 source datasets, 3 seeds each	Source-only selection; same $10^{-8}$ true residual; update counts	Exploratory paired comparison; second source supports direction
Fresh-task selection	64 test tasks/stratum; 2 libraries of 12 models	Same 32 validation labels/stratum; choices locked before test generation	Revision-stage frozen protocol; no new training or universal time claim
Adaptation	128 tasks/equation; pretrained, scratch, frozen	Matched training/validation labels; fixed-budget prediction error	Controlled comparisons; fixed-80-label selection study is separate
Online batch gain	8 new groups of 32; one fixed model state and node	No retraining; full warm online time; $10^{-8}$ acceptance	Fresh-task confirmation after two discovery groups; groups not pooled
Official HINTS	750 official cases; timing on a 32-case subset	Fixed official weights; one-time versus repeated neural correction	Native 29-interior-DOF check; no added source training
Scaled official NOWS	48 tasks; three trained states	Official VINO objective; 100 epochs; matched zero/coarse controls	Reduced-budget native-code audit; 9,216 timed records

Table 1: Primary comparisons and their inferential scope. Counts are physical tasks or batch groups.

Figure 1: Contribution is conditional on how state is used. Left: 6000-update data-FNO update savings versus zero under joint shift, $n=31$ , tolerance $10^{-8}$ . Right: pretrained-minus-scratch advantage expressed as error reduction, source 0, 256 training plus 64 validation labels, 128 test tasks per equation. Budget points share trajectories. Intervals are descriptive paired-task 95% intervals.

Equation	Corrector	Source set 0	Source set 1
Diffusion	CG	-8.06 [-10.44, -5.71]	-8.27 [-10.77, -5.71]
Diffusion	Poisson-PCG	+1.125 [0.67, 1.58]	+1.29 [0.83, 1.73]
Diffusion	AMG-PCG	+0.23 [0.04, 0.44]	+0.29 [0.10, 0.48]
Advection–diffusion	GMRES	+34.42 [19.08, 50.38]	+41.88 [27.56, 57.77]
Advection–diffusion	Poisson-GMRES	-0.29 [-1.67, 1.06]	+0.46 [-0.56, 1.85]
Advection–diffusion	ILU-GMRES	+0.00 [0.00, 0.00]	+0.00 [0.00, 0.00]

Table 2: One retained predictor, different correction effects. Updates saved relative to zero; positive favors the data-trained FNO. Joint shift,

n=31

, tolerance

10^{-8}

, 6000-update budget. Each cell averages three optimizer seeds within each of 16 physical tasks; brackets are paired-task 95% bootstrap intervals.

Tasks	Grid	Source	CG	Poisson-PCG
ID	31	0	0.000 [0.000, 0.000]	0.125 [0.047, 0.219]
ID	31	1	-0.516 [-1.438, 0.391]	0.016 [-0.078, 0.109]
ID	63	0	2.500 [1.047, 3.844]	0.328 [0.219, 0.453]
ID	63	1	3.328 [1.531, 5.016]	0.375 [0.266, 0.500]
Joint	31	0	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]
Joint	31	1	-0.156 [-1.266, 0.891]	-0.094 [-0.234, 0.031]

Table 3: New-task conditional model selection. Test updates saved by selecting on validation work rather than prediction error (prediction minus work; positive is better). Paired-task 95% intervals use 64 test tasks per distribution. Source libraries 0/1 each contain 12 frozen candidates. No multiplicity adjustment.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Claim	Required observation or comparison	Not established by that observation alone
Output correctness	Independent acceptance and the stated reference/continuum checks	A benefit attributable to source learning
Predictive reuse	Frozen prediction versus an appropriate source-independent prediction control	Reduced numerical correction work
Finite-budget adaptation value	Pretrained versus same-architecture scratch, with matched labels and selection opportunities	An irreducible asymptotic error advantage
Initialization value	Learned versus zero/coarse under the same corrector and tolerance	Advantage under another corrector or lower total runtime
Complete solver utility	Accepted full paths versus competent alternatives, with matched workload and reuse permissions	Lifecycle superiority when offline costs are unaccounted for

Appendix

Table 4: The control determines the claim. All paired rows require declared target information, budgets, and evaluation rules. These are complementary comparisons, not interchangeable proofs of neural utility.

Quantity	Initial wave	Expanded wave	Total
Source-training trajectories	48	192	240
Target-adaptation trajectories	36	108	144
Main correction endpoints	34368	460224	494592
Independent main tasks	256	128	384
Timing repetitions	2184	37440	39624
Spectral records	64	41463	41527

Appendix

Table 5: Completed core waves before the deployment and nonlinear extensions. These are counts of records or trajectories, not interchangeable sample sizes.

Equation	Corrector	Tolerance	Failed	Total
Advection–diffusion	GMRES	$10^{-8}$	953	2336
Advection–diffusion	GMRES	$10^{-6}$	476	2336
Advection–diffusion	GMRES	$10^{-4}$	132	2336
Diffusion	CG	$10^{-8}$	1009	2336
Diffusion	CG	$10^{-6}$	21	2336

Appendix

Table 6: All failed expanded-wave main endpoints. All other family/grid/corrector/tolerance groups pass; failures remain in their original denominators.

Record	Source locator	Retained object / target process	Existing evidence and scope
P01	( Li et al., 2024 ) . 2111.03794v4; Sec.3.1; Table 1; Appendix C	operator weights. coefficient or initial field; prescribed PDE; forward prediction.	data-only versus data-plus-PDE training. not a uniform claim across equations.
P02	( Li et al., 2024 ) . 2111.03794v4; Sec.3.2; Appendix C	weights and optional pretrained anchor. target residual and conditions; instance optimization.	predictions before/after optimization. anchor state must be included in a reset.
P03	( Li et al., 2024 ) . 2111.03794v4; Table 4	operator pretraining. target Kolmogorov equation; target optimization.	zero versus nonzero source training budgets. not an all-budget matched-protocol effect.
P04	( Zhang et al., 2024 ) . 2208.13273v2; Sec.2.2; Fig.2; Sec.2.5	DeepONet correction. target operator and residual; alternating neural and numerical updates.	relaxation; one-time initializer; multigrid comparisons. spectral complementarity is established prior work.
P05	( Rudikov et al., 2024 ) . 2402.05598v1; Sec.3.3; Tables 1–3	neural nonlinear preconditioner. matrix and residual; flexible conjugate gradients.	classical preconditioners; loss and residual-sampling ablations. iterations are not complete runtime accounting.
P06	( Rudikov et al., 2024 ) . 2402.05598v1; Sec.3.4	low-resolution learned preconditioner. higher-resolution discrete target; flexible CG.	resolution and architecture comparisons. not a new physical equation family.

Appendix

Table 7: Versioned, targeted full-text protocol records. Controls described here belong to the cited papers.

PDE	$n$	Source	Direct	Zero	Coarse	Alt.	FNO-D	FNO-P	CNN-D	CNN-P
Diff.	31	0	1.79	5.51	5.87	10.33	7.32	7.22	6.09	6.01
Diff.	31	1	1.79	5.51	5.87	10.33	7.31	7.23	6.13	6.02
Diff.	63	0	8.63	10.23	11.32	17.73	12.04	11.90	11.01	11.05
Diff.	63	1	8.63	10.23	11.32	17.73	12.02	11.89	11.07	11.08
Diff.	127	0	40.87	28.57	33.46	46.19	30.36	29.89	30.32	30.42
Diff.	127	1	40.87	28.57	33.46	46.19	30.29	29.92	30.34	30.53

Appendix

Table 8: Full warm-path milliseconds per accepted query, using source-selected 6000-update states. D/P denote data/positive-residual objectives. Classical controls repeat across source-dataset rows for readability; they are not new runs. At grids 31/63/127, three optimizer seeds are averaged within each of 12 physical tasks. Grid 255 uses four new tasks and source 0, seed 11. Every listed endpoint succeeds. The primary corrector is Poisson-PCG or Poisson-GMRES; the alternate zero control uses AMG or ILU.

PDE	$n$	Batch	Direct	Zero	Coarse	FNO-D	FNO-P	CNN-D	CNN-P
Diff.	31	8	1.69	5.54	5.83	5.73	5.59	5.48	5.37
Diff.	31	32	1.66	5.32	5.65	5.27	5.26	5.17	5.10
Diff.	63	8	7.27	10.43	11.32	10.63	10.47	10.69	10.55
Diff.	63	32	7.26	10.04	11.03	10.12	9.91	10.27	10.21
Diff.	127	8	39.56	29.42	33.63	29.70	29.23	30.91	30.97
Diff.	127	32	40.59	28.81	33.09	29.26	28.63	30.49	30.17

Appendix

Table 9: Variable-matrix batched inference: milliseconds per accepted query. Two independent batch groups per PDE/grid/size, source 0 and seed 11, three repeated measurements. Batch 8 is the first-eight subset of batch 32; the sizes are paired, not independent cohorts. All 10080 query results pass. Two groups provide only a narrow descriptive throughput check.

PDE	Control	ms saved	95% interval	Control/learned	Groups
Diff.	zero	0.182	[0.055, 0.310]	1.0063	2
Diff.	coarse	4.459	[4.310, 4.609]	1.1557	2
Diff.	direct	11.958	[11.652, 12.263]	1.4176	2
Adv.	zero	1.242	[0.922, 1.563]	1.0118	2
Adv.	coarse	3.006	[2.861, 3.150]	1.0284	2
Adv.	direct	-65.078	[-65.237, -64.918]	0.3853	2

Appendix

Table 10: Batch-32, grid-127, source-0/seed-11 physics-FNO contrasts. Time saved is control minus learned time, in milliseconds per query. Speed ratios are paired geometric means. The bootstrap uses only two batch groups: these are descriptive intervals, not reliable population-level significance evidence. Every listed method solves every query.

PDE	$n$	$Q$	Status	Direct	Zero	Coarse	Alt.	FNO-D	FNO-P	CNN-D	CNN-P
Diff.	31	1	projection	1.89	5.68	6.15	10.52	7.51	7.45	6.33	6.28
Diff.	31	8	observed	0.33	5.30	5.22	3.88	7.14	7.07	5.96	5.90
Diff.	31	10	projection	0.29	5.29	5.19	3.69	7.13	7.06	5.95	5.89
Diff.	31	100	projection	0.13	5.25	5.10	3.01	7.08	7.02	5.91	5.85
Diff.	31	1000	projection	0.11	5.25	5.09	2.94	7.08	7.02	5.90	5.85
Diff.	63	1	projection	8.49	10.42	11.57	17.70	12.34	12.17	11.41	11.38

Appendix

Table 11: Fixed-matrix costs in milliseconds per query. Each of four matrix tasks has eight measured right-hand sides. Setup is counted once per matrix and repeats are summarized before averaging matrices. Q=8 is the observed workload; all other Q values are projections of setup/Q plus the measured mean query cost. These are inference-side projections and exclude training cost.

PDE	Method	ms/query	Continuation	Probe	Total updates	Accepted
Diff.	direct	8.80	0.00	0.00	0.00	32/32
Diff.	zero	10.22	60.38	0.00	60.38	32/32
Diff.	coarse	11.26	54.53	0.00	54.53	32/32
Diff.	fno_data	12.06	59.19	0.00	59.19	32/32
Diff.	min_residual	14.25	60.38	0.00	60.38	32/32
Diff.	probe5	15.30	49.59	15.00	64.59	32/32

Appendix

Table 12: Complete selector costs on 32 independent tasks per PDE, grid 63. Probe work includes discarded candidate updates; continuation alone is not total iterative work. Direct-solve update counts are bookkeeping zeros and are not comparable Krylov complexity. All candidate construction, inference, probes, continuation, and residual checks are timed.

PDE	Tolerance	Direct	Zero	Coarse	FNO-D	FNO-P	CNN-D	CNN-P
Diff.	1e-04	8/8	3/8	4/8	3/8	3/8	3/8	3/8
Diff.	1e-06	8/8	2/8	2/8	2/8	2/8	2/8	2/8
Diff.	1e-08	8/8	0/8	1/8	0/8	0/8	0/8	0/8
Adv.	1e-04	8/8	8/8	8/8	8/8	8/8	8/8	8/8
Adv.	1e-06	8/8	6/8	6/8	6/8	6/8	6/8	6/8
Adv.	1e-08	8/8	5/8	5/8	5/8	5/8	5/8	5/8

Appendix

Table 13: Full success denominators under the coefficient-log scale-three intervention, grid 63. All other tested conditions accept all endpoints. One numerical repeat is run per task/configuration; the same eight base fields are shared across conditions. Iterative failures are retained and no successful-subset speed is reported.

PDE	$n$	State	Resident	Reload	Increment	95% interval
Diff.	31	fno_data	6.66	12.31	5.65	[5.60, 5.69]
Diff.	31	fno_physics	6.59	12.25	5.66	[5.61, 5.72]
Diff.	31	cnn_data	5.46	10.33	4.87	[4.79, 4.98]
Diff.	31	cnn_physics	5.43	10.26	4.83	[4.78, 4.91]
Diff.	63	fno_data	10.83	16.44	5.62	[5.55, 5.70]
Diff.	63	fno_physics	10.66	16.33	5.66	[5.59, 5.76]

Appendix

Table 14: Within-process weight reloading versus already resident weights. Milliseconds per query and paired reload increment over four physical tasks per PDE/grid. This includes deserialization/device transfer in an existing Python/CUDA process; it is not a fresh-process cold-start result. Fresh-process results are reported separately in Table 15 .

PDE	$n$	State	Mean(s)	95% interval	Task-median range	Accepted
Diff.	31	direct	0.4133	[0.4107, 0.4159]	[0.4102, 0.4160]	12/12
Diff.	31	zero	0.4183	[0.4170, 0.4196]	[0.4165, 0.4202]	12/12
Diff.	31	coarse	0.4340	[0.4163, 0.4518]	[0.4161, 0.4591]	12/12
Diff.	31	fno_data	2.3341	[2.3177, 2.3576]	[2.3142, 2.3698]	12/12
Diff.	31	cnn_data	2.3094	[2.3014, 2.3174]	[2.3005, 2.3197]	12/12
Diff.	127	direct	0.4964	[0.4933, 0.4991]	[0.4919, 0.4999]	12/12

Appendix

Table 15: Fresh Python-process latency in seconds, from subprocess launch until the parent receives the true-residual acceptance notification. Each of four physical tasks per PDE/grid is repeated in three distinct processes; repetitions are reduced to task medians before aggregation. FNO/CNN are source-0/seed-11 data models followed by Poisson correction. The task range and descriptive task-bootstrap interval are shown; n=4 limits population inference. Costs include dependency imports, field generation, weight loading and new CUDA contexts for neural paths. Output-vector serialization, child shutdown, and the parent reference solve occur after the endpoint. OS and driver caches are not flushed. All 240 paths accept.

Path	Mean cost (95% CI)	Difference from zero (95% CI)
Direct	40.139 (39.593, 40.513)	12.670 (12.040, 13.331)
Zero + PCG	27.469 (26.953, 28.020)	—
Coarse + PCG	31.892 (31.419, 32.397)	4.423 (4.326, 4.506)
CNN (data) + PCG	29.041 (28.535, 29.575)	1.572 (1.541, 1.606)
CNN (physics) + PCG	28.984 (28.480, 29.516)	1.515 (1.474, 1.563)
FNO (data) + PCG	27.746 (27.253, 28.271)	0.278 (0.241, 0.316)

Appendix

Table 16: New batch confirmation only. Complete online milliseconds per query; parentheses show 95% group-bootstrap intervals. Differences are path minus zero, so negative is faster. All seven paths have 8/8 accepted groups.

PDE	Stratum	Loss	$N_{s}=128$	384	1024	128–1024 reduction [95% CI]
Diff.	id	D	0.0227	0.0144	0.0125	0.0102 [0.0082, 0.0125]
Diff.	id	P	0.0205	0.0129	0.0112	0.0092 [0.0073, 0.0115]
Diff.	joint	D	0.1849	0.1416	0.1262	0.0587 [0.0519, 0.0653]
Diff.	joint	P	0.1803	0.1348	0.1239	0.0564 [0.0454, 0.0683]
Adv.	id	D	0.0203	0.0156	0.0146	0.0057 [0.0046, 0.0072]
Adv.	id	P	0.0192	0.0139	0.0131	0.0061 [0.0049, 0.0076]

Appendix

Table 17: Complete prediction configurations, initial relative discrete

L^{2}

error. D is data-only; P adds the fixed residual weight

\lambda=0.001

. Both PDEs use one nested source realization, common validation 64, FNO base, and a common 6000-update schedule. Within each of 16 physical test tasks per PDE/stratum, optimizer seeds 11/23/37 are averaged before summaries and descriptive 4000-draw bootstrap intervals. Correction copies do not create extra prediction evidence.

PDE	Stratum	Loss	128–384	384–1024	128–1024
Diff.	id	D	0.0083 [0.0068, 0.0100]	0.0019 [0.0010, 0.0029]	0.0102 [0.0082, 0.0125]
Diff.	id	P	0.0075 [0.0062, 0.0090]	0.0017 [0.0007, 0.0028]	0.0092 [0.0073, 0.0115]
Diff.	joint	D	0.0433 [0.0345, 0.0518]	0.0154 [0.0091, 0.0221]	0.0587 [0.0519, 0.0653]
Diff.	joint	P	0.0456 [0.0340, 0.0591]	0.0109 [0.0054, 0.0160]	0.0564 [0.0454, 0.0683]
Adv.	id	D	0.0048 [0.0036, 0.0064]	0.0010 [0.0005, 0.0016]	0.0057 [0.0046, 0.0072]
Adv.	id	P	0.0053 [0.0041, 0.0068]	0.0008 [0.0003, 0.0014]	0.0061 [0.0049, 0.0076]

Appendix

Table 18: All 24 predeclared paired source-count prediction contrasts. Positive values mean lower prediction error with more source examples, holding validation information and optimizer-update schedule fixed. These intervals are conditional on the single source pool and fitted states; they are not multiplicity-adjusted or population-level data-scaling guarantees.

Stratum	Loss	Corrector	Zero	Coarse	128	384	1024
id	D	CG	152.88	144.56	149.06	147.10	146.58
id	D	Poisson-PCG	18.56	16.62	16.62	16.46	16.38
id	D	AMG-PCG	7.88	6.12	6.92	6.73	6.73
id	P	CG	152.88	144.56	148.46	146.40	145.44
id	P	Poisson-PCG	18.56	16.62	16.40	15.98	15.83
id	P	AMG-PCG	7.88	6.12	6.81	6.54	6.44

Appendix

Table 19: Diffusion complete mean correction counts at grid 31 and true relative residual

10^{-8}

. Each neural cell averages three seeds within 16 physical tasks; each classical control has 16 endpoints and is repeated in D/P rows only for readability. All 3840 endpoints across both PDEs succeed. Controls are independent of source sample count. Counts are not wall-clock utility.

Stratum	Loss	Corrector	128–384	384–1024	128–1024
id	D	CG	1.96 [0.96, 2.79]	0.52 [-0.04, 1.04]	2.48 [1.40, 3.44]
id	D	Poisson-PCG	0.17 [0.06, 0.29]	0.08 [-0.02, 0.19]	0.25 [0.10, 0.42]
id	D	AMG-PCG	0.19 [-0.02, 0.40]	0.00 [-0.08, 0.08]	0.19 [0.00, 0.38]
id	P	CG	2.06 [1.37, 2.79]	0.96 [0.25, 1.73]	3.02 [2.10, 4.06]
id	P	Poisson-PCG	0.42 [0.27, 0.54]	0.15 [0.06, 0.25]	0.56 [0.42, 0.69]
id	P	AMG-PCG	0.27 [0.12, 0.44]	0.10 [0.00, 0.23]	0.38 [0.23, 0.52]

Appendix

Table 20: Diffusion all paired changes in correction work with source count. Values are updates saved by the larger count, with descriptive task-bootstrap 95% intervals. Negative or near-zero changes are retained. The same 16 tasks are paired across counts and seeds are averaged within task; the interval does not resample the source dataset.

Stratum	Loss	Corrector	Zero	Coarse	128	384	1024
id	D	GMRES	218.81	199.62	164.48	166.96	159.48
id	D	Poisson-GMRES	40.62	40.31	39.33	39.21	38.92
id	D	ILU-GMRES	3.00	3.00	3.00	3.00	3.00
id	P	GMRES	218.81	199.62	165.71	164.10	158.08
id	P	Poisson-GMRES	40.62	40.31	38.60	38.71	38.52
id	P	ILU-GMRES	3.00	3.00	3.00	3.00	3.00

Appendix

Table 21: Advection complete mean correction counts at grid 31 and true relative residual

10^{-8}

Stratum	Loss	Corrector	128–384	384–1024	128–1024
id	D	GMRES	-2.48 [-7.35, 2.44]	7.48 [2.23, 12.90]	5.00 [-0.04, 9.92]
id	D	Poisson-GMRES	0.13 [-0.31, 0.60]	0.29 [-0.02, 0.67]	0.42 [0.13, 0.79]
id	D	ILU-GMRES	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]
id	P	GMRES	1.60 [-2.17, 5.23]	6.02 [0.65, 11.00]	7.63 [0.85, 14.44]
id	P	Poisson-GMRES	-0.10 [-0.56, 0.27]	0.19 [-0.10, 0.56]	0.08 [-0.29, 0.35]
id	P	ILU-GMRES	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]

Appendix

Table 22: Advection all paired changes in correction work with source count. Values are updates saved by the larger count, with descriptive task-bootstrap 95% intervals. Negative or near-zero changes are retained. The same 16 tasks are paired across counts and seeds are averaged within task; the interval does not resample the source dataset.

PDE	Loss	Corrector	Saved vs zero [95% CI]	Saved vs coarse [95% CI]
Diff.	D	CG	-5.81 [-8.08, -3.83]	-14.31 [-16.67, -12.10]
Diff.	D	Poisson-PCG	1.77 [1.42, 2.13]	-2.04 [-2.35, -1.77]
Diff.	D	AMG-PCG	0.40 [0.19, 0.62]	-0.54 [-0.77, -0.31]
Diff.	P	CG	-5.83 [-7.73, -4.08]	-14.33 [-16.96, -11.92]
Diff.	P	Poisson-PCG	2.12 [1.79, 2.48]	-1.69 [-2.00, -1.40]
Diff.	P	AMG-PCG	0.48 [0.27, 0.69]	-0.46 [-0.67, -0.23]

Appendix

Table 23: Joint-shift largest-source states versus both competent initialization controls. Positive values favor the learned state. Increased source information can improve the predictor while leaving a CG disadvantage or a disadvantage against coarse initialization. Fixed residual training is not a newly selected or retuned loss in this experiment.

Equation	State	Train/val	$\eta=3\times 10^{-4}$	$\eta=10^{-3}$
Diff.	Source 0	16/64	$+30.536\;[+25.942,+35.029]$	$+25.908\;[+21.878,+29.886]$
Diff.	Source 0	40/40	$+9.168\;[+5.938,+12.474]$	$+9.544\;[+6.319,+12.815]$
Diff.	Source 0	64/16	$+2.645\;[+0.932,+4.362]$	$+2.916\;[+1.121,+4.721]$
Diff.	Source 1	16/64	$+32.450\;[+27.984,+37.026]$	$+31.000\;[+27.586,+34.498]$
Diff.	Source 1	40/40	$+13.045\;[+7.635,+18.642]$	$+15.093\;[+10.768,+19.748]$
Diff.	Source 1	64/16	$+4.651\;[+2.237,+7.260]$	$+5.294\;[+2.897,+7.751]$

Appendix

Table 24: Same-trajectory checkpoint selection at budget 1600. Entries are fixed-final minus validation-best relative

L^{2}

errors, in units of

10^{-3}

; positive values favor validation selection. Intervals are descriptive paired-task 95% bootstraps with 4000 resamples after averaging three optimization seeds within each of 128 test tasks per equation. No multiplicity adjustment is applied. Both rates are fixed in their respective columns; all other budgets are retained in the accompanying CSVs.

Equation	Initial state	Train/val	Fixed-final	Validation-best	Selected steps
Diff.	Source 0	16/64	0.1955	0.1677	50,25,25
Diff.	Source 1	16/64	0.1990	0.1665	50,25,25
Diff.	Scratch	16/64	0.3032	0.2525	350,375,300
Diff.	Source 0	40/40	0.1657	0.1581	100,250,75
Diff.	Source 1	40/40	0.1693	0.1567	50,50,25
Diff.	Scratch	40/40	0.2308	0.2164	525,775,400

Appendix

Table 25: Fixed 80-label allocation, budget 1600. Each checkpoint policy selects its own learning rate on validation data. Errors average three seeds within each of 128 independent tasks per equation; selected validation-best steps list the three seeds. These columns compare complete selection policies, not only checkpoint timing.

Stratum	Objective	Error 1200	Error 6000	Steps 1200	Steps 6000	Steps saved [95% CI]
ID	data	0.0465	0.0146	2.771	2.167	+0.604 [+0.417,+0.771]
ID	physics	0.0453	0.0137	2.771	2.188	+0.583 [+0.396,+0.771]
Low diffusion	data	0.5513	0.3130	4.958	4.458	+0.500 [+0.271,+0.708]
Low diffusion	physics	0.5494	0.3204	4.958	4.542	+0.417 [+0.208,+0.625]
Strong reaction	data	0.2398	0.1907	3.792	3.729	+0.062 [-0.125,+0.250]
Strong reaction	physics	0.2360	0.1759	3.833	3.708	+0.125 [-0.063,+0.312]

Appendix

Table 26: Exploratory common-task source-training sensitivity at

n=63

and residual tolerance

10^{-8}

. Six states are retrained with a stretched 6000-step cosine schedule; 112 old test tasks provide paired contrasts, with 16 tasks per stratum. Positive update differences favor 6000 steps. This is not a same-training-trajectory causal budget contrast.

Stratum	Objective	Initial error	Learned steps	Zero steps	Coarse steps	Saved vs zero [95% CI]
ID	data	0.0103	2.083	4.625	2.000	+2.542 [+2.208,+2.875]
ID	physics	0.0098	2.083	4.625	2.000	+2.542 [+2.208,+2.875]
Low diffusion	data	0.2623	4.333	5.625	2.000	+1.292 [+0.958,+1.625]
Low diffusion	physics	0.2730	4.292	5.625	2.000	+1.333 [+1.042,+1.625]
Strong reaction	data	0.8148	3.833	4.625	2.000	+0.792 [+0.292,+1.375]
Strong reaction	physics	0.7060	3.708	4.625	2.000	+0.917 [+0.417,+1.500]

Appendix

Table 27: Independent confirmation: 56 new physical tasks, eight per stratum, reuse the same six 6000-step models without additional training.

n=63

, true residual tolerance

10^{-8}

. Task bootstrap intervals are conditional on these fitted states and small strata; they are not combined with the original 112 tasks.

Figure 2: Training-recipe sensitivity retained from the source manuscript. Left: the same 112 tasks under two training schedules. Right: 56-task independent confirmation of the longer-budget states, compared with zero and coarse controls. At $n=63$ and residual tolerance $10^{-8}$ , the extreme-stratum contrast against zero is not sign-confirmed in the new cohort. Intervals are descriptive paired-task 95% intervals.

Shift	Grid	Corrector	Neural accepted	Neural updates	$\Delta$ vs zero [95% CI]; $N$	$\Delta$ vs coarse [95% CI]; $N$
ID	64	CG	16/16	296.38	12.31 [10.75, 13.79]; 16	-2.88 [-4.85, -0.98]; 16
ID	64	Poisson-PCG	16/16	14.02	1.92 [1.81, 2.00]; 16	0.10 [-0.02, 0.27]; 16
ID	64	AMG-PCG	16/16	7.10	1.02 [0.94, 1.12]; 16	-0.04 [-0.10, 0.00]; 16
ID	128	CG	16/16	613.23	20.15 [17.12, 23.23]; 16	-38.60 [-43.23, -34.17]; 16
ID	128	Poisson-PCG	16/16	14.94	1.50 [1.25, 1.75]; 16	-0.38 [-0.62, -0.12]; 16
ID	128	AMG-PCG	16/16	8.62	1.12 [0.94, 1.31]; 16	-0.62 [-0.81, -0.44]; 16

Appendix

Table 28: Scaled official NOWS audit at

10^{-8}

. Neural counts require every optimization seed to pass. Positive

\Delta

means that the numerical control takes more updates. Each paired cell gives the mean, task-bootstrap interval, and number of jointly accepted tasks; any incomplete acceptance remains a conditional comparison. All strata and both grids are shown.

Shift	Grid	Mean initial error	Max final error	Max true residual	Min accepted
ID	64	2.14e-02	6.47e-10	9.99e-09	16/16
ID	128	2.26e-02	4.18e-10	1.00e-08	16/16
Contrast	64	5.71e-01	7.44e-10	9.99e-09	16/16
Contrast	128	5.76e-01	4.39e-10	9.99e-09	16/16
Roughness	64	2.56e-02	4.88e-10	9.99e-09	16/16
Roughness	128	3.47e-02	4.82e-10	9.99e-09	16/16

Appendix

Table 29: Initial prediction error and final discrete verification for the scaled NOWS audit at

10^{-8}

. Initial error is shown from the fixed CG configuration. Final error/residual are maxima across the three correctors, tasks and optimization seeds. Residual acceptance is not an error certificate for the continuous PDE.

Host	Shift	Method	Accepted/ $N$	Total med.	Total P95	Assembly	Init.	Setup	Solve/verify
Host-B	ID	Direct	10/10	1332.238	1367.106	1292.957	0.087	0.005	39.159
Host-B	ID	Zero+CG	10/10	1359.059	1372.133	1291.839	0.086	0.002	66.365
Host-B	ID	VINO+CG	10/10	1366.133	1381.798	1300.307	4.266	0.013	64.482
Host-B	ID	Zero+AMG-PCG	10/10	1336.855	1363.314	1293.188	0.089	24.267	19.904
Host-B	ID	VINO+AMG-PCG	10/10	1345.768	1360.770	1300.535	4.268	23.989	17.076
Host-B	Contrast	Direct	16/16	1329.791	1340.329	1290.252	0.086	0.002	39.278

Appendix

Table 30: Within-host online cost in milliseconds for the fixed grid128,

10^{-8}

display.

N

is the verified task subset assigned to that host, not the global 16-task stratum denominator. Successful-subset numbers are conditional when acceptance is incomplete. The fixed display methods are direct, zero/VINO with CG, and zero/VINO with AMG-PCG. Component medians need not sum to total median.

Corrector	Smaller summary	Reversals/eligible pairs	Mean task fraction [95% CI]
CG	Solution error	423/935	45.47% [41.51, 49.38]
CG	Initial residual	260/935	27.81% [24.01, 32.03]
CG	Energy error	243/935	26.04% [22.08, 30.31]
CG	All three (strict)	72/362	23.50% [17.61, 30.39]
PCG	Solution error	145/935	15.42% [11.15, 20.10]
PCG	Initial residual	7/935	0.73% [0.21, 1.35]

Appendix

Table 31: Real learned-state reversals on the 64 new ID tasks at

n=63

, restricted to pairs for which both relative prediction errors are at most 5%. The estimate is the mean of per-task reversal fractions with a task-bootstrap 95% interval, not the pooled-pair fraction. All rows have 64 eligible tasks. Work ties are non-reversals; zero observed reversals do not bound population risk.

Tasks	Grid	Source	Corrector	Prediction-selected	Work-selected
ID	31	0	CG	149.297	149.297
ID	31	0	Poisson-PCG	17.109	16.984
ID	31	1	CG	148.953	149.469
ID	31	1	Poisson-PCG	17.047	17.031
ID	63	0	CG	320.656	318.156
ID	63	0	Poisson-PCG	18.359	18.031

Appendix

Table 32: Absolute held-out updates for the prediction-selected and work-selected states. Each row is 64 accepted tasks; all paths use the same tolerance. Corresponding differences and intervals appear in Table 3 . Full controls and descriptive CPU timings accompany the raw-data analysis.

Evidence family	Reported record coverage	Independence and interpretation
Original supplied aggregates	Historical table retained in workbook	Original run-count semantics; not reconstructed into missing per-task weights or pooled with controlled waves.
Two core linear waves	240 source trajectories; 144 adaptation trajectories; 494,592 correction endpoints	Main tasks: 256 plus 128. Budget checkpoints, seeds, grids, and tolerances are dependent.
Core timing and spectral records	39,624 timing repetitions; 41,527 spectral records	Subsets and oracle constructions have their own units. The 41,463 expanded spectral records exclude nine follow-ups after one failed matching gate.
Completed deployment suites	360 units; 16,008 JSON records, including 240 fresh-process records	Query expansion yields 25,584 final-query records, with 142 failures; none of these totals is an independent-task count.
Nested source count	24 new trajectories plus 12 reused states; 3,840 accepted endpoints	One nested source pool; 64 new physical test tasks. Fixed update budget changes exposure per example.
Fixed target-information allocation	108 actual trajectories; 648 fixed-rate checkpoint/policy states; 324 rate-selected states	128 test tasks per equation, disjoint from selection. Three allocations reuse one 80-label pool per equation.

Appendix

Table 35

Explore similar work

Jun 25, 2026cs.LG

Error-Conditioned Neural Solvers

Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss--Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error-conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching

10\times

on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero-shot parameter changes and cross-equation transfer, where its relative advantage is largest in the ill-conditioned regimes where residual minimization is least reliable. Project website: https://neuralsolver.github.io/.

Haina Jiang, Liam Wang, Peng-Chen Chen +4

University of Michigan · KAIST AI · Los Alamos National Laboratory

Neural PDE Solvers PDE Surrogate Modeling

Ask PDF

May 28, 2026cs.LG

Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation

Cross-Reynolds generalisation in neural PDE solvers remains poorly characterised. On the canonical forced 2D Navier-Stokes benchmark, a trained Fourier Neural Operator reaches 46.68% relative L2 error under a 10x Reynolds-number shift, yet zero-forward-model retrieval baselines already improve to 41-42%. This suggests representation geometry as a major organising variable among the tested methods. We test this hypothesis through ConvAE-Relay, which matches states in a source-trained convolutional autoencoder latent space and borrows dynamics from a source-regime database, achieving 38.34+/-0.07% using only a source-regime database and no target-regime fitting, labels, or database entries. A 2x2 ablation isolates matching quality as dominant over the update rule. Oracle experiments confirm that source-regime dynamics directions remain transferable (cosine similarity ~0.84) when matching stays on-manifold; autoregressive drift is the primary bottleneck (~12 percentage points). From the learned-prediction side, a U-Net with multi-scale skip connections achieves 34.72+/-0.60%, consistent with the retrieval-side finding that local, multi-scale representations organise cross-Reynolds transfer among tested methods. All claims are scoped to this benchmark.

Jianing Shi

London School of Economics and Political Science.

Neural PDE Solvers Neural Representation Geometry Computational Fluid Dynamics

Ask PDF

Sep 29, 2026cs.AI

Transolver- $σ$ : Joint Spectral-Physical Subspace Modeling for Neural PDE Solving

Neural solvers offer efficient surrogates for numerical simulation of partial differential equations (PDEs). For time-dependent problems, strong one-step accuracy does not necessarily translate into reliable autoregressive rollout. We observe that a solver based only on physical-state modeling can achieve lower one-step error, whereas its spectral-only counterpart can become more accurate at later rollout steps. Motivated by this observation, we present Transolver-

σ

, a neural PDE solver based on joint spectral--physical subspace modeling. Within each block, adaptive physical-state interactions and spectral transformations are modeled in dedicated latent subspaces, whose responses are recomposed to enable information exchange between the two representations. Within the physical subspace, we introduce Slice-Residual Physics-Attention (SRPA), which preserves an explicit slice-space identity path while retaining learnable cross-slice interaction. In parallel, an axis-factorized Fourier operator captures global spectral structure. Across five well-established PDE benchmarks spanning steady-state prediction and time-dependent dynamics, Transolver-

σ

achieves state-of-the-art with a benchmark-averaged relative error reduction of 33.4% over the strongest baseline for each metric, while consistently improving autoregressive rollout over single-operator counterparts. Transolver-

σ

further delivers strong gains on coupled multiphysics systems and real-world fluid and combustion measurements from RealPDEBench, demonstrating its effectiveness beyond standard simulation benchmarks.

Haonan Shangguan, Hang Zhou, Haixu Wu +3

School of Software, BNRist, Tsinghua University · MIT CSAIL

Fourier Neural Operator Neural PDE Solvers PDE Surrogate Modeling PDE Operator Learning

Ask PDF

Abstract

Explore similar work

Error-Conditioned Neural Solvers

Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation

Transolver-σσσ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solving

Transolver- $σ$ : Joint Spectral-Physical Subspace Modeling for Neural PDE Solving