PDE-PFN: Prior-Data Fitted Neural PDE Solver

Jaehyeon Park; Mingu Kang; Dongseok Lee; Woojin Cho; Kookjin Lee; Anthony Gruber; Youngjoon Hong; Noseong Park

cs.LGOct 9, 2026

PDE-PFN: Prior-Data Fitted Neural PDE Solver

Authors: Jaehyeon Park, Mingu Kang, Dongseok Lee, Woojin Cho, Kookjin Lee, Anthony Gruber, Youngjoon Hong, Noseong Park

Organizations: SolverX · KAIST · Seoul National University · TelePIX ·

Abstract

Motivated by the success of large language models (LLMs) with broad generalizability and robustness to noisy or unreliable pre-training data, we explore in-context prediction of PDE solution fields using approximate pre-training data. In addition, inspired by the posterior predictive mean-based inference mechanism of the in-context learning in prior-data fitted networks (PFNs), we propose PDE-PFN, a prior-data fitted neural solver that approximates the posterior predictive mean of PDE solutions via in-context learning. PDE-PFN builds on a PFN architecture with self- and cross-attention mechanisms of Transformer and is pre-trained on noisy approximate solutions generated by physics-informed neural networks, serving as diverse but not necessarily exact priors. Through experiments on a range of two-dimensional PDEs, we demonstrate that PDE-PFN achieves empirical generalization across heterogeneous equations, robustness under noisy priors, and zero-gradient-update in-context inference capability. PDE-PFN achieves competitive or improved predictive accuracy relative to the evaluated baselines while supporting flexible context and query representations.

Figures & tables

Figure 1: The overall workflow of PDE-PFN. Our model performs in-context learning (ICL) based on given observations (i.e., context) to infer solutions. At inference, PDE-PFN conditions on observations from the target instance without separate operator-level demonstrations or target-instance gradient updates. Even when trained with an approximated PINN prior, our method can improve over noisy approximate pretraining data.

Figure 2: A schematic diagram of our model. (Left) Given the contexts $D$ and queries $T$ drawn from the prior distribution $\mathcal{D}$ , our model $\tilde{u}_{\theta}$ is trained to infer the solutions of the queried points $T$ in the training phase. ICL is leveraged with self-attention (blue) within $D$ and cross-attention (red) from $T$ to $D$ . (Right) In the testing phase, the pre-trained $\tilde{u}_{\theta}$ takes contexts $\overline{D}$ and queries $\overline{T}$ drawn from the ground truth distribution $\mathcal{U}$ , and predicts for the queried points $\overline{T}$ .

Section	Task	Coeff/Dataset	Equation	Objective
4.2.1	Inter & Extra	Seen	Family of 2D CDR equations	Within-family task generalization
4.2.2	Inter & Extra	Unseen	Family of 2D CDR equations	Within-family coefficient generalization
4.3.1	Inter & Extra	Seen & Unseen	SWE	Cross-PDE transfer after fine-tuning
4.3.2	Operator Learning	Unseen	CNSE	Cross-PDE transfer after fine-tuning

Table 1: Roadmap of the experiment section. ‘Inter’ and ‘Extra’ denote spatiotemporal interpolation and temporal extrapolation tasks, respectively. The ‘Coeff/dataset’ column specifies either the coefficients of the equation or the dataset used in evaluation.

Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Spatiotemporal interpolation	$L_{1}$ Abs	0.01218	0.01507	0.01922	-	-	-	-	-	-
	$L_{2}$ Rel	0.01643	0.02057	0.02479
	$L_{\infty}$ Rel	0.04600	0.06584	0.05980
Temporal extrapolation	$L_{1}$ Abs	0.01474	0.01940	0.06529	0.03302	0.02731	0.05800	0.02876	0.14252	0.04924
	$L_{2}$ Rel	0.02261	0.02705	0.08212	0.04058	0.03430	0.07915	0.03665	0.15580	0.08587
	$L_{\infty}$ Rel	0.08444	0.08802	0.22802	0.09553	0.09877	0.28905	0.10948	0.28285	0.08587

Table 2: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Coefficients	Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Interpolation	Spatiotemporal interpolation	$L_{1}$ Abs	0.01241	0.01277	0.01347	-	-	-	-	-	-
		$L_{2}$ Rel	0.01658	0.01761	0.01731
		$L_{\infty}$ Rel	0.04274	0.05503	0.04114
	Temporal extrapolation	$L_{1}$ Abs	0.08957	0.08709	0.11918	0.11625	0.11194	0.12220	0.11085	0.17740	0.13879
		$L_{2}$ Rel	0.11990	0.12465	0.14903	0.14658	0.14348	0.18191	0.14913	0.21340	0.15085
		$L_{\infty}$ Rel	0.29526	0.36472	0.35196	0.28689	0.30813	0.64429	0.33280	0.39890	0.15085

Table 3: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations. They are measured at the unseen coefficients. The best performance is marked in bold and the second-best performance is marked with underline .

Evaluation dataset	Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Seen	Spatiotemporal interpolation	$L_{1}$ Abs	0.00208	0.00269	0.00271	-	-	-	-	-	-
		$L_{2}$ Rel	0.00739	0.00822	0.00997
		$L_{\infty}$ Rel	0.05897	0.06950	0.06951
	Temporal extrapolation	$L_{1}$ Abs	0.01548	0.01418	0.06994	0.03271	0.02292	0.05036	0.02902	0.02964	0.03327
		$L_{2}$ Rel	0.03273	0.03221	0.08319	0.05429	0.04110	0.07436	0.05269	0.05208	0.05860
		$L_{\infty}$ Rel	0.20590	0.24596	0.28736	0.24954	0.24478	0.38030	0.24832	0.25668	0.29727

Table 4: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to SWE. They are measured at the seen/unseen datasets. The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Operator learning	$L_{1}$ Abs	0.09497	0.09429	0.17797	0.09581	0.08734	0.13637	0.10366	0.13313	0.09164
	$L_{2}$ Rel	0.54833	0.54369	1.09567	0.57905	0.55420	0.65798	0.62536	0.66904	0.67869
	$L_{\infty}$ Rel	0.63533	0.63374	1.09165	0.70708	0.72876	0.85729	0.66936	0.92909	1.15028

Table 5: The evaluation results for the operator learning task applied to CNSE. The best performance is marked in bold and the second-best performance is marked with underline .

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

PDE (Section)	Task	Train	Valid		Test
PDE (Section)	Task	Train	Context	Query	Context	Query
Family of 2D CDR equations (Section 4.2 )	Spatiotemporal interpolation	$t\in[0.0,0.5]$	predefined in $t\in[0.0,0.5]$	grid at $t=0.25$	predefined in $t\in[0.0,0.5]$	grids at $t\in\{0.05,0.15,0.35,0.45\}$
	Temporal extrapolation	$t\in[0.0,0.5]$	grid at $t=0.5$	grid at $t=0.6$	grid at $t=0.6$	grids at $t\in[0.6,1.0]$
	Unseen coeff temporal extrapolation	-	-	-	grid at $t=0.1$	grids at $t\in[0.2,0.5]$
2D SWE (Section 4.3.1 )	Spatiotemporal interpolation	$t\in[0.0,0.5]$	predefined in $t\in[0.0,0.5]$	grid at $t=0.25$	predefined in $t\in[0.0,0.5]$	grids at $t\in\{0.05,0.15,0.35,0.45\}$
2D SWE (Section 4.3.1 )	Temporal extrapolation	$t\in[0.0,0.5]$	grid at $t=0.5$	grid at $t=0.6$	grid at $t=0.6$	grids at $t\in[0.6,1.0]$

Appendix

Table 6: Time range of data points used for training, validation, and testing in all experiments.

Figure 3: A diagram of PDE-PFN’s architecture. The diagram is presented as an extension of Figure 2 . The model consists of three main components: an encoder, a Transformer block, and a decoder. FFE denotes the Fourier feature embedding, and Act denotes the activation function.

Task	Metric	Ours	Ours (PINN)	Ours (Gaussian 0.02)	Ours (Uniform $[-0.01,0.01]$ )
Spatiotemporal interpolation	$L_{1}$ Abs	0.01218	0.01507	0.01244	0.01060
	$L_{2}$ Rel	0.01643	0.02057	0.01670	0.01450
	$L_{\infty}$ Rel	0.04600	0.06584	0.04659	0.04269
Temporal extrapolation	$L_{1}$ Abs	0.01474	0.01940	0.01562	0.01513
	$L_{2}$ Rel	0.02261	0.02705	0.02383	0.02320
	$L_{\infty}$ Rel	0.08444	0.08802	0.08750	0.08213

Appendix

Table 8: The evaluation results of our model trained using systematically noisy data for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations. The text in parentheses following “Ours” indicates the noise type and its standard deviation or range. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Spatiotemporal interpolation	$L_{1}$ Abs	0.01953	0.02445	0.02540	-	-	-	-	-
	$L_{2}$ Rel	0.02755	0.03467	0.03413	-	-	-	-	-	-
	$L_{\infty}$ Rel	0.10158	0.10716	0.09286	-	-	-	-	-	-
Temporal extrapolation	$L_{1}$ Abs	0.01655	0.02099	0.06888	0.03458	0.02816	0.05794	0.03048	0.14018	0.05051
	$L_{2}$ Rel	0.02486	0.02893	0.08591	0.04268	0.03582	0.07903	0.03907	0.15380	0.08624
	$L_{\infty}$ Rel	0.08830	0.09333	0.23386	0.10203	0.10677	0.28805	0.11730	0.28481	0.08624

Appendix

Table 9: The evaluation results for the spatiotemporal interpolation and temporal extrapolation task applied to the family of 2D CDR equations given a noisy PINN prior at evaluation. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Bias	No noise	0.1%	0.5%	1.0%	1.5%	2.0%	2.5%	3.0%	3.5%	4.0%
$L_{1}$ Abs	0.01474	0.01483	0.01472	0.01481	0.01514	0.01572	0.01664	0.01800	0.01985	0.02212
$L_{2}$ Rel	0.02261	0.02271	0.02246	0.02244	0.02271	0.02330	0.02255	0.02573	0.02772	0.03009
$L_{\infty}$ Rel	0.08444	0.08556	0.08459	0.08426	0.08490	0.08646	0.08901	0.09265	0.09733	0.10276

Appendix

Table 10: The evaluation results for the temporal extrapolation tasks applied to the family of 2D CDR equations. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

Coefficients	Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Interpolation	Temporal extrapolation	$L_{1}$ Abs	0.00736	0.00939	0.05139	0.10863	0.01405	0.04695	0.01839	0.13852	0.04610
		$L_{2}$ Rel	0.00954	0.01178	0.06179	0.14380	0.01636	0.06227	0.02049	0.14993	0.04676
		$L_{\infty}$ Rel	0.03193	0.03693	0.16008	0.34858	0.03847	0.21750	0.04512	0.26293	0.04676
Extrapolation	Temporal extrapolation	$L_{1}$ Abs	0.01027	0.02063	0.05514	0.03887	0.02312	0.05013	0.02560	0.14661	0.05186
		$L_{2}$ Rel	0.01068	0.02169	0.06450	0.05114	0.02470	0.06351	0.02677	0.15650	0.05189
		$L_{\infty}$ Rel	0.01661	0.03474	0.16555	0.13713	0.04591	0.20955	0.04911	0.27502	0.05189

Appendix

Table 11: The evaluation results for the temporal extrapolation tasks applied to the family of 2D CDR equations. They are measured at the unseen coefficients and unseen temporal domain. The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Spatiotemporal interpolation	$L_{1}$ Abs	0.01339	0.07394	-	-	-	-	-	-
	$L_{2}$ Rel	0.01885	0.09376
	$L_{\infty}$ Rel	0.06606	0.26320
Temporal extrapolation	$L_{1}$ Abs	0.02017	0.03988	0.03184	0.03371	0.08342	0.04696	0.09397	0.12628
	$L_{2}$ Rel	0.02929	0.05282	0.04160	0.04192	0.09967	0.06282	0.10352	0.13633
	$L_{\infty}$ Rel	0.10793	0.17487	0.12671	0.10980	0.27716	0.18313	0.23855	0.13633

Appendix

Table 12: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the new family of 2D CDR equations. They are measured in the same way of Section 4.2.1 . The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
Spatiotemporal interpolation	$L_{1}$ Abs	0.1599	0.1513	0.1401	-	-	-	-	-	-
	$L_{2}$ Rel	0.2005	0.1884	0.2177
	$L_{\infty}$ Rel	0.4253	0.3874	0.6944
Temporal extrapolation	$L_{1}$ Abs	0.0432	0.0553	0.0788	0.0492	0.0449	0.0710	0.0480	0.1581	0.0720
	$L_{2}$ Rel	0.0578	0.0719	0.0985	0.0641	0.0578	0.0940	0.0603	0.1766	0.2117
	$L_{\infty}$ Rel	0.1588	0.1667	0.2706	0.1750	0.1591	0.3008	0.1631	0.3449	0.2117

Appendix

Table 13: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations with random initial condition sampled from uniform distribution. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Metric	Ours	Ours (PINN)	DeepONet	Oformer
$L_{1}$ Abs	0.11047	0.10755	0.23160	0.12039
$L_{2}$ Rel	0.18831	0.18357	0.44101	0.19744
$L_{\infty}$ Rel	0.33584	0.32887	0.73076	0.32318

Appendix

Table 14: The evaluation results for the operator learning task applied to the Airfoil dataset. The best performance is marked in bold and the second-best performance is marked with underline .

Metric	Ours	Ours (PINN)	DeepONet	FNO	F-FNO	A-FNO	LNO	Poseidon	DPOT
$L_{1}$ Abs	0.00796	0.00742	0.00996	0.00792	0.00806	0.00906	0.01413	0.00998	0.00726
$L_{2}$ Rel	0.28771	0.28091	0.40759	0.29292	0.34149	0.40357	0.51778	0.41001	0.51505
$L_{\infty}$ Rel	0.39694	0.40124	0.59057	0.46398	0.59381	0.80674	0.61245	0.52933	0.51505

Appendix

Table 15: The evaluation results for the operator learning task applied to Darcy Flow. The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	Ours (PINN)	DeepONet	FNO	DPOT
Spatiotemporal interpolation	$L_{1}$ Abs	0.02685	0.02843	0.02816	-	-
	$L_{2}$ Rel	0.03711	0.04144	0.03981
	$L_{\infty}$ Rel	0.10938	0.12823	0.12724
Temporal extrapolation	$L_{1}$ Abs	0.01031	0.01039	0.02421	0.01521	0.01485
	$L_{2}$ Rel	0.01376	0.01391	0.03033	0.01824	0.01638
	$L_{\infty}$ Rel	0.04180	0.04545	0.11530	0.05545	0.01638

Appendix

Table 16: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the 3D diffusion reaction equations. They are measured at the seen coefficients (

\nu_{x},\nu_{y},\nu_{z},\rho_{1}\in\{0.0,0.5,1.0,1.5\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Metric	Ours	Ours (PINN)	DeepONet	FNO	DPOT
$L_{1}$ Abs	0.00152	0.00177	0.01622	0.00558	0.01458
$L_{2}$ Rel	0.02626	0.03004	0.24546	0.08019	0.22860
$L_{\infty}$ Rel	0.07482	0.10394	0.57927	0.19076	0.26938

Appendix

Table 17: The evaluation results for the operator learning task in 3D Gray-Scott equation. The best performance is marked in bold and the second-best performance is marked with underline .

Figure 4: Sensitivity to the number of test context points

\overline{D}

in the family of 2D CDR equations. Prediction errors decrease as more context observations are provided, which is consistent with the posterior-predictive-mean interpretation under squared loss.

Task	Metric	Ours	Ours w.o. agf	Ours w.o. act	Ours w.o. all
Spatiotemporal interpolation	$L_{1}$ Abs	0.01218	0.01242	0.01040	0.01780
	$L_{2}$ Rel	0.01643	0.01625	0.01495	0.02281
	$L_{\infty}$ Rel	0.04600	0.04297	0.04399	0.05736
Temporal extrapolation	$L_{1}$ Abs	0.01474	0.01776	0.01725	0.02454
	$L_{2}$ Rel	0.02261	0.02779	0.02665	0.03437
	$L_{\infty}$ Rel	0.08444	0.09554	0.09376	0.10625

Appendix

Table 18: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations. “Ours w.o. agf,” “Ours w.o. act,” and “Ours w.o. all” denote replacing AGF with a vanilla Transformer block, replacing the rational activation with GELU, and replacing both, respectively. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Task	Metric	Ours	Ours w.o. agf	Ours with sinusoidal
Spatiotemporal interpolation	$L_{1}$ Abs	0.01507	0.01667	0.01482
	$L_{2}$ Rel	0.02057	0.02171	0.02096
	$L_{\infty}$ Rel	0.06584	0.05536	0.06946
Temporal extrapolation	$L_{1}$ Abs	0.01940	0.02105	0.02671
	$L_{2}$ Rel	0.02705	0.03030	0.04159
	$L_{\infty}$ Rel	0.08802	0.09518	0.14922

Appendix

Table 19: The evaluation results for the spatiotemporal interpolation and temporal extrapolation tasks applied to the family of 2D CDR equations. “Ours w.o. agf,” “Ours w.o. act,” and “Ours w.o. all” denote replacing AGF with a vanilla Transformer block, replacing the rational activation with GELU, and replacing both, respectively. They are measured at the seen coefficients (

\beta_{x},\beta_{y},\nu_{x},\nu_{y},\rho_{1},\rho_{2},\rho_{3}\in\{0.0,0.5,1.0\}

). The best performance is marked in bold and the second-best performance is marked with underline .

Context length	1,024	2,048	4,096	8,192	12,288	16,384
Training time (sec)	0.104	0.102	0.101	0.107	0.136	0.165
Training memory (MB)	280.0	329.8	420.7	617.2	800.4	987.4
Inference time (sec)	0.015	0.015	0.022	0.037	0.052	0.069
Inference memory (MB)	182.1	228.7	323.6	510.7	702.1	889.7

Appendix

Table 20: Training and inference performance with different context lengths .

Query length	1,024	2,048	4,096	8,192	12,288	16,384
Training time (sec)	0.101	0.100	0.101	0.123	0.148	0.176
Training memory (MB)	872.7	1010.1	1265.4	1788.6	2303.0	2831.2
Inference time (sec)	0.02	0.02	0.021	0.024	0.027	0.03
Inference memory (MB)	242.3	270.1	323.1	433.6	540.7	648.1

Appendix

Table 21: Training and inference performance with different query lengths .

$m$	Separation recovery	$L_{2}$ Rel: $q_{1}$	$L_{2}$ Rel: $q_{2}$
0	0.000	0.3059	0.2649
4	0.073 $\pm$ 0.195	0.3041 $\pm$ 0.0021	0.2638 $\pm$ 0.0012
16	0.270 $\pm$ 0.310	0.2970 $\pm$ 0.0057	0.2600 $\pm$ 0.0017
32	0.570 $\pm$ 0.337	0.2874 $\pm$ 0.0060	0.2550 $\pm$ 0.0013
64	0.887 $\pm$ 0.342	0.2692 $\pm$ 0.0109	0.2465 $\pm$ 0.0025

Appendix

Table 22: Context identifiability with fixed context cardinality. Replacing initial observations with later observations introduces information about the differing dynamics. Standard deviations reflect observation-selection variability, not training-seed variability.

Context time $t_{c}$	$L_{1}$ Abs	$L_{2}$ Rel	$L_{\infty}$ Rel
0.45	0.01458	0.02199	0.08245
0.50	0.01437	0.02165	0.08024
0.60	0.01474	0.02261	0.08444
0.65	0.01604	0.02472	0.09530

Appendix

Table 23: Sensitivity to the time of the observed context snapshot, with fixed context cardinality and future query times. The same trained checkpoint is used throughout.

Model	Periodic BC RMSE	Relative periodic BC error
DeepONet	0.07097	0.06898
PDE-PFN	0.02081	0.02119

Appendix

Table 24: Periodic endpoint mismatch for seen-coefficient CDR temporal extrapolation. Lower values indicate greater consistency of independently predicted periodic endpoint pairs.

Task	Model	$L_{1}$ Abs	$L_{2}$ Rel	$L_{\infty}$ Rel
Seen coefficients: spatiotemporal interpolation	PDE-PFN	0.01218	0.01643	0.04600
Seen coefficients: spatiotemporal interpolation	GNOT	0.01477	0.01928	0.04619
Seen coefficients: temporal extrapolation	PDE-PFN	0.01474	0.02261	0.08444
Seen coefficients: temporal extrapolation	GNOT	0.02377	0.03126	0.08807
Coefficient interpolation: spatiotemporal interpolation	PDE-PFN	0.01241	0.01658	0.04274
Coefficient interpolation: spatiotemporal interpolation	GNOT	0.01269	0.01657	0.03930

Appendix

Table 25: Comparison with GNOT on the 2D CDR family. PDE-PFN uses numerical priors. temporal extrapolation with unseen coefficients remains within the training-time domain. Lower errors are better; the lower value for each metric in each setting is bold.

Task	Model	$L_{1}$ Abs	$L_{2}$ Rel	$L_{\infty}$ Rel
Seen initial conditions: spatiotemporal interpolation	PDE-PFN	0.00208	0.00739	0.05897
Seen initial conditions: spatiotemporal interpolation	GNOT	0.00317	0.00816	0.06265
Seen initial conditions: temporal extrapolation	PDE-PFN	0.01548	0.03273	0.20590
Seen initial conditions: temporal extrapolation	GNOT	0.03076	0.07564	0.42872
Unseen initial conditions: spatiotemporal interpolation	PDE-PFN	0.00213	0.00754	0.05986
Unseen initial conditions: spatiotemporal interpolation	GNOT	0.00317	0.00818	0.06233

Appendix

Table 26: Comparison with GNOT on SWE after downstream fine-tuning. PDE-PFN is initialized from pre-training on numerical CDR priors. Seen/unseen refers to initial-condition samples used/not used in downstream training. Lower errors are better; the lower value for each metric in each setting is bold.

Model	$L_{1}$ Abs	$L_{2}$ Rel	$L_{\infty}$ Rel
PDE-PFN	0.09497	0.54833	0.63533
GNOT	0.09601	0.57140	0.65714

Appendix

Table 27: Comparison with GNOT on CNSE trajectory prediction. PDE-PFN is fine-tuned from pre-training on numerical CDR priors. Lower errors are better; the lower value for each metric is bold.

Metric	Seen Coeffcient		Coeff Interpolation		Coeff Extrapolation
Metric	Spatiotemporal Interpolation	Temporal Extrapolation	Spatiotemporal Interpolation	Temporal Extrapolation	Spatiotemporal Interpolation	Temporal Extrapolation
$L_{1}$ Abs	0.01218 $\pm$ 0.00218	0.01474 $\pm$ 0.00346	0.01241 $\pm$ 0.00170	0.08957 $\pm$ 0.01627	0.01661 $\pm$ 0.00132	0.01027 $\pm$ 0.00348
$L_{2}$ Rel	0.01643 $\pm$ 0.00250	0.02261 $\pm$ 0.00449	0.01658 $\pm$ 0.00176	0.11990 $\pm$ 0.02624	0.02398 $\pm$ 0.00202	0.01068 $\pm$ 0.00443
$L_{\infty}$ Rel	0.04600 $\pm$ 0.00383	0.08444 $\pm$ 0.01088	0.04274 $\pm$ 0.00036	0.29526 $\pm$ 0.08702	0.08324 $\pm$ 0.00441	0.01661 $\pm$ 0.01137

Appendix

Table 28: Detailed results of our model on the family of 2D CDR equations. All results are reported as mean

\pm

standard deviation over 5 random seeds.

Panel of Figure 5 — Figure 5: Visualization for the family of 2D CDR equation experiments. The left figure is for the spatiotemporal interpolation task and right figure is for the temporal extrapolation

Panel of Figure 6 — Figure 6: Visualization for the SWE experiments. The left figure is for the spatiotemporal interpolation task and right figure is for the temporal extrapolation

Model	Number of parameters	inference time per sample(s)	GPU memory usage(MB)
Ours	4,488,873	0.24	26.64
DeepONet	723,969	0.01	11.06
FNO	143,268,097	0.10	1098.86
F-FNO	23,631,105	0.02	98.47
A-FNO	7,155,472	0.08	35.62
LNO	5,337,857	0.08	24.85

Appendix

Table 29: Cost comparisons between baselines and our model in the family of 2D CDR equations temporal extrapolation test process.

PDE	Task	Train	Valid		Test
PDE	Task	Train	Context	Query	Context	Query
Family of 2D CDR equations	Spatiotemporal interpolation	6,144	1,024	1,024	1,024	4,096
	Temporal extrapolation	6,144	1,024	1,024	1,024	4,096
	Unseen coeff temporal extrapolation	-	-	-	1,024	4,096
SWE	Spatiotemporal interpolation	98,304	16,384	16,384	16,384	65,536
SWE	Temporal extrapolation	98,304	16,384	16,384	16,384	65,536

Appendix

Table 30: Number of data points used for training, validation, and testing in all experiments.

Model	Hyperparameter Name	Family of 2D CDR equations		SWE
		Spatiotemporal interpolation	Temporal extrapolation	Spatiotemporal interpolation	Temporal extrapolation
Ours	Attention layers number	16
	Attention hidden dimension	64
	Attention head number	5
	FFE dimension	0
	AGF depth	15

Appendix

Table 31: Best hyperparameter for each model used in the family of 2D CDR equations and SWE experiments.

Model	Hyperparameter Name	CNSE	Darcy Flow
Ours	Attention layers number	16
	Attention hidden dimension	64
	Attention head number	5
	FFE dimension	0	3
	AGF depth	15
	En/Decoder hidden dimension	512

Appendix

Table 32: Best hyperparameter for each model used in the CNSE and Darcy Flow experiments.

Model	Hyperparameter Name	Operator Learining
Ours	Attention layers number	16
	Attention hidden dimension	64
	Attention head number	5
	FFE dimension	2
	AGF depth	15
	En/Decoder hidden dimension	512

Appendix

Table 33: Best hyperparameter for each model used in the Airfoil experiments.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 269 papers

Error-Conditioned Neural Solvers
Jun 25, 2026Haina Jiang, Liam Wang, Peng-Chen Chen +4Neural PDE Solvers PDE Surrogate Modeling
University of Michigan · KAIST AI · Los Alamos National Laboratory
Unsupervised Adaptation of PDE Foundation Models
Aug 7, 2026Ziye Song, Zhao Wei, Xin Yu +2Unsupervised Domain Adaptation Low-Rank Adaptation
Nanyang Technological University · CFAR, A*STAR · Adelaide University
Flow Learners for PDEs: Toward a Physics-to-Physics Paradigm for Scientific Computing
Apr 2, 2026Yilong Dai, Shengyu Chen, Xiaowei Jia +1Neural PDE Solvers PDE Solving
The University of Alabama, Tuscaloosa, Alabama, USA · University of Pittsburgh, Pittsburgh, Pennsylvania, USA

Jun 25, 2026J/K move · Enter open · S save

Error-Conditioned Neural Solvers

Haina Jiang, Liam Wang, Peng-Chen Chen, Min Seop Kwak+3

University of Michigan · KAIST AI · Los Alamos National Laboratory

Neural PDE Solvers PDE Surrogate Modeling

Open paper AskPDF arXiv

Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss--Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error-conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching

10\times

on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero-shot parameter changes and cross-equation transfer, where its relative advantage is largest in the ill-conditioned regimes where residual minimization is least reliable. Project website: https://neuralsolver.github.io/.

Abstract

Neural PDE Solvers

Explore similar work

Error-Conditioned Neural Solvers

Unsupervised Adaptation of PDE Foundation Models

Flow Learners for PDEs: Toward a Physics-to-Physics Paradigm for Scientific Computing