Neural operators offer fast surrogates for solving PDEs, but training them typically requires expensive input-solution pairs. Two solver-free alternatives include physics-informed learning and training on manufactured data. Physics-informed learning approximates solutions by minimizing PDE residuals. While the residual norm must be strong enough to control solution error, stronger norms increase mini-batch variance and hinder optimization. Manufactured data constructs input-solution pairs for learning by applying the PDE operator to artificial solutions. However, the resulting training distribution can be mutually singular with the target distribution, yielding poor accuracy on the target distribution. We introduce the Wasserstein Data Synthesis Pipeline (WaSP), which trains on manufactured input-solution pairs while gradually mitigating the distribution shift. WaSP synthesizes new pairs by applying the learned operator to a path of intermediate distributions converging to the target input distribution. Training proceeds in stages along this path. At each stage, the previous surrogate generates solutions, and the exact PDE operator derives corresponding inputs, yielding training pairs for the next surrogate. Theoretically, we prove a stepwise error-propagation bound for the distribution shift introduced during synthesis. Empirically, WaSP reduces solution error by 3.9× to 9.5× compared to existing solver-free methods on PDEs such as Allen-Cahn, Kuramoto-Sivashinsky, and Ginzburg-Landau, and better reproduces the two chaotic systems' statistical observables.
Figures & tables
Figure 1: Illustration of how WaSP training distributions are constructed. Top: the McCann interpolation computed for 2D Gaussian point clouds. The gray segments are the estimated optimal transport map T between samples drawn from a reference distribution ν0 and the target ν1 . Each intermediate distribution is defined by the sampling rule in ( 17 ), where a sample from the reference is drawn, mapped by T , and linearly interpolated. Bottom: an input–solution pair for the Kuramoto–Sivashinsky equation interpolated between the manufactured input distribution ( s=0 , left) and the target input distribution ( s=1 , right). The top row is the forcing component of the input, which anneals to near zero as s increases; the bottom row is the corresponding solution field u(t,x) , obtained from an ETD-RK4 solver. As the forcing decreases in magnitude and the initial state moves onto the target attractor, the chaotic dynamics emerge, so the interpolation presents the operator with a sequence of progressively harder problems, forming a training curriculum.
Figure 2: Intuitive picture for local generalization of neural operator and its implications. Left: the operator G0 is trained at s=0 on μ0 with the manufactured-data loss and on ν0=N#μ0 with the PDE loss, then applied at νs,νs′,νs′′ farther along the McCann path (dashed orange); the resulting μ^s,μ^s′,μ^s′′ drift increasingly far from the solution distributions G#νs . Right: mapping each μ^s back through N# (red) gives the realized input distribution ν^s , whose Wasserstein gap W(ν,ν^) to the intended νs (green segments) grows with s . WaSP chooses each increment small enough that νn+1 stays close to νn so that local generalization holds and updates the operator Gn+1 on μ^n+1 and νn+1 , repeating until s=1 .
Figure 3: The data synthesis chain of WaSP described in the text. The input loss ℓPDE (blue) is the residual on the sampled v and reaches Gθ through the first application; the solution loss ℓMD (red) regresses u′ given the realized input v′=N(u) onto the synthesized solution u and reaches Gθ only through the second, because sg(⋅) is a stop-gradient that holds the synthesized pair fixed rather than making it self-referential.
Figure 4: Training dynamics of the practical WaSP algorithm ( Section 4.3 ), plotted against optimizer step, for the complex Ginzburg–Landau experiment of Section 5.1 , whose input is the pair v=(u(0,x),f(t,x)) of initial state and forcing. (a) Dynamic normalization: the EMA of the root-mean-square magnitude of the solution field, the output of Gθ (left), and of the forcing component f of the input (right). (b) Adaptive transport schedule: the interpolation parameter s on the vertical axis. The adaptive schedule advances to the next intermediate distribution only once LPDE has fallen below a threshold, spending more steps on harder portions of the path, whereas the baseline advances on a fixed cosine schedule.
Figure 5: Top row, representative solutions of the three benchmark equations: (a) Allen–Cahn on the training domain [0,5]t×[0,2π]x with u∈[−1,1] , showing phase separation into ±1 domains, and (b) Kuramoto–Sivashinsky on [0,0.64]t×[0,5]x and (c) complex Ginzburg–Landau on [0,32]t×[0,50]x , both with the time axis extended to 5× the training horizon to expose the spatio-temporal chaos. Bottom row, representative samples u∼μ0 from the manufactured solution distribution ( Section 6.1 ), on the training domains (d) [0,5]t×[0,2π]x , (e) [0,0.128]t×[0,5]x and (f) [0,6.4]t×[0,50]x . The complex Ginzburg–Landau fields in (c) and (f) are colored by phase and magnitude (inset color wheel); (c) shows the phase singularities ( ∣u∣=0 ) of the defect-turbulence regime.
Equation
Method
Sol. Rel. L2
Inv. Meas. Dist.
Spat. Corr. RMSE
Temp. Corr. RMSE
Spectrum RMSE
Allen–Cahn
WaSP (ours)
8.52×10−3
—
—
—
—
PINO
3.29×10−2
—
—
—
—
PINO+OT
4.73×10−2
—
—
—
—
MD
1.10×10−1
—
—
—
—
MD+PINO
1.06×10−1
—
—
—
—
KS equation
WaSP (ours)
2.19×10−2
8.69×100
1.02×10−1
8.45×10−2
1.04×10−1
Table 1: Evaluation of WaSP against two baselines and two ablations on the Allen–Cahn, Kuramoto–Sivashinsky (KS), and complex Ginzburg–Landau (CGLE) equations. Every method follows the solver-free training protocol. The baselines are PINO, which minimizes the PDE residual and initial-condition losses, and MD, which regresses on the fixed manufactured pairs (N(u),u) with u∼μ0 . Each ablation is listed under the baseline it extends: PINO+OT trains the residual loss along the McCann interpolation, and MD+PINO is the WaSP objective ( 27 ) frozen at s=0 . Columns are metrics 1–5 of Section 5.3 ; dashes mark the metrics reported only for the chaotic systems. WaSP attains the best or statistically indistinguishable value on every metric and equation.
Figure 6: The panels plot the validation absolute L2 error of the predicted solution on the target distribution ν1 (log scale) against optimizer step, for WaSP, the two baselines (PINO and MD), and the two ablations (PINO+OT and MD+PINO).
Figure 7: The panels plot the validation L2 norm of the predicted PDE residual on the target distribution ν1 (log scale) against optimizer step, for WaSP, the two baselines (PINO and MD), and the two ablations (PINO+OT and MD+PINO). Note that smaller PDE residual does not imply better accuracy as argued in Section 2 .
Figure 8: Function-valued statistical observables for the Kuramoto–Sivashinsky and complex Ginzburg–Landau equations, plotted against the wavenumber k and the separations Δx and Δt . Within each panel, the left plot shows the raw curves against the ETD-RK4 reference and the right plot the absolute deviation from that reference on a symmetric-log scale, whose linear region is marked by evenly spaced minor ticks. The legend at the top applies to all six panels and distinguishes the five methods, the ETD-RK4 reference, and the real and imaginary parts of each complex observable.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Definition
First used
U
Solution function space
Section 2
V
Input function space
Section 2
N:U→V
Differential operator mapping solution to input
Section 2
G:V→U
True solution operator
Section 2
Gθ
Neural solution operator
Section 2
P(X)
Probability distributions on X
Section 2
Appendix
Table 2: Notation used in the theoretical sections.
Allen–Cahn
KS
CGLE
Temporal extent T
5
0.128
6.4
Spatial extent L
2π
5
50
Grid (nt,nx)
(128,128)
(128,256)
(128,256)
Data type
float32
float32
complex64
Diffusion/Hyperviscosity ν
0.01
0.01
—
(α,β)
—
—
(2.0,−2.0)
Appendix
Table 3: Discretization and equation parameters.
Allen–Cahn
KS
CGLE
Input channels
3
3
5
Output channels
1
1
2
Fourier modes (t,x)
(64,128)
(64,64)
(64,64)
Hidden channels
32
32
64
Projection channel ratio
2
2
2
Domain padding (t,x)
(0.1,0)
Appendix
Table 4: FNO architecture. Domain padding and normalization are shared.
Allen–Cahn
KS
CGLE
Optimizer
AdamW
Learning rate
4×10−4
Weight decay
10−4
(β1,β2)
(0.9,0.999)
LR decay factor γ
0.9
Step-decay interval
2000
10000
20000
Appendix
Table 5: Optimizer and learning-rate schedule. Optimizer, learning rate, weight decay, and momenta are shared across equations.
Group
Component
Weight
Solution ( αu )
interior
10
initial condition
1.0
Input ( αv )
interior residual
1.0
initial-condition residual
0.1
Appendix
Table 6: Loss-term weights, common to all equations. The solution group realizes αu in ( 27 ) and the input group realizes αv .
Allen–Cahn
KS
CGLE
Buffer capacity
5000
20000
20000
Initial buffer fill
5000
20000
20000
Inputs reused per fill
32
16
16
Generation chunk size
1000
Buffer (solution) batch
32
Generated (PDE residual) batch
32
Appendix
Table 7: Replay-buffer, batch-size, and normalization settings. Generation chunk size, solution batch, evaluation batch, and normalizer momentum are shared.
Allen–Cahn
KS
CGLE
Sinkhorn transport map
Low-rank cost rank
-
64
96
Fitting samples
-
5000
5000
Entropic regularization
-
10−2
10−3
Max iterations
-
1000
1000
Stopping threshold
-
10−5
10−5
Appendix
Table 8: Optimal-transport and interpolation-schedule settings. Allen–Cahn fits no transport map (its manufactured initial condition distribution already matches the target, the only transport is the zero map of forcing f ), so the Sinkhorn solver settings do not apply; the schedule parameters are shared except where noted.
Configuration
Transport
Buffer batch
Residual batch
Active objective
WaSP (ours)
enabled
32
32
( 27 ) along the path νs
PINO
disabled
0
64
LPDE at ν1 ( s=1 )
PINO+OT
enabled
0
64
LPDE at νs ( s∈[0,1] )
MD
disabled
64
0
LMD on fixed μ0
MD+PINO
disabled
32
32
( 27 ) frozen at (μ0,ν0)
Appendix
Table 9: Configuration of WaSP, the two baselines, and the two ablations. Buffer and residual batches are the solution-loss and residual-loss sub-batches of Table 7 ; they sum to 64 in every case. PINO and PINO+OT additionally set the initial-condition residual weight to 1.0 (equal to the interior residual weight).
Allen–Cahn
KS
CGLE
Solution L2 (metric 1)
yes
yes
yes
Attractor metrics (2–5)
no
yes
yes
Test-set batch size
64
Short-horizon eval (metric 1)
Snapshots
1000
10000
10000
Coarse rollout (metrics 2, 3, 5)
Appendix
Table 10: Evaluation-protocol settings. Trajectory and snapshot counts are shared across equations; every rollout burns in for 10000δt ( 10 time units for KS, 500 for CGLE), and the coarse-rollout interval Δt and Sinkhorn ϵ are set per equation. The four attractor-based metrics are disabled for Allen–Cahn.
Training neural PDE solvers is often bottlenecked by expensive data generation or unstable physics-informed neural network (PINN) involving challenging optimization landscapes due to higher-order derivatives. To tackle this issue, we propose an alternative approach using Monte Carlo approaches to estimate the solution to the PDE as a stochastic process for weak supervision during training. Leveraging the Walk-on-Spheres method, we introduce a learning scheme called Walk-on-Spheres Neural Operator (WoS-NO) which uses weak supervision from WoS to train any given neural operator. We propose to amortize the cost of Monte Carlo walks across the distribution of PDE instances using stochastic representations from the WoS algorithm to generate cheap, noisy, estimates of the PDE solution during training. This is formulated into a data-free physics-informed objective where a neural operator is trained to regress against these weak supervisions, allowing the operator to learn a generalized solution map for an entire family of PDEs. This strategy does not require expensive pre-computed datasets, avoids computing higher-order derivatives for loss functions that are memory-intensive and unstable, and demonstrates zero-shot generalization to novel PDE parameters and domains. Experiments show that for the same number of training steps, our method exhibits up to 8.75× improvement in L2-error compared to standard physics-informed training schemes, up to 6.31× improvement in training speed, and reductions of up to 2.97× in GPU memory consumption. We present the code at https://github.com/neuraloperator/WoS-NO
Hrishikesh Viswanath, Hong Chul Nam, Xi Deng +3
Purdue University, USA · ETH Zurich, Switzerland · California Institute of Technology, USA +1
We present PDEFlow, an autonomous agentic framework that turns user-level ODE and PDE descriptions into solver-backed neural-operator pipelines. The workflow links problem specification, data generation, operator training, and checkpoint-based inference. A stateful input graph converts multi-turn natural-language input and user edits into validated problem specifications. The data-generation module then samples parameters, solves the configured governing-equation with FEniCSx finite-element backend, and stores the solutions as operator-ready tensors. The training and inference stages use a registry-based interface, allowing different neural operators to be trained and deployed without changing the surrounding pipeline. In the current implementation, we instantiate this interface with a multi-branch Bayesian DeepONet. Experiments on benchmark ODE and PDE tasks show that PDEFlow can construct valid specifications, generate solver-backed datasets, train neural operators across steady and transient problem classes, and provide solver-free predictions from saved checkpoints. The framework is designed for repeatable scientific and engineering workflows where many related physics configurations must be specified, simulated, learned, and queried with minimal manual intervention.
Partial differential equation (PDE) simulation holds extensive significance in scientific research. Currently, the integration of deep neural networks to learn solution operators of PDEs has introduced great potential. In this paper, we present UniFluids, a conditional flow-matching framework that harnesses the scalability of diffusion Transformer to unify learning of solution operators across diverse PDEs with varying dimensionality and physical variables. Unlike the autoregressive PDE foundation models, UniFluids adopts flow-matching to achieve parallel sequence generation, making it the first such approach for unified operator learning. Specifically, the introduction of a unified four-dimensional spatiotemporal representation for the heterogeneous PDE datasets enables joint training and conditional encoding. Furthermore, we find the effective dimension of the PDE dataset is much lower than its patch dimension. We thus employ x-prediction in the flow-matching operator learning, which is verified to significantly improve prediction accuracy. We conduct a large-scale evaluation of UniFluids on several PDE datasets covering spatial dimensions 1D, 2D and 3D. Experimental results show that UniFluids achieves strong prediction accuracy and demonstrates good scalability and cross-scenario generalization capability. The code will be released later.
Haosen Li, Qi Meng, Jiahao Li +4
Academy of Mathematics and Systems Science, Chinese Academy of Sciences · Microsoft Research Asia · Renmin University of China +1