cs.LGNov 29, 2022

FC-PINO: High Precision Physics-Informed Neural Operators via Fourier Continuation

Authors: Adarsh Ganeshram, Valentin Duruisseaux, Haydn Maust, Zongyi Li, Yixuan Wang, Daniel Leibovici, Oscar Bruno, Thomas Hou, 

Organizations: Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The physics-informed neural operator (PINO) is a machine learning paradigm that has demonstrated promising results for learning solutions to partial differential equations (PDEs). It leverages the Fourier Neural Operator to learn solution operators in function spaces and leverages physics losses during training to penalize deviations from known physics laws. Spectral differentiation provides an efficient way to compute derivatives for the physics losses, but it inherently assumes periodicity. When applied to non-periodic functions, this assumption can lead to significant errors, including Gibbs phenomena near domain boundaries which degrade the accuracy of both function representations and derivative computations. To overcome this limitation, we introduce the FC-PINO (Fourier-Continuation-based Physics-Informed Neural Operator) architecture which extends the accuracy and efficiency of PINO and spectral differentiation to non-periodic and non-smooth PDEs. In FC-PINO, we propose integrating Fourier continuation into the PINO framework, and test two different continuation approaches: FC-Legendre and FC-Gram. By transforming non-periodic signals into periodic functions on extended domains in a well-conditioned manner, Fourier continuation enables fast and accurate derivative computations. This approach avoids the discretization sensitivity of finite differences and the memory overhead of automatic differentiation. We demonstrate that standard PINO fails (without padding) or struggles (even with padding) to solve non-periodic and non-smooth PDEs with high precision, across challenging benchmarks. In contrast, the proposed FC-PINO provides accurate, robust, and scalable solutions, substantially outperforming PINO alternatives, and demonstrating that Fourier continuation is critical for extending PINO to a wider range of PDE problems when high-precision solutions are needed.

Figures & tables

Figure 1: Examples of periodic extensions obtained using the Fourier continuation methods. (top) Extensions of f(x)=sin⁡(16x)−cos⁡(8x)f(x)=\sin(16x)-\cos(8x) from [0,1][0,1] to [−0.25,1.25][-0.25,1.25] . (bottom) Extensions of f(x,y)=sin⁡(12x)−cos⁡(14y)+3xyf(x,y)=\sin(12x)-\cos(14y)+3xy from [0,1]2[0,1]^{2} to [−0.25,1.25]2[-0.25,1.25]^{2} .
Figure 2: The Fourier Neural Operator (FNO) architecture (extracted from ( Li et al., 2020 ) ).
Figure 3: Visualization of the Gibbs phenomenon when applying spectral differentiation to a non-periodic solution U(y)U(y) to the 1D Burgers’ equation ( 14 ). ( top ) Applying spectral differentiation on the extended domain after using FC–Gram . ( bottom two ) Applying spectral differentiation on the original domain without Fourier continuation. We observe large spurious oscillations, whose magnitude grows rapidly with the derivative order.
Figure 4
Figure 5
Figure 4: Illustration of the various variables used in the Fourier continuation. The given signal ff of nn points given on [a,b][a,b] is extended to a periodic signal with (n+c)(n+c) points, and the extension is split and appended on both sides (each with c/2c/2 points). FC–Legendre and FC–Gram construct periodic extension using only the information contained in the boundary vectors fℓf_{\ell} and frf_{r} of width dd , and not the remaining part fintf_{\text{int}} of the signal.
ModelPDE ResidualBC LossSmoothnessTotal Loss
FC–Gram    
FC–PINO1.9×10−111.9\times 10^{-11}9.2×10−159.2\times 10^{-15}5.5×10−95.5\times 10^{-9}6.7×10−106.7\times 10^{-10}
FC–Legendre    
FC–PINO1.8×10−121.8\times 10^{-12}1.9×10−141.9\times 10^{-14}2.8×10−82.8\times 10^{-8}2.8×10−92.8\times 10^{-9}
Baselines    
Pad–PINO1.4×10−81.4\times 10^{-8}1.1×10−101.1\times 10^{-10}4.9×10−54.9\times 10^{-5}4.9×10−64.9\times 10^{-6}
Table 1: PDE residual, boundary condition (BC) loss, smoothness loss, and total loss for FC–PINO with FC–Gram and FC–Legendre , and baseline PINO architectures when solving the self-similar Burgers’ equation ( 14 ) for λ=0.5\lambda=0.5 .
Model ( FC–Gram)PDE ResidualBC LossSmoothness
λ=1/2\lambda=1/28.8×10−108.8\times 10^{-10}3.7×10−123.7\times 10^{-12}5.9×10−85.9\times 10^{-8}
λ=1/12\lambda=1/122.3×10−102.3\times 10^{-10}6.7×10−136.7\times 10^{-13}6.1×10−86.1\times 10^{-8}
λ=1/22\lambda=1/221.5×10−101.5\times 10^{-10}8.3×10−118.3\times 10^{-11}1.3×10−71.3\times 10^{-7}
λ=1/32\lambda=1/328.3×10−108.3\times 10^{-10}1.8×10−101.8\times 10^{-10}1.6×10−71.6\times 10^{-7}
λ=1/52\lambda=1/522.2×10−92.2\times 10^{-9}8.6×10−108.6\times 10^{-10}3.4×10−73.4\times 10^{-7}
Fine-tuning Average8.6×10−108.6\times 10^{-10}2.3×10−102.3\times 10^{-10}1.5×10−71.5\times 10^{-7}
Table 2: PDE residual, boundary condition (BC) loss, and smoothness loss when solving the self-similar Burgers’ equation with FC–PINO over a family of values of λ\lambda . Here, we report the errors obtained when fine-tuning the pretrained model on the individual values λ∈{1/2,1/12,1/22,1/32,1/52}\lambda\in\{1/2,1/12,1/22,1/32,1/52\} , and the average over these 5 instances.
Figure 5: PDE residual loss across different λ\lambda ’s when fine-tuning the pretrained FC–PINO . The black curve represents the pretrain phase where we use the Adam optimizer for 3000 epochs, learning a solution operator over multiple λ\lambda values. We subsequently switch to the L–BFGS optimizer, as signified by the dashed line, and fine-tune the operator individually on 5 different PDE instances corresponding to different values of λ\lambda .
ModelPDE ResidualIC LossBC Loss
FC–Gram   
FC–PINO1.35×10−71.35\times 10^{-7}7.46×10−7\bm{7.46\times 10^{-7}}5.09×10−75.09\times 10^{-7}
FC–Legendre   
FC–PINO1.17×10−7\bm{1.17\times 10^{-7}}8.25 ×10−78.25\ \times 10^{-7}5.06×10−7\bm{5.06\times 10^{-7}}
Baselines   
Pad–PINO2.8×10−52.8\times 10^{-5}7.0×10−67.0\times 10^{-6}5.3×10−75.3\times 10^{-7}
Table 3: PDE residual, initial condition (IC) and boundary condition (BC) losses for FC–PINO with FC–Gram and FC–Legendre , and baseline PINO architectures for the 2D Burgers’ equation ( 17 ).
ModelPDE ResidualIC LossBC Loss
FC–Gram   
FC–PINO7.5×10−77.5\times 10^{-7}4.5×10−74.5\times 10^{-7}6.7×10−76.7\times 10^{-7}
FC–Legendre   
FC–PINO1.1×10−61.1\times 10^{-6}1.4×10−61.4\times 10^{-6}1.1×10−61.1\times 10^{-6}
Baselines   
Pad–PINO8.0×10−68.0\times 10^{-6}9.2×10−69.2\times 10^{-6}3.5×10−63.5\times 10^{-6}
Table 4: PDE residual, initial condition (IC) loss, and boundary condition (BC) loss for FC–PINO with FC–Gram and FC–Legendre , and baseline PINO architectures for the Navier–Stokes equations ( 19 ).
Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12
Figure 13
ModelPDE ResidualBC LossSmoothnessTotal Loss
FC–Gram    
FC–PINO1.9×10−111.9\times 10^{-11}9.2×10−159.2\times 10^{-15}5.5×10−95.5\times 10^{-9}6.7×10−106.7\times 10^{-10}
PINO In–FC7.9×10−137.9\times 10^{-13}2.9×10−142.9\times 10^{-14}8.3×10−88.3\times 10^{-8}8.3×10−98.3\times 10^{-9}
PINO Out–FC2.1×10−122.1\times 10^{-12}2.1×10−132.1\times 10^{-13}1.3×10−71.3\times 10^{-7}1.3×10−81.3\times 10^{-8}
FC–Legendre    
FC–PINO1.8×10−12\bm{1.8\times 10^{-12}}2.2×10−15\bm{2.2\times 10^{-15}}2.8×10−8\bm{2.8\times 10^{-8}}2.8×10−9\bm{2.8\times 10^{-9}}
Appendix
Table 5: PDE residual, boundary condition (BC) loss, smoothness loss, and total loss for the FC-based PINO architectures using FC–Legendre and FC–Gram when solving the 1D Burgers’ equation for λ=0.5\lambda=0.5 .
ModelPDE ResidualIC LossBC Loss
FC–Gram   
FC–PINO1.35×10−7\bm{1.35\times 10^{-7}}7.46×10−7\bm{7.46\times 10^{-7}}5.09×10−7\bm{5.09\times 10^{-7}}
PINO In–FC1.73×10−71.73\times 10^{-7}7.55×10−77.55\times 10^{-7}5.21×10−75.21\times 10^{-7}
PINO Out–FC3.48×10−53.48\times 10^{-5}6.89×10−66.89\times 10^{-6}6.21×10−76.21\times 10^{-7}
FC–Legendre   
FC–PINO1.17×10−7\bm{1.17\times 10^{-7}}8.25×10−7\bm{8.25\times 10^{-7}}5.06×10−7\bm{5.06\times 10^{-7}}
Appendix
Table 6: PDE residual, initial condition (IC) loss, and boundary condition (BC) loss for the FC-based PINO architectures using FC–Legendre and FC–Gram when solving the 2D Burgers’ equation ( 17 ).
ModelPDE ResidualIC LossBC Loss
FC–Gram   
FC–PINO7.5×10−77.5\times 10^{-7}4.5×10−74.5\times 10^{-7}6.7×10−76.7\times 10^{-7}
PINO In–FC1.3×10−61.3\times 10^{-6}9.6×10−79.6\times 10^{-7}1.9×10−61.9\times 10^{-6}
PINO Out–FC1.3×10−51.3\times 10^{-5}1.7×10−61.7\times 10^{-6}7.2×10−67.2\times 10^{-6}
FC–Legendre   
FC–PINO1.1×10−6\bm{1.1\times 10^{-6}}1.4×10−61.4\times 10^{-6}1.1×10−6\bm{1.1\times 10^{-6}}
Appendix
Table 7: PDE residual, initial condition (IC) loss, and boundary condition (BC) loss for the FC-based PINO architectures using FC–Legendre and FC–Gram when solving the Navier–Stokes equations ( 19 ).
Figure 6: Effect of varying continuation parameters cc (continuation points) and d (degree) for FC–Legendre on Total Loss (left) and PDE Residual (right)
 Total LossPDE Residual
d=2d=21.05×10−81.05\times 10^{-8}3.74×10−103.74\times 10^{-10}
d=3d=34.89×10−94.89\times 10^{-9}9.52×10−109.52\times 10^{-10}
d=4d=47.34×10−107.34\times 10^{-10}1.25×10−101.25\times 10^{-10}
d=5d=58.29×10−108.29\times 10^{-10}1.12×10−91.12\times 10^{-9}
d=6d=66.72×10−10\mathbf{6.72\times 10^{-10}}1.92×10−11\mathbf{1.92\times 10^{-11}}
d=7d=76.60×10−96.60\times 10^{-9}1.07×10−91.07\times 10^{-9}
Appendix
Table 8: Effect of the choice of continuation parameters dd for FC–Gram on the Total Loss and PDE Residual when using FC–PINO to solve the 1D Burgers’ equation ( 14 ).
ModelFC MethodddccFourier ModesLearning Rate
FC–PINOFC–Gram650240.0001
FC–Legendre470480.001
PINO In–FCFC–Gram450240.005
FC–Legendre570240.0001
PINO Out–FCFC–Gram450240.001
FC–Legendre650480.005
Appendix
Table 9: Optimal Hyperparameters for the PINO models for the 1D Burgers equation.
ModelFC MethodddccLearning Rate
FC–PINOFC–Gram4500.0001
FC–Legendre6300.001
PINO In–FCFC–Gram6500.0001
FC–Legendre6400.001
PINO Out–FCFC–Gram6500.0001
FC–Legendre6800.001
Appendix
Table 10: Optimal Hyperparameters for the PINO models for the 2D Burgers equation.
ModelFC MethodddccSpatial ModesTemporal Modes
FC–PINOFC–Gram8503220
FC–Legendre7403220
PINO In–FCFC–Gram7503224
FC–Legendre8603224
PINO Out–FCFC–Gram8503220
FC–Legendre8603224
Appendix
Table 11: Optimal Hyperparameters for the PINO models for the 3D Navier–Stokes equation.
Figure 7: Predicted solutions of PINO architectures for the 1D Burger’s equation.
Figure 8: Predicted solutions when fine-tuning pretrained FC–PINO models on different λ\lambda values for the 1D Burgers’ equation.
Figure 9: Predicted solutions ( top five ) and corresponding PDE residuals ( bottom five ) for baseline PINO architectures and FC–PINO variants for the 2D Burgers’ equation.
Figure 10: Predicted solutions ( top five ) and corresponding PDE residuals ( bottom five ) for baseline PINO architectures and FC–PINO variants for the horizontal velocity component of the Navier–Stokes equations.
Figure 11: Predicted solutions ( top five ) and corresponding PDE residuals ( bottom five ) for baseline PINO architectures and FC–PINO variants for the horizontal velocity component of the Navier–Stokes equations.
Figure 12: Evolution of the PDE residual loss for the 1D Burgers’ equation with FC–PINO .
Figure 13: Evolution of the PDE residual loss for the 2D Burgers’ equation with FC–PINO .
Figure 14: Evolution of the PDE residual for the Navier–Stokes equations with FC–PINO .
Figure 15: Evolution of the total loss as a function of time for the 1D Burgers’ equation with the different PINO approaches. To allow for a cleaner visualization, we display the time taken to reach certain threshold loss values for the first time as a solid line. The dotted line then connects the loss at the last threshold value and the loss at 360360 seconds.
Figure 16: Evolution of the PDE residual loss as a function of time for the 2D Burgers’ equation with the different PINO approaches. To allow for a cleaner visualization, we display the time taken to reach certain threshold loss values for the first time as a solid line. The dotted line then connects the loss at the last threshold value and the loss at 6060 minutes.
Figure 17: Evolution of the PDE residual as a function of time for the Navier–Stokes equations with the different PINO approaches. To allow for a cleaner visualization, we display the time taken to reach certain threshold loss values for the first time as a solid line. The dotted line then connects the loss at the last threshold value and the final loss at the final value.
More in this topic

Physics-Informed Neural Operators

Physics-informed neural operators learn mappings between physical fields while embedding governing equations or structural constraints for faster, more reliable simulation.

See all 72 papers

Explore similar work

CardsList
Feb 16, 2026

Pseudo-differential-enhanced physics-informed neural networks

We present pseudo-differential enhanced physics-informed neural networks (PINNs), an extension of gradient enhancement but in Fourier space. Gradient enhancement of PINNs dictates that the PDE residual is taken to a higher differential order than prescribed by the PDE, added to the objective as an augmented term in order to improve training and overall learning fidelity. We propose the same procedure after application via Fourier transforms, since differentiating in Fourier space is multiplication with the Fourier wavenumber under suitable decay. Our methods are fast and efficient. Our methods oftentimes achieve superior PINN versus numerical error in fewer training iterations, potentially pair well with few/fixed samples in collocation, and can achieve breakthrough rather than gradual effects on the error. Moreover, our methods are suitable for fractional derivatives. We establish that our methods, due to the dynamical effects, improve spectral eigenvalue decay of the neural tangent kernel (NTK), and so our methods contribute towards the learning of high frequencies in early training, mitigating the effects of frequency bias up to the polynomial order and possibly greater with smooth activations. In particular, our primary contribution can be characterized as gradient enhancement affects spectral bias, and we specialize to Fourier space, although our theory does not prove the ambient case exactly due to a result with the Plancherel theorem on the descent paths. Our methods accommodate advanced techniques in PINNs, such as Fourier feature embeddings. A pitfall of discrete Fourier transforms via the Fast Fourier Transform (FFT) is mesh subjugation, and so we demonstrate compatibility of our methods for greater mesh flexibility and invariance on alternative Euclidean and non-Euclidean domains via Monte Carlo methods and otherwise, although possessing trade-offs of their own.

Andrew Gracyk

Department of Mathematics, Purdue University, West Lafayette, IN 47907, United States

PDE SolvingSpectral BiasPhysics-Informed ML
AskPDF
Oct 2, 2026

Operator-informed initialization for Fourier features physics-informed neural networks

Physics-Informed Neural Networks (PINNs) typically exhibit spectral bias, where some frequencies of the target function converge more slowly than others. In this work, we analyze the training dynamics of Fourier Feature PINNs in the Neural Tangent Kernel regime to address this limitation. We derive an explicit evolution equation to estimate the residual error in the frequency domain, demonstrating that the convergence rate of specific frequencies is primarily governed by the product of the differential operator's symbol and the spectral density of the initialization weights. Leveraging this theoretical insight, we propose an informative initialization strategy that tailors the initial weight distribution to the specific PDE being solved. With this method, we can diminish the operator-induced spectral bias, balancing the convergence rates across the frequency spectrum and achieving better prediction accuracy. Numerical experiments on linear and nonlinear partial differential equations confirm that this initialization strategy improves learning dynamics and approximation accuracy across frequencies compared to standard initialization methods, with no additional training cost.

Juan Molina, Paris Perdikaris, Mircea Petrache +2

National Center for Artificial Intelligence, Santiago, Chile · Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Pontificia Universidad Católica de Chile, Santiago, Chile +3

Physics-Informed MLNeural Network InitializationSpectral BiasFourier Feature Embeddings
AskPDF
Jun 4, 2026

On the training of physics-informed neural operators for solving parametric partial differential equations

Physics-informed neural operators (PINOs) aim to learn solution operators for partial differential equations by using the governing physics as supervision, rather than relying solely on paired input-output simulation data. By incorporating physical constraints into the training objective, PINOs combine the cross-instance generalization of neural operators with the data efficiency of physics-informed learning. Despite this promise, how to train PINOs efficiently and robustly remains less well-understood than the training of either data-driven neural operators or physics-informed neural networks (PINNs). To bridge this gap, we examine key components of the PINO training pipeline, including architecture design, optimizer choice, loss balancing, and collocation-point sampling strategy. We study three representative operator backbones, Deep Operator Network (DeepONet), Fourier Neural Operator (FNO), and Continuous Vision Transformer (CViT), across five diverse parametric PDE systems. Our results show that CViT provides consistently strong and stable performance across the considered benchmarks. Beyond architecture, we find that several optimization pathologies previously identified in PINN training naturally arise in PINOs, including gradient conflicts and causal violation. We also find that mitigation algorithms developed for PINNs remain effective in the PINO setting. We further compare physics-informed and data-driven training under different data regimes, revealing that a carefully designed physics-informed training pipeline can match, and in some cases, outperform purely data-driven neural operators. Taken together, these findings provide a systematic empirical understanding of the optimization challenges in PINO training and inform a practical pipeline for efficient and robust physics-informed operator learning. Code and data are available at https://github.com/NanxiiChen/PI-CViT.

Nanxi Chen, Chuanjie Cui, Airong Chen +2

College of Civil Engineering, Tongji University, Shanghai 200092 · Department of Engineering Science, University of Oxford, Oxford OX1 3PJ · Institute for Foundations of Data Science, Yale University, New Haven, CT 06520

Neural Network OptimizationGradient InterferencePDE Operator LearningPhysics-Informed MLPhysics-Informed Neural Operators
AskPDF