cs.LGOct 3, 2026

CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations

Authors: Jeahan Jung, Heechang Kim, Hyomin Shin, Minseok Choi

Organizations: Department of Mathematics, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Despite the growing popularity of physics-informed neural networks (PINNs), their applicability in the long-time integration of partial differential equations (PDEs) remains constrained. We argue that this problem stems from the lack of consideration of temporal causality in the original PINN formulation, resulting in a bias towards satisfying governing equations at later times before learning the initial condition and hence leading to erroneous solutions. To this end, we propose a novel method that seamlessly integrates temporal causality into the training process. Drawing inspiration from classical numerical methods where the temporal causality is reflected, we divide the time domain into nonoverlapping subintervals, assign a unique neural network to each subinterval, and construct a loss function founded on the integral form of PDEs within these subintervals. The proposed networks undergo sequential training, beginning with the initial time step. Our method demonstrates significant improvement in accuracy for long-time simulations of various PDE problems where the original PINN method fails while it requires less computational cost and memory compared to the PINN method. A parallelization algorithm is provided to further enhance the computational efficiency, showing a significant speedup for solving time-dependent PDEs.

Figures & tables

Figure 1: PINN performance for the one-dimensional convection equation. (a) The prediction of a trained conventional PINN versus reference solution. The PINN model provides an inaccurate prediction. (b) Initial condition loss of PINN and residual loss of PINN during training. (c) Residual losses of PINN at t=0t=0 and 1. The residual loss of PINN at t=1t=1 remains low, while that of PINN at t=0t=0 remains large. This suggests a violation of temporal causality in the learning process.
Figure 2: Schematic of CEEN. The temporal domain is discretized into subintervals, with a neural network assigned to each time step. The solution is obtained by sequentially training each network, beginning from the initial time. This approach reflects temporal causality in the training process.
ExampleRelative error
CEENPINN
KdV5.314e-31.237e-0
Schrödinger4.793e-32.346e-1
Convection5.465e-31.000e-0
Allen-Cahn4.222e-21.394e-0
Table 1: Relative errors of CEEN and PINN at the final time t=5t=5 for four PDE problems.
Figure 3: KdV equation: Comparison of the prediction of CEEN and PINN against the reference solution. (a) displays the prediction across the entire spatio-temporal domain. (b) displays it at three different times: t=1,3,5t=1,3,5 . CEEN agrees well with the reference in the entire domain while PINN exhibits a less accurate approximation of the solution even at an earlier time t=1t=1 .
Figure 4: Residual losses of CEEN and PINN for the KdV equation ( 12 ). (a) Epoch vs residual loss (b) Time vs residual loss.
Figure 5: Schrödinger equation: Comparison of the prediction of CEEN and PINN against the reference solution. (a) displays the prediction across the entire spatio-temporal domain. (b) displays it at three different times: t=1,3,5t=1,3,5 . CEEN agrees well with the reference in the entire domain while PINN closely approximates the solution at an earlier time t=1t=1 . However, discrepancies between PINN predictions and the reference solution become apparent at later times t=3t=3 and t=5t=5 .
Figure 6: Residual losses of CEEN and PINN for the Schrödinger equation ( 13 ). (a) Epoch vs residual loss (b) Time vs residual loss.
Figure 7: Convection equation: Comparison of the prediction of CEEN and PINN against the reference solution. (a) displays the prediction across the entire spatio-temporal domain. (b) displays it at three different times: t=0.25,0.5,5t=0.25,0.5,5 . CEEN agrees well with the reference in the entire domain while PINN fails to approximate the solution.
Figure 8: Relative errors of CEEN and PINN for the convection equation ( 14 ). Shaded area is a standard deviation band for the prediction of CEEN obtained by running ten independent simulations.
Figure 9: Relative error of our method at final time (T=5)(T=5) for different time step sizes hh and tolerances ϵ\epsilon for the convection equation ( 14 ). (a) Time step size vs relative error. The second order of accuracy with respect to the time step size hh is achieved for sufficiently small tolerance ϵ\epsilon . (b) Tolerance vs relative error. 1/2 order of accuracy with respect to the tolerance ϵ\epsilon is achieved for sufficiently small hh .
Figure 10: Allen-Cahn equation: Comparison of the prediction of CEEN and PINN against the reference solution. (a) displays the prediction across the entire spatio-temporal domain. (b) displays it at three different times: t=0.5,1.5,5t=0.5,1.5,5 . CEEN agrees well with the reference in the entire domain while PINN fails to approximate the solution.
Figure 11: Relative errors of CEEN and PINN for the Allen-Cahn equation ( 16 ). Shaded area is a standard deviation band for the prediction of CEEN obtained by running ten independent simulations.
 Architecture Training
NN shape[2,15,15,1]optimizerAdam
activationsin⁡\sinlearning rate10−310^{-3}
MM10optimizer parametersβ=(0.999,0.999)\beta=(0.999,0.999)
KK1ϵ\epsilon10−510^{-5}
  Maximum iteration100,000100,000
Table 2: CEEN parameters for the convection equation
Figure 12: Comparison of CEEN and finite difference method (FDM). (a) Relative errors at the final time t=5t=5 vs. NxN_{x} . (b) Predictions of solution with Nx=32N_{x}=32 at three different times.
Figure 13: Relative L2L_{2} errors vs. dtdt at the final time t=5t=5 for FDM (a) and CEEN (b).
ExampleSettings
ArchitectureActivationNxN_{x}NtN_{t}ϵ\epsilonKK
KdV[5,16,16,1]tanh15150110−410^{-4}3
Schrödinger[3,16,16,2]tanh15110110−510^{-5}2
Convection[2,15,15,1]tanh101100110−510^{-5}1
Allen-Cahn[1,32,32,1]tanh20150110−510^{-5}2
Table 3: CEEN hyperparameters for four PDE problems.
Figure 14: Comparison of computational cost between PINN and CEEN ( M=1M=1 ). y-axis: the ratio of the computation time of CEEN to that of PINN. For all four problems, the computational cost of PINN is much larger than that of CEEN.
Figure 15: Comparison of computational cost across different MM values in CEEN. y-axis: the ratio of the computation time for the cases in parallel ( M>1M>1 ) to that for the case without parallelization ( M=1M=1 ).
ExampleM
12510
KdV2.159e-22.433e-21.700e-22.600e-2
Schrödinger3.584e-33.573e-35.080e-34.129e-3
Convection9.232e-39.293e-39.199e-39.204e-3
Allen-Cahn4.200e-23.325e-22.229e-22.049e-2
Table 4: The relative L2L^{2} error at final time for four PDE problems across different values of MM
More in this topic

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 272 papers

Explore similar work

CardsList
Sep 29, 2026

CI-PINN: Causal Integral Physics-Informed Neural Network for Solving Evolution Equations

Physics-informed neural networks (PINNs) solve partial differential equations (PDEs) by incorporating governing physical laws into the training loss. For evolution equations, however, their conventional pointwise space--time representation does not explicitly encode temporal dependence, which can hinder accurate prediction. To mitigate this limitation, this work proposes a novel neural architecture termed a causal integral neural network (CinNet). The core module of CinNet is a Volterra-type causal integral term, which aggregates historical features to encode temporal dependence, thereby incorporating temporal causality at the architectural level rather than through training-level modifications as in many existing methods. Building on CinNet, we further develop a causal integral physics-informed neural network (CI-PINN) for solving evolution equations. Extensive numerical experiments on benchmark evolution equations demonstrate that the presented method outperforms various baseline PINN variants in terms of solution accuracy, with pronounced superiority under sparse-collocation scenarios. Additional empirical analyses show that CI-PINN exhibits low sensitivity to hyperparameter choices, while ablation studies confirm the effectiveness of the proposed network components.

Xiaodong Feng, Ziyu Sun, Tao Tang +2

Institute for Advanced Study, Beijing Normal-Hong Kong Baptist University, Zhuhai, Guangdong, China. · School of Mathematical Sciences, Beijing Normal University, Beijing 100875, China. · School of Mathematics and Statistics, Guangzhou Nanfang College, Guangzhou, Guangdong, China. +2

Neural PDE SolversPDE SolvingPhysics-Informed ML
AskPDF
May 29, 2026

Oscillatory State-Space Models as Inductive Biases for Physics-Informed Neural PDE Solvers

Solving time-dependent partial differential equations (PDEs) is an important problem in computational science and engineering. Physics-informed neural networks (PINNs) learn PDE solutions from governing equations. However, accurately capturing temporal evolution remains challenging. Recent sequence-model-based approaches parameterize time evolution using general-purpose sequence models, which capture temporal dependencies but do not explicitly encode the structured dynamics of PDE solutions. In addition, their memory requirements can scale unfavorably with sequence length and resolution, limiting applicability in large-scale or high-dimensional settings. This work introduces a PINN approach that incorporates oscillatory state-space dynamics to represent the modal structure of PDE solutions. The proposed method leverages a linear-oscillator-based temporal evolution, together with a PDE-aware spectral basis in space. This design enables closed-form spatial differentiation and consistent enforcement of boundary conditions. The method is evaluated on forward, inverse, and high-dimensional PDE problems, including cases up to 100 spatial dimensions. The results show improved accuracy and reduced memory usage compared to recent sequence-model-based PINN approaches. Overall, this work highlights the benefits of incorporating structured dynamical priors into the temporal evolution of neural PDE solvers and suggests designing more physics-aligned and computationally efficient PINN architectures.

Abhishek Chandra, Taniya Kapoor

KTH Royal Institute of Technology Stockholm, Sweden · Wageningen University & Research Wageningen, The Netherlands

Neural PDE SolversPhysics-Informed MLPDE Inverse ProblemsState Space Models
AskPDF
Jul 13, 2026

SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations

Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs), but suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions. We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation. We present SPARC-Net, a unified architecture and training framework that jointly addresses all four pathologies. SPARC-Net leverages an adaptive multi-scale spectral encoder with a learnable spectral gate, a gated residual backbone, adaptive activations, and a hard-constraint output ansatz that exactly enforces initial and boundary conditions, structurally eliminating loss-weight collapse. Training employs stabilized gradient-norm loss balancing, floored causality-respecting residual weighting, and residual-based adaptive collocation (RAD). Validated against exact analytic and high-order spectral reference solutions across four canonical benchmarks -- viscous Burgers', Allen-Cahn, convection (beta=30), and reaction -- SPARC-Net yields substantial improvements over vanilla PINNs: relative L2 error drops from 1.47e-1 to 1.14e-1 on Burgers' (22% reduction), 9.93e-1 to 5.78e-2 on Allen-Cahn (94% reduction), and 9.82e-1 to 3.54e-3 on reaction (100% reduction). A characteristic-coordinate encoder for hyperbolic transport further reduces convection error from 5.14e-1 to 9.88e-5 (100% reduction). We report five-seed mean +/- standard deviation errors, Wilcoxon significance tests, full ablation studies, hyperparameter sensitivities, an extension to the 2D heat equation, and comparisons against parameter-matched baselines.

Divyavardhan Singh, Dimple Sonone, Hammad Mohammad +1

Dept. of Electronics Engg. SVNIT Surat, India

PDE SolvingSpectral BiasPhysics-Informed MLPartial Differential Equations
AskPDF