CI-PINN: Causal Integral Physics-Informed Neural Network for Solving Evolution Equations
Authors: Xiaodong Feng, Ziyu Sun, Tao Tang, Xiaoliang Wan, Tao Zhou
Organizations: Institute for Advanced Study, Beijing Normal-Hong Kong Baptist University, Zhuhai, Guangdong, China. · School of Mathematical Sciences, Beijing Normal University, Beijing 100875, China. · School of Mathematics and Statistics, Guangzhou Nanfang College, Guangzhou, Guangdong, China. · Department of Mathematics and Center for Computation and Technology, Louisiana State University, Baton Rouge 70803, USA. · SKLMS & Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences.
Physics-informed neural networks (PINNs) solve partial differential equations (PDEs) by incorporating governing physical laws into the training loss. For evolution equations, however, their conventional pointwise space--time representation does not explicitly encode temporal dependence, which can hinder accurate prediction. To mitigate this limitation, this work proposes a novel neural architecture termed a causal integral neural network (CinNet). The core module of CinNet is a Volterra-type causal integral term, which aggregates historical features to encode temporal dependence, thereby incorporating temporal causality at the architectural level rather than through training-level modifications as in many existing methods. Building on CinNet, we further develop a causal integral physics-informed neural network (CI-PINN) for solving evolution equations. Extensive numerical experiments on benchmark evolution equations demonstrate that the presented method outperforms various baseline PINN variants in terms of solution accuracy, with pronounced superiority under sparse-collocation scenarios. Additional empirical analyses show that CI-PINN exhibits low sensitivity to hyperparameter choices, while ablation studies confirm the effectiveness of the proposed network components.
Figures & tables
Figure 1: Allen–Cahn equation. Time-dependent residual loss and relative L2 error for PINN and CI-PINN. Both methods are trained on the same fixed grid with (Nt,Nx)=(20,256) . The CI-PINN result is included as a preview, and the method is introduced in Section 3 .
Figure 2: Schematic of CinNet. (a) Overall propagation of the current-time and historical feature streams through successive causal integral layers. (b) Construction of a single CinNet layer, in which the local feature and the history-dependent integral feature are combined through a learnable gate.
Figure 3: Allen–Cahn equation. Relative L2 errors ( 4.1 ) versus the causality parameter ϵ in ( B.3 ) for Causal PINN and the number of quadrature nodes Ns in ( 3.4 ) for CI-PINN.
Nt
Nx
PINN
Causal PINN
CI-PINN
10
64
(6.98±0.17)×10−1
(2.45±0.32)×10−1
(1.35±0.54)×10−2
128
(5.16±0.04)×10−1
(1.98±1.29)×10−2
(2.99±0.31)×10−3
256
(5.16±0.03)×10−1
(1.13±0.33)×10−2
(3.96±0.53)×10−3
20
64
(6.96±0.19)×10−1
(9.40±4.10)×10−2
(1.35±0.51)×10−2
128
(5.13±0.01)×10−1
(4.15±1.34)×10−3
(1.80±0.48)×10−3
256
(5.15±0.01)×10−1
(2.96±0.29)×10−3
(1.94±0.39)×10−3
Table 1: Allen–Cahn equation. Relative L2 errors of different methods for different training grid sizes (Nt,Nx) . The reported values are the mean and standard deviation over five independent runs. For Causal PINN, the selected parameter ϵ=102 is used.
ωic
PINN
Causal PINN
CI-PINN
100
(6.93±3.82)×10−1
(3.94±1.85)×10−3
(1.51±0.36)×10−3
101
(4.79±0.45)×10−1
(1.35±0.44)×10−3
(7.05±1.56)×10−4
102
(5.15±0.01)×10−1
(2.96±0.29)×10−3
(1.94±0.39)×10−3
103
(5.16±0.01)×10−1
(6.35±1.28)×10−3
(3.07±0.75)×10−3
104
(5.49±0.03)×10−1
(4.33±0.26)×10−1
(6.54±1.95)×10−3
Table 2: Allen–Cahn equation. Relative L2 errors of different methods for different initial-condition loss weights ωic . The reported values are the mean and standard deviation over five independent runs. For Causal PINN, the selected parameter ϵ=102 is used.
Figure 4: KdV equation. Comparison of the reference solution with the PINN and CI-PINN solution profiles at t=0 , 0.5 , and 1 . For each method, the prediction from the best-performing run among five independent runs is shown. The corresponding relative L2 errors are 1.87×10−1 for PINN and 6.94×10−3 for CI-PINN.
Figure 5: KdV equation. Relative L2 errors ( 4.1 ) versus the causality parameter ϵ in ( B.3 ) for Causal PINN and the number of quadrature nodes Ns in ( 3.4 ) for CI-PINN.
Figure 6: Cahn–Hilliard equation. Comparison of the reference solution with the PINN and CI-PINN solution profiles at t=0 , 0.5 , and 1 . For each method, the prediction from the best-performing run among five independent runs is shown. The corresponding relative L2 errors are 4.07×10−1 for PINN and 5.68×10−3 for CI-PINN.
Figure 7: Cahn–Hilliard equation. Relative L2 errors ( 4.1 ) versus the causality parameter ϵ in ( B.3 ) for Causal PINN and the number of quadrature nodes Ns in ( 3.4 ) for CI-PINN.
Temporal weight
Gating mechanism
Relative L2 error
✓
✓
(6.06±0.28)×10−3
✓
✗
(1.95±0.52)×10−1
✗
✓
(3.35±0.39)×10−1
✗
✗
(4.31±0.22)×10−1
Table 3: Cahn–Hilliard equation. Ablation study of the architectural components in CinNet with ωic=100 , (Nt,Nx)=(40,256) , and Ns=160 . For the temporal weight, ✗ denotes A(ℓ)(t,s)=1 . For the gating mechanism, ✗ denotes that z(ℓ) and 1−z(ℓ) are both fixed to 0.5 in ( 3.2 ).
Figure 8: Two-dimensional Allen–Cahn equation. Comparison of the reference solution with the CI-PINN prediction and pointwise absolute error. CI-PINN is trained with (Nt,Nxy)=(20,1024) and uses Ns=80 quadrature nodes. Panel (a) shows the two-dimensional spatial snapshots at t=0 , 0.5 , and 1.0 , where each row corresponds to one time level and the three columns show the reference solution, prediction, and pointwise absolute error, respectively. Panel (b) shows the corresponding space–time cross sections on the plane y=0 , with t on the horizontal axis and x on the vertical axis. The relative L2 error of CI-PINN is 4.29×10−3 from the best-performing run among five independent runs.
Figure 9: Two-dimensional Allen–Cahn equation. Relative L2 errors ( 4.1 ) versus the causality parameter ϵ in ( B.3 ) for Causal PINN and the quadrature node counts Ns in ( 3.4 ) for CI-PINN.
Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.
Despite the growing popularity of physics-informed neural networks (PINNs), their applicability in the long-time integration of partial differential equations (PDEs) remains constrained. We argue that this problem stems from the lack of consideration of temporal causality in the original PINN formulation, resulting in a bias towards satisfying governing equations at later times before learning the initial condition and hence leading to erroneous solutions. To this end, we propose a novel method that seamlessly integrates temporal causality into the training process. Drawing inspiration from classical numerical methods where the temporal causality is reflected, we divide the time domain into nonoverlapping subintervals, assign a unique neural network to each subinterval, and construct a loss function founded on the integral form of PDEs within these subintervals. The proposed networks undergo sequential training, beginning with the initial time step. Our method demonstrates significant improvement in accuracy for long-time simulations of various PDE problems where the original PINN method fails while it requires less computational cost and memory compared to the PINN method. A parallelization algorithm is provided to further enhance the computational efficiency, showing a significant speedup for solving time-dependent PDEs.
Jeahan Jung, Heechang Kim, Hyomin Shin +1
Department of Mathematics, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs), but suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions. We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation. We present SPARC-Net, a unified architecture and training framework that jointly addresses all four pathologies. SPARC-Net leverages an adaptive multi-scale spectral encoder with a learnable spectral gate, a gated residual backbone, adaptive activations, and a hard-constraint output ansatz that exactly enforces initial and boundary conditions, structurally eliminating loss-weight collapse. Training employs stabilized gradient-norm loss balancing, floored causality-respecting residual weighting, and residual-based adaptive collocation (RAD). Validated against exact analytic and high-order spectral reference solutions across four canonical benchmarks -- viscous Burgers', Allen-Cahn, convection (beta=30), and reaction -- SPARC-Net yields substantial improvements over vanilla PINNs: relative L2 error drops from 1.47e-1 to 1.14e-1 on Burgers' (22% reduction), 9.93e-1 to 5.78e-2 on Allen-Cahn (94% reduction), and 9.82e-1 to 3.54e-3 on reaction (100% reduction). A characteristic-coordinate encoder for hyperbolic transport further reduces convection error from 5.14e-1 to 9.88e-5 (100% reduction). We report five-seed mean +/- standard deviation errors, Wilcoxon significance tests, full ablation studies, hyperparameter sensitivities, an extension to the 2D heat equation, and comparisons against parameter-matched baselines.
Divyavardhan Singh, Dimple Sonone, Hammad Mohammad +1
Solving time-dependent partial differential equations (PDEs) is an important problem in computational science and engineering. Physics-informed neural networks (PINNs) learn PDE solutions from governing equations. However, accurately capturing temporal evolution remains challenging. Recent sequence-model-based approaches parameterize time evolution using general-purpose sequence models, which capture temporal dependencies but do not explicitly encode the structured dynamics of PDE solutions. In addition, their memory requirements can scale unfavorably with sequence length and resolution, limiting applicability in large-scale or high-dimensional settings. This work introduces a PINN approach that incorporates oscillatory state-space dynamics to represent the modal structure of PDE solutions. The proposed method leverages a linear-oscillator-based temporal evolution, together with a PDE-aware spectral basis in space. This design enables closed-form spatial differentiation and consistent enforcement of boundary conditions. The method is evaluated on forward, inverse, and high-dimensional PDE problems, including cases up to 100 spatial dimensions. The results show improved accuracy and reduced memory usage compared to recent sequence-model-based PINN approaches. Overall, this work highlights the benefits of incorporating structured dynamical priors into the temporal evolution of neural PDE solvers and suggests designing more physics-aligned and computationally efficient PINN architectures.
Abhishek Chandra, Taniya Kapoor
KTH Royal Institute of Technology Stockholm, Sweden · Wageningen University & Research Wageningen, The Netherlands