A hybrid FEM-PINN method for time-dependent partial differential equations
Authors: Xiaodong Feng, Haojiong Shangguan, Tao Tang, Xiaoliang Wan
Organizations: Division of Science and Technology, BNU-HKBU United International College, Zhuhai 519087, China · School of Electrical and Computer Engineering, Guangzhou Nanfang College, Guangzhou 510970, China · Department of Mathematics and Center for Computation and Technology, Louisiana State University, Baton Rouge 70803, USA
In this work, we present a hybrid numerical method for solving evolution partial differential equations (PDEs) by merging the time finite element method with deep neural networks. In contrast to the conventional deep learning-based formulation where the neural network is defined on a spatiotemporal domain, our methodology utilizes finite element basis functions in the time direction where the space-dependent coefficients are defined as the output of a neural network. We then apply the Galerkin or collocation projection in the time direction to obtain a system of PDEs for the space-dependent coefficients which is approximated in the framework of PINN. The advantages of such a hybrid formulation are twofold: statistical errors are avoided for the integral in the time direction, and the neural network's output can be regarded as a set of reduced spatial basis functions. To further alleviate the difficulties from high dimensionality and low regularity, we have developed an adaptive sampling strategy that refines the training set. More specifically, we use an explicit density model to approximate the distribution induced by the PDE residual and then augment the training set with new time-dependent random samples given by the learned density model. The effectiveness and efficiency of our proposed method have been demonstrated through a series of numerical experiments.
Figures & tables
Figure 1: Schematic of the proposed approach.
Figure 2: Convection equation. Exact solutions. From left to right: β=10,30 and 50 .
Figure 3: Convection equation. Numerical results with linear finite element discretization.
Figure 4: Convection equation. Convergence rate for linear finite element basis functions versus quadratic finite element basis functions.
N
32
64
96
128
Galerkin (linear)
1.69e+00 ± 2.38e-05
7.73e-02 ± 1.23e-05
1.91e-02 ± 1.06e-05
8.69e-03 ± 1.18e-05
Galerkin (quadratic)
3.83e-02 ± 8.53e-06
5.58e-03 ± 1.15e-05
2.26e-03 ± 2.67e-05
1.13e-03 ± 3.74e-05
Collocation (Hermite polynomial)
1.97e-01 ± 1.78e-06
1.18e-02 ± 6.76e-07
2.34e-03 ± 2.17e-04
7.50e-04 ± 2.92e-04
Collocation (spline)
6.44e-01 ± 2.29e-05
1.83e-02 ± 1.13e-06
3.16e-03 ± 2.31e-05
9.83e-04 ± 7.18e-07
Table 1: Convection equation ( β=50 ). Relative L2 error (mean ± std) at different N for different methods.
N
8
16
32
64
96
128
β=10
Galerkin(quadratic)
1.55e-02
3.52e-03
1.03e-03
4.67e-04
6.91e-04
3.91e-01
DABG
1.36e-02
1.25e-04
8.73e-04
1.04e-05
3.33e+00
1.32e+00
β=30
Galerkin(quadratic)
3.37e-01
4.98e-02
8.03e-03
1.90e-03
9.03e-04
5.04e-04
DABG
1.12e+00
2.87e-01
2.93e-04
4.34e-04
5.75e-05
1.06e-04
β=50
Galerkin(quadratic)
4.67e+00
2.15e-01
3.83e-02
5.17e-03
2.08e-03
1.32e-03
DABG
1.05e+00
1.03e+00
3.30e-03
1.44e-03
6.46e-04
6.39e-04
Table 2: Convection equation. Different relative L2 errors between the proposed method and DABG.
β
10
30
50
Error
Galerkin(quadratic)
2.95e-04
1.32e-03
3.57e-03
Causal PINNs
7.58e-04
1.12e-02
1.65e+00
Running time
Galerkin(quadratic)
57
62
63
Causal PINNs
6639
8087
8057
Table 3: Convection equation. Relative L2 error and running time between the proposed method and causal PINNs.
Nsegment
1
2
4
Error
–
1.17e-02 ± 4.36e-03
7.67e-03 ± 1.27e-03
Table 4: Allen-Cahn equation. Relative L2 error (mean ± std) for different segments. (Linear finite element basis function with N=100 )
Nsegment
1
2
4
Error
2.25e-01 ± 8.51e-03
7.57e-03 ± 1.68e-03
5.08e-03 ± 1.87e-03
Table 5: Allen-Cahn equation. Relative L2 error (mean ± std) for different segments. (Quadratic finite element basis function with N=30 )
Figure 5: Allen Cahn equation. From left to right: the prediction and absolute error. Relative L2 error is 5.08e-3 (Quadratic basis function, Nsegment =4, N =30).
N
10
20
30
Galerkin (quadratic)
4.07e-03 ± 3.65e-04
7.87e-03 ± 1.43e-03
7.57e-03 ± 1.68e-03
Collocation (Hermite polynomial)
4.12e-03 ± 2.31e-04
5.51e-03 ± 3.64e-04
7.65e-03 ± 1.04e-03
Collocation (spline)
3.81e-03 ± 4.79e-04
3.89e-03 ± 5.39e-04
3.56e-03 ± 6.73e-04
Table 6: Allen-Cahn equation. Relative L2 error (mean ± std) for different approaches ( Nsegment=2 )
Galerkin(quadratic)
Causal PINNs
Running time
71
2524
Error ( Nsegment =1)
1.95e-02
1.33e-02
Error ( Nsegment =2)
3.58e-03
3.73e-03
Table 7: Allen-Cahn equation. Relative L2 error and running time for the proposed method and causal PINNs.
Figure 6: Heat equation on complex geometry. Exact solutions for different time. From left to right: t=0.2,0.6 and 1.0 .
N
10
20
40
80
Galerkin (linear)
1.27e+00 ± 1.27e-03
1.44e-01 ± 5.39e-04
3.53e-02 ± 9.29e-04
2.04e-02 ± 9.68e-04
Galerkin (quadratic)
9.82e-02 ± 1.26e-04
2.96e-02 ± 9.75e-04
2.01e-02 ± 8.27e-04
2.50e-02 ± 9.78e-04
Collocation (Hermite polynomial)
2.52e-02 ± 1.96e-03
9.53e-03 ± 6.13e-03
9.23e-03 ± 4.87e-03
1.44e-02 ± 9.68e-03
Collocation (spline)
5.94e-01 ± 4.71e-03
1.19e-02 ± 4.53e-03
6.64e-03 ± 2.50e-03
9.54e-03 ± 2.14e-03
Table 8: Heat equation on complex geometry. Relative L2 errors (mean ± std) at different N.
N
10
20
40
80
Galerkin (linear)
1.04e+00 ± 3.11e-05
2.48e-01 ± 7.97e-05
5.75e-02 ± 1.91e-04
1.48e-02 ± 2.51e-04
Galerkin (quadratic)
4.45e-02 ± 1.25e-04
7.61e-03 ± 2.73e-05
1.74e-03 ± 2.07e-05
9.67e-04 ± 2.03e-05
Collocation (Hermite polynomial)
2.28e-02 ± 3.28e-06
1.41e-03 ± 1.87e-05
6.58e-04 ± 3.48e-05
1.04e-03 ± 8.73e-05
Collocation (spline)
4.89e-02 ± 6.66e-05
1.42e-02 ± 3.44e-05
6.42e-04 ± 3.51e-05
8.89e-04 ± 1.07e-04
Table 9: High-dimensional linear problem. Relative L2 error (mean ± std) for different N .
N
10
20
40
80
Galerkin (quadratic)
4.48e-02 ± 5.34e-05
7.92e-03 ± 7.63e-05
1.89e-03 ± 3.21e-04
1.16e-03 ± 3.57e-04
Collocation (Hermite polynomial)
2.48e-02 ± 6.38e-05
2.21e-03 ± 5.92e-04
1.72e-03 ± 4.46e-04
1.33e-03 ± 5.31e-04
Collocation (spline)
5.04e-02 ± 6.21e-06
1.61e-02 ± 9.75e-05
2.42e-03 ± 3.18e-04
2.31e-03 ± 2.36e-04
Table 10: High-dimensional nonlinear problem. Relative L2 error (mean ± std) for different N .
Figure 7: Low regularity test problem. Exact solutions for different time ( β =1000). From left to right: t=0,0.25 and 0.5 .
Figure 8: Low regularity test problem. Relative L2 error of different adaptivity iterations. ( β=200 , collocation projection)
Figure 9: Low regularity test problem. Temporal density of the 1st, 3rd, 5th adaptivity iteration. ( β=200 , collocation projection)
Figure 11: Low regularity test problem. Relative L2 error for different sampling strategies. ( β =1000, Galerkin projection).
Figure 12: Low regularity test problem. Temporal density of the 1st, 3rd, 5th adaptivity iteration. ( β=1000 , Galerkin projection)
Figure 13: Low regularity test problem. From top to bottom: The generated samples of the 1st and 5th adaptivity iteration. ( β=1000, Galerkin projection)
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 14: Low regularity test problem. Relative L2 error of different adaptivity iterations. ( β=200 , Galerkin projection)
Figure 15: Low regularity test problem. Temporal weights λi of the 1st, 3rd, 5th adaptivity iteration. ( β=200 , Galerkin projection)
Figure 16: Low regularity test problem. From top to bottom: The generated samples of the first, and the last adaptivity iteration. ( β=200 , Galerkin projection)
Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains. In this work, a deep adaptive sampling framework for PINNs is extended to the time-dependent setting by treating space and time as a unified domain without any explicit time marching. A normalising flow neural network model effectively learns the distribution induced by the PDE residual and generates new collocation points concentrated in regions where the solution is most difficult to learn. Unlike conventional adaptive strategies that require explicit time stepping or moving meshes, high-residual regions are automatically identified and tracked across both space and time, driven purely by the PDE residual distribution. The effectiveness of the proposed strategy is assessed on a range of benchmark problems, from sharp and moving features in two spatial dimensions to localised structures in up to eight spatial dimensions.
Anshima Singh, David J. Silvester
Department of Mathematics, University of Manchester, Oxford Road, Manchester M13 9PL, UK
Despite the growing popularity of physics-informed neural networks (PINNs), their applicability in the long-time integration of partial differential equations (PDEs) remains constrained. We argue that this problem stems from the lack of consideration of temporal causality in the original PINN formulation, resulting in a bias towards satisfying governing equations at later times before learning the initial condition and hence leading to erroneous solutions. To this end, we propose a novel method that seamlessly integrates temporal causality into the training process. Drawing inspiration from classical numerical methods where the temporal causality is reflected, we divide the time domain into nonoverlapping subintervals, assign a unique neural network to each subinterval, and construct a loss function founded on the integral form of PDEs within these subintervals. The proposed networks undergo sequential training, beginning with the initial time step. Our method demonstrates significant improvement in accuracy for long-time simulations of various PDE problems where the original PINN method fails while it requires less computational cost and memory compared to the PINN method. A parallelization algorithm is provided to further enhance the computational efficiency, showing a significant speedup for solving time-dependent PDEs.
Jeahan Jung, Heechang Kim, Hyomin Shin +1
Department of Mathematics, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
Physics-informed neural networks and finite element methods provide two different paradigms for the numerical approximation of partial differential equations: the former are commonly trained by minimizing pointwise strong residuals, whereas the latter are naturally built from weak variational formulations and the finite-dimensional systems obtained after discretization. In this work, we introduce a common framework based on the discretization of functional Gauss--Newton problems by finite families of linear measurements. We show that, through an appropriate duality pairing, the linear measurements can be represented by test functions. The resulting Gauss--Newton system is then precisely a Petrov--Galerkin discretization of the linearized functional problem. This perspective recovers pointwise collocation and natural-gradient constructions as particular cases, while making the choice of test functions an explicit algorithmic design choice. We specialize this framework to elliptic problems, where it naturally leads to weak residual formulations and to a hybrid finite element--neural construction acting on complementary approximation spaces. Numerical experiments support the proposed framework and demonstrate the effectiveness of weak Gauss--Newton formulations and hybrid finite element--neural approximations.