A hybrid FEM-PINN method for time-dependent partial differential equations
Authors: Xiaodong Feng, Haojiong Shangguan, Tao Tang, Xiaoliang Wan
Organizations: Division of Science and Technology, BNU-HKBU United International College, Zhuhai 519087, China · School of Electrical and Computer Engineering, Guangzhou Nanfang College, Guangzhou 510970, China · Department of Mathematics and Center for Computation and Technology, Louisiana State University, Baton Rouge 70803, USA
In this work, we present a hybrid numerical method for solving evolution partial differential equations (PDEs) by merging the time finite element method with deep neural networks. In contrast to the conventional deep learning-based formulation where the neural network is defined on a spatiotemporal domain, our methodology utilizes finite element basis functions in the time direction where the space-dependent coefficients are defined as the output of a neural network. We then apply the Galerkin or collocation projection in the time direction to obtain a system of PDEs for the space-dependent coefficients which is approximated in the framework of PINN. The advantages of such a hybrid formulation are twofold: statistical errors are avoided for the integral in the time direction, and the neural network's output can be regarded as a set of reduced spatial basis functions. To further alleviate the difficulties from high dimensionality and low regularity, we have developed an adaptive sampling strategy that refines the training set. More specifically, we use an explicit density model to approximate the distribution induced by the PDE residual and then augment the training set with new time-dependent random samples given by the learned density model. The effectiveness and efficiency of our proposed method have been demonstrated through a series of numerical experiments.
Figures & tables
Figure 1: Schematic of the proposed approach.
Figure 2: Convection equation. Exact solutions. From left to right: β=10,30 and 50 .
Figure 3: Convection equation. Numerical results with linear finite element discretization.
Figure 4: Convection equation. Convergence rate for linear finite element basis functions versus quadratic finite element basis functions.
N
32
64
96
128
Galerkin (linear)
1.69e+00 ± 2.38e-05
7.73e-02 ± 1.23e-05
1.91e-02 ± 1.06e-05
8.69e-03 ± 1.18e-05
Galerkin (quadratic)
3.83e-02 ± 8.53e-06
5.58e-03 ± 1.15e-05
2.26e-03 ± 2.67e-05
1.13e-03 ± 3.74e-05
Collocation (Hermite polynomial)
1.97e-01 ± 1.78e-06
1.18e-02 ± 6.76e-07
2.34e-03 ± 2.17e-04
7.50e-04 ± 2.92e-04
Collocation (spline)
6.44e-01 ± 2.29e-05
1.83e-02 ± 1.13e-06
3.16e-03 ± 2.31e-05
9.83e-04 ± 7.18e-07
Table 1: Convection equation ( β=50 ). Relative L2 error (mean ± std) at different N for different methods.
N
8
16
32
64
96
128
β=10
Galerkin(quadratic)
1.55e-02
3.52e-03
1.03e-03
4.67e-04
6.91e-04
3.91e-01
DABG
1.36e-02
1.25e-04
8.73e-04
1.04e-05
3.33e+00
1.32e+00
β=30
Galerkin(quadratic)
3.37e-01
4.98e-02
8.03e-03
1.90e-03
9.03e-04
5.04e-04
DABG
1.12e+00
2.87e-01
2.93e-04
4.34e-04
5.75e-05
1.06e-04
β=50
Galerkin(quadratic)
4.67e+00
2.15e-01
3.83e-02
5.17e-03
2.08e-03
1.32e-03
DABG
1.05e+00
1.03e+00
3.30e-03
1.44e-03
6.46e-04
6.39e-04
Table 2: Convection equation. Different relative L2 errors between the proposed method and DABG.
β
10
30
50
Error
Galerkin(quadratic)
2.95e-04
1.32e-03
3.57e-03
Causal PINNs
7.58e-04
1.12e-02
1.65e+00
Running time
Galerkin(quadratic)
57
62
63
Causal PINNs
6639
8087
8057
Table 3: Convection equation. Relative L2 error and running time between the proposed method and causal PINNs.
Nsegment
1
2
4
Error
–
1.17e-02 ± 4.36e-03
7.67e-03 ± 1.27e-03
Table 4: Allen-Cahn equation. Relative L2 error (mean ± std) for different segments. (Linear finite element basis function with N=100 )
Nsegment
1
2
4
Error
2.25e-01 ± 8.51e-03
7.57e-03 ± 1.68e-03
5.08e-03 ± 1.87e-03
Table 5: Allen-Cahn equation. Relative L2 error (mean ± std) for different segments. (Quadratic finite element basis function with N=30 )
Figure 5: Allen Cahn equation. From left to right: the prediction and absolute error. Relative L2 error is 5.08e-3 (Quadratic basis function, Nsegment =4, N =30).
N
10
20
30
Galerkin (quadratic)
4.07e-03 ± 3.65e-04
7.87e-03 ± 1.43e-03
7.57e-03 ± 1.68e-03
Collocation (Hermite polynomial)
4.12e-03 ± 2.31e-04
5.51e-03 ± 3.64e-04
7.65e-03 ± 1.04e-03
Collocation (spline)
3.81e-03 ± 4.79e-04
3.89e-03 ± 5.39e-04
3.56e-03 ± 6.73e-04
Table 6: Allen-Cahn equation. Relative L2 error (mean ± std) for different approaches ( Nsegment=2 )
Galerkin(quadratic)
Causal PINNs
Running time
71
2524
Error ( Nsegment =1)
1.95e-02
1.33e-02
Error ( Nsegment =2)
3.58e-03
3.73e-03
Table 7: Allen-Cahn equation. Relative L2 error and running time for the proposed method and causal PINNs.
Figure 6: Heat equation on complex geometry. Exact solutions for different time. From left to right: t=0.2,0.6 and 1.0 .
N
10
20
40
80
Galerkin (linear)
1.27e+00 ± 1.27e-03
1.44e-01 ± 5.39e-04
3.53e-02 ± 9.29e-04
2.04e-02 ± 9.68e-04
Galerkin (quadratic)
9.82e-02 ± 1.26e-04
2.96e-02 ± 9.75e-04
2.01e-02 ± 8.27e-04
2.50e-02 ± 9.78e-04
Collocation (Hermite polynomial)
2.52e-02 ± 1.96e-03
9.53e-03 ± 6.13e-03
9.23e-03 ± 4.87e-03
1.44e-02 ± 9.68e-03
Collocation (spline)
5.94e-01 ± 4.71e-03
1.19e-02 ± 4.53e-03
6.64e-03 ± 2.50e-03
9.54e-03 ± 2.14e-03
Table 8: Heat equation on complex geometry. Relative L2 errors (mean ± std) at different N.
N
10
20
40
80
Galerkin (linear)
1.04e+00 ± 3.11e-05
2.48e-01 ± 7.97e-05
5.75e-02 ± 1.91e-04
1.48e-02 ± 2.51e-04
Galerkin (quadratic)
4.45e-02 ± 1.25e-04
7.61e-03 ± 2.73e-05
1.74e-03 ± 2.07e-05
9.67e-04 ± 2.03e-05
Collocation (Hermite polynomial)
2.28e-02 ± 3.28e-06
1.41e-03 ± 1.87e-05
6.58e-04 ± 3.48e-05
1.04e-03 ± 8.73e-05
Collocation (spline)
4.89e-02 ± 6.66e-05
1.42e-02 ± 3.44e-05
6.42e-04 ± 3.51e-05
8.89e-04 ± 1.07e-04
Table 9: High-dimensional linear problem. Relative L2 error (mean ± std) for different N .
N
10
20
40
80
Galerkin (quadratic)
4.48e-02 ± 5.34e-05
7.92e-03 ± 7.63e-05
1.89e-03 ± 3.21e-04
1.16e-03 ± 3.57e-04
Collocation (Hermite polynomial)
2.48e-02 ± 6.38e-05
2.21e-03 ± 5.92e-04
1.72e-03 ± 4.46e-04
1.33e-03 ± 5.31e-04
Collocation (spline)
5.04e-02 ± 6.21e-06
1.61e-02 ± 9.75e-05
2.42e-03 ± 3.18e-04
2.31e-03 ± 2.36e-04
Table 10: High-dimensional nonlinear problem. Relative L2 error (mean ± std) for different N .
Figure 7: Low regularity test problem. Exact solutions for different time ( β =1000). From left to right: t=0,0.25 and 0.5 .
Figure 8: Low regularity test problem. Relative L2 error of different adaptivity iterations. ( β=200 , collocation projection)
Figure 9: Low regularity test problem. Temporal density of the 1st, 3rd, 5th adaptivity iteration. ( β=200 , collocation projection)
Figure 11: Low regularity test problem. Relative L2 error for different sampling strategies. ( β =1000, Galerkin projection).
Figure 12: Low regularity test problem. Temporal density of the 1st, 3rd, 5th adaptivity iteration. ( β=1000 , Galerkin projection)
Figure 13: Low regularity test problem. From top to bottom: The generated samples of the 1st and 5th adaptivity iteration. ( β=1000, Galerkin projection)
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 14: Low regularity test problem. Relative L2 error of different adaptivity iterations. ( β=200 , Galerkin projection)
Figure 15: Low regularity test problem. Temporal weights λi of the 1st, 3rd, 5th adaptivity iteration. ( β=200 , Galerkin projection)
Figure 16: Low regularity test problem. From top to bottom: The generated samples of the first, and the last adaptivity iteration. ( β=200 , Galerkin projection)