TENG-BC: Unified Time-Evolving Natural Gradient for Neural PDE Solvers with General Boundary Conditions

Hongjie Jiang; Di Luo

cs.LGFeb 28, 2026

TENG-BC: Unified Time-Evolving Natural Gradient for Neural PDE Solvers with General Boundary Conditions

Authors: Hongjie Jiang, Di Luo

Organizations: School of Mathematical Sciences, Peking University, Beijing, China · Department of Physics, Tsinghua University, Beijing, China · Institute of Advanced Study, Tsinghua University, Beijing, China

Abstract

Accurately solving time-dependent partial differential equations (PDEs) with neural networks remains challenging due to long-time error accumulation and the difficulty of enforcing general boundary conditions. We introduce TENG-BC, a high-precision neural PDE solver based on the Time-Evolving Natural Gradient, designed to perform under general boundary constraints. At each time step, TENG-BC performs a boundary-aware optimization that jointly enforces interior dynamics and boundary conditions, accommodating Dirichlet, Neumann, Robin, and mixed types within a unified framework. This formulation admits a natural-gradient interpretation, enabling stable time evolution without delicate penalty tuning. Across benchmarks over diffusion, transport, and nonlinear PDEs with various boundary conditions, TENG-BC achieves solver-level accuracy under comparable sampling budgets, outperforming conventional solvers and physics-informed neural network (PINN) baselines.

Figures & tables

Figure 1: Overview of the boundary-aware TENG-BC update. The neural representation is advanced by solving a unified least-squares problem that enforces both interior dynamics and boundary constraints. Different boundary types are incorporated through the same optimization operator, which is repeatedly applied to realize temporal evolution.

\Delta\theta\leftarrow\arg\min_{\Delta\theta}\|\mathcal{R}-\mathcal{J}\Delta\theta\|_{L^{2}(\overline{\Omega})}^{2}

Algorithm 1 Boundary-Aware TENG Update at One Time Step

Panel of Figure 2 — Figure 2: Performance of TENG-BC on heat equation with different boundary conditions. (a) TENG-Heun predictions at $T=1.0$ are shown with analytic references and pointwise error maps. (b) Per-time-step relative $L^{2}$ -error comparison among TENG-BC variants and other reference solvers.

$L^{2}$ -error $\downarrow$	Dirichlet	Neumann	Nonzero Neumann	Robin	Mixed
TENG-Euler	2.17e-04	2.05e-04	9.04e-04	6.40e-04	1.25e-04
TENG-Heun	3.08e-06	3.60e-06	4.41e-06	2.42e-06	1.15e-05
TENG-RK4	3.07e-06	1.14e-05	4.72e-06	5.00e-06	3.31e-06

Table 1: Full-time-range relative

L^{2}

-error of TENG-BC with three propagation schemes on heat equation with different boundary conditions.

Figure 3: Performance of TENG-BC on the two-dimensional transport equation with nonzero dirichlet boundary. Comparison of per-time-step relative

L^{2}

-error among TENG-BC variants, PINN, and FEM baselines.

Panel of Figure 4 — Figure 4: Performance of TENG-BC on the Burgers equation with periodic boundary. (a) Predicted solutions by TENG-Heun at representative time steps. (b) Discrepancies between TENG-Heun and FEM solutions across multiple spatial resolutions (c) the scaling trend of discrepancies. between TENG-Heun and FEM solutions at $T=4$ . (d) Pairwise $L^{2}$ discrepancies among TENG-Heun, FEM-256, and spectral-1024 solvers.

$L^{2}$ -error $\downarrow$	$T=1$	$T=2$	$T=3$	$T=4$
TENG-Euler	2.97e-04	1.72e-03	1.28e-03	1.60e-03
TENG-Heun	2.24e-06	2.52e-05	1.15e-04	2.24e-04
TENG-RK4	2.63e-06	2.94e-05	1.42e-04	3.21e-04
FEM	3.21e-06	1.11e-03	4.83e-03	4.50e-03
PINN-BFGS	1.28e-01	1.99e-01	2.08e-01	1.71e-01

Table 2: Relative

L^{2}

errors of different solvers for the Burgers equation with periodic boundary conditions, measured against the spectral-1024 reference.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Comparison between the exact solution and the TENG-BC prediction for the heat equation with dirichlet boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 6: Comparison between the exact solution and the TENG-BC prediction for the heat equation with zero neumann boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 7: Comparison between the exact solution and the TENG-BC prediction for the heat equation with nonzero neumann boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 8: Comparison between the exact solution and the TENG-BC prediction for the heat equation with robin boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 9: Comparison between the exact solution and the TENG-BC prediction for the heat equation with mixed boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 10: Comparison between the exact solution and the TENG-BC prediction for the transport equation with dirichlet boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 11: Comparison between the exact solution and the TENG-BC prediction for the Burgers equation with periodic boundary at representative time instances. Top: exact solution. Middle: solution obtained by TENG-BC. Bottom: absolute error.

Figure 12: Per-step loss of TENG-BC on the heat equation with different boundaries. Here training loss means the per-step least-squares residuals showing local fitting accuracy within each update.

Figure 13: Per-step loss of TENG-BC on the transport equation with dirichlet boundary. Here training loss means the per-step least-squares residuals showing local fitting accuracy within each update.

Figure 14: Per-step loss of TENG-BC on the Burgers equation with periodic boundary. Here training loss means the per-step least-squares residuals showing local fitting accuracy within each update.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 269 papers

A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations
May 28, 2026Qihong Yang, Yangtao Deng, Qiaolin He +1Neural PDE Solvers PDE Solving
School of Mathematics, Sichuan University, Chengdu, 610065, China
BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations
Feb 16, 2026Jonathan Gorard, Ammar Hakim, James JunoNeural PDE Solvers Partial Differential Equations
Princeton University Princeton, NJ 08544, USA · Princeton Plasma Physics Laboratory Princeton, NJ 08540, USA
Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers
Jul 24, 2026Zhangyong Liang, Huanhuan GaoNeural PDE Solvers Riemannian Optimization
National Center for Applied Mathematics Tianjin University Tianjin, 300072, China · School of Mechanical and Aerospace Engineering Jilin University Changchun, 130025, China

May 28, 2026J/K move · Enter open · S save

A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations

Qihong Yang, Yangtao Deng, Qiaolin He, Shiquan Zhang

School of Mathematics, Sichuan University, Chengdu, 610065, China

Neural PDE Solvers PDE Solving

Open paper AskPDF arXiv

This paper presents the Tensor Product Network (TPNet), a novel neural architecture for efficient and accurate function approximation and PDE solving. The core of the proposal involves constructing the solution explicitly as a linear combination of basis functions integrated into the network, with coefficients determined by a direct least-squares solve, thereby bypassing traditional gradient-based training. The key methodological contribution include: (1) an efficient tensor-product scheme that generates multi-dimensional basis functions from combinations of two sets of subnetwork outputs, significantly reducing model complexity and parameter count while maintaining expressivity; (2) a block time-marching strategy to improve computational efficiency in long-time simulations; and (3) a linear reformulation strategy for handling nonlinear PDEs by treating known nonlinear terms as sources. TPNet achieves superior accuracy and shorter training times than conventional neural network solvers. This performance gain stems from its structured design and deterministic least-squares fitting, which contrast with the iterative, often computationally intensive optimization required by mainstream methods like Physics-Informed Neural Networks (PINNs).

Abstract

Neural PDE Solvers

Explore similar work

A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations

BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations

Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solvers