FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations

Shun Zhang

math.NAOct 6, 2026

FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations

Authors: Shun Zhang

Organizations: Department of Mathematics, City University of Hong Kong, Kowloon Tong, Hong Kong, China

Abstract

We construct neural approximation classes native to the graph spaces H(div) and H(curl), in two and three dimensions and, for H(div), in any dimension. Every realization lies in the space for all parameter values, and with kinked potentials, such as ReLU networks, the admissible jumps appear at finite width. The classes are images of scalar and componentwise networks under fixed operators of the de Rham complex, and do not involve a mesh or finite element emulation. For H(div) in R^n two native classes are given on an equal footing, with a skew-symmetric potential $A$ : $\mathrm{Div}\,A+R_nq+\mathbf{h}$ , with the divergence $q$ as an explicit unknown, and $\mathrm{Div}\,A+\mathbf{z}$ with an $H^1$ field $\mathbf{z}$ ; for H(curl) the analogous classes are $\mathrm{grad}\,φ+Sr+\mathbf{h}$ in two dimensions and $\mathrm{grad}\,φ+\mathbf{z}$ in two and three dimensions. In all of them every interface jump of the field is carried by the potential term, $\mathrm{Div}\,A$ or $\mathrm{grad}\,φ$ , while the remaining part has no interface jump (it is an $H^1$ field in the regular-decomposition classes); the classes with $\mathbf{z}$ are the componentwise approach enriched by this term. Known or learned interface geometry enters the potential through factors with trainable amplitudes, and the remaining part if the divergence jumps. The classes lead to the FOSLS-deRhaNN method, first-order system least squares with de Rham neural networks, whose loss is the least-squares functional posed in the natural spaces of the weak formulation; for elliptic equations this includes $H^{-1}$ right-hand sides and $H^{1/2}$ Dirichlet data. Elliptic equations with discontinuous coefficients and curl-curl problems are treated as instances, with the functional equivalent to the error; linear transport with discontinuous solutions and conservation laws with shocks use the same flux classes.

Figures & tables

Figure 1. Example 1, generic level, first class: (a) $\log_{10}|\mbox{\boldmath$\sigma$}_{\theta}-\mbox{\boldmath$\sigma$}|$ ; (b) the kink sets of the three ReLU layers and $\Gamma$ ; (c) the tangential jump of $\sigma$ along $\Gamma$ against the polar angle about $C$ : exact, and of $\mbox{\boldmath$\sigma$}_{\theta}$ across $\pm 0.005$ . — Figure 1. Example 1, generic level, first class: (a) $\log_{10}|\mbox{\boldmath$ \sigma $}_{\theta}-\mbox{\boldmath$ \sigma $}|$ ; (b) the kink sets of the three ReLU layers and $\Gamma$ ; (c) the tangential jump of $\sigma$ along $\Gamma$ against the polar angle about $C$ : exact, and of $\mbox{\boldmath$ \sigma $}_{\theta}$ across $\pm 0.005$ .

Figure 2. Example 2, Level II. (a) Section $x_{3}=\cdots=x_{n}=0$ of the unit ball: $\Gamma=\{x_{1}=s\}$ (gray), the initial zero set of $g_{\eta}$ (red) and the learned zero sets for $n=4$ (blue) and $n=8$ (green). (b), (c) Position of the learned zero set on the section $x_{4}=\cdots=x_{n}=0$ : $x_{1}^{*}-s$ with $g_{\eta}(x_{1}^{*},x_{2},x_{3},0,\ldots,0)=0$ , for $n=4$ (scale $10^{-3}$ ) and $n=8$ (scale $10^{-4}$ ).

Figure 3. Section 8.1 , Level II. (a) The interface $x_{1}=\frac{1}{2}$ (dashed), the initial zero set of $g_{\eta}$ (red) and the learned zero set for the first initial network (blue). (b)–(d) Position of the learned zero set for the three initial networks: $x_{1}^{*}-\frac{1}{2}$ with $g_{\eta}(x_{1}^{*},x_{2},x_{3})=0$ , over $(x_{2},x_{3})\in(0,1)^{2}$ .

Figure 4. Linear transport with a learned interface (Level II), first class, after the main run: (a) the learned curve $\{g_{\eta}=0\}$ (solid), $\Gamma$ (dashed) and the initial line $x_{2}=0.5$ (dotted); (b) $v_{\theta}$ ; (c) $\log_{10}|u-v_{\theta}|$ , with $\Gamma$ ; (d) the jump of $v_{\theta}$ across $\Gamma$ (circles) and that of $u$ (dashed), against $x_{1}$ .

Figure 5. Linear transport, generic level, first class: (a) the zero sets of the last layer, colored by the mean step of $v_{\theta}$ along them, $\Gamma$ dashed; (b) $v_{\theta}$ ; (c) $\log_{10}|u-v_{\theta}|$ , with $\Gamma$ ; (d) $v_{\theta}$ and $u$ on $x_{1}=0.5$ .

Figure 6. Linear transport, generic level, second class ( $\mbox{\boldmath$\sigma$}_{\theta}=\operatorname{\mathbf{curl}}\psi_{\theta}+{\bf z}_{\theta}$ ): (a) the zero sets of the last layer, colored by the mean step of $v_{\theta}$ along them, $\Gamma$ dashed; (b) $v_{\theta}$ ; (c) $\log_{10}|u-v_{\theta}|$ , with $\Gamma$ ; (d) $v_{\theta}$ and $u$ on $x_{1}=0.5$ . — Figure 6. Linear transport, generic level, second class ( $\mbox{\boldmath$ \sigma $}_{\theta}=\operatorname{\mathbf{curl}}\psi_{\theta}+{\bf z}_{\theta}$ ): (a) the zero sets of the last layer, colored by the mean step of $v_{\theta}$ along them, $\Gamma$ dashed; (b) $v_{\theta}$ ; (c) $\log_{10}|u-v_{\theta}|$ , with $\Gamma$ ; (d) $v_{\theta}$ and $u$ on $x_{1}=0.5$ .

Figure 7. Burgers’ equation, Riemann data, generic level. (a) $v_{\theta}$ on $Q$ , the exact shock $x=t/2$ white. (b) The kinks of $\psi_{\theta}$ across which $\mbox{\boldmath$\tau$}_{\Theta}$ has a tangential jump of at least $2\%$ of the exact one, colored by the jump; the exact shock dashed. (c) $v_{\theta}$ (solid) and $(\mbox{\boldmath$\tau$}_{\Theta})_{t}$ (dashed) at $t=0.25$ , $0.5$ , $1$ ; the exact solution dotted. — Figure 7. Burgers’ equation, Riemann data, generic level. (a) $v_{\theta}$ on $Q$ , the exact shock $x=t/2$ white. (b) The kinks of $\psi_{\theta}$ across which $\mbox{\boldmath$ \tau $}_{\Theta}$ has a tangential jump of at least $2\%$ of the exact one, colored by the jump; the exact shock dashed. (c) $v_{\theta}$ (solid) and $(\mbox{\boldmath$ \tau $}_{\Theta})_{t}$ (dashed) at $t=0.25$ , $0.5$ , $1$ ; the exact solution dotted.

Explore similar work

Aug 7, 2026math.NA

Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations

\mathfrak L_βu=f

using linearized ReLU

^k

neural networks on the sphere, where

\mathfrak L_β

is a positive elliptic spectral multiplier of order

β

. Given a parameter set

Θ_n=\{θ_{j}^*\}_{j=1}^n\subset\mathbb S^d

, we approximate

u

in the linearized network space

L_n^k(Θ_n)

by the discrete residual on the collocation points

\{η_i^*\}_{i=1}^m

\begin{equation*} u_{n,m}\in\arg\min_{v_n\in L_n^k(Θ_n)}\frac1m\sum_{i=1}^m\left(f(η_i^)-\mathfrak L_βv_n(η_i^)\right)^2. \end{equation*} With

k>\frac{d-1}{2}+β

, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with

m\gtrsim n

, we prove that \begin{equation*} |u-u_{n,m}|{\mathcal H^β(\mathbb S^d)}\eqsim|f-\mathfrak L_βu{n,m}|{\mathcal L^2(\mathbb S^d)}\lesssim n^{-\frac{r}{d}} \begin{cases} |f|{\mathcal W^{r,p}(\mathbb S^d)},&\frac{d}{p}<r\leq \frac{d}{2},~p>2,\ |f|{\mathcal H^r(\mathbb S^d)},&r>\frac{d}{2}. \end{cases} \end{equation*} We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d.\ uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLU $^k$ network spaces. If $\underline h$ denotes the antipodal separation distance of the network parameters, then \begin{equation*} |v_n|{\mathcal H^r(\mathbb S^d)}\lesssim\underline h^{-(r-s)}|v_n|_{\mathcal H^s(\mathbb S^d)},\qquad 0\leq s<r<k+\tfrac12. \end{equation*}

Xinliang Liu, Tong Mao, Jinchao Xu

School of Mathematical Sciences, Ocean University of China, Qingdao 266100, China · Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China

Neural Approximations Spectral Representation Method Bayesian Quadrature Least Squares Rectified Linear Unit Networks Empirical Study

Ask PDF

Sep 23, 2026math.NA

A Hybrid Iterative Deep Ritz Method for Elliptic Interface Problems

In this work, we propose a hybrid iterative deep Ritz method (H-IDRM) for a class of interface problems for second-order elliptic operators. It is based on a new mixed formulation of the problem and involves solving a sequence of convex minimization problems. We employ a level-set neural network architecture, featuring a level-set representation of the interface, to accommodate the piecewise smoothness of the solution and the flux. The approach involves only volumetric representations instead of duality pairing on the interface and avoids explicit interface sampling that is inconvenient for complex interface geometries. Further, we present an analysis of the method, including the errors arising from the neural network approximation, Monte Carlo approximation, iterative scheme, and penalty parameters. Numerical experiments indicate that the H-IDRM outperforms existing neural solvers on problems with high-dimensional domains, intricate interface geometries, and lower subdomain regularity.

Tianhao Hu, Bangti Jin, Fengru Wang +1

Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong · Department of Mathematics & Scientific Computing Key Laboratory of Shanghai Universities, Shanghai Normal University, Shanghai 200234, China.

Neural Solvers Variational Formulation Fixed-Point Iteration Ratio

Ask PDF

Apr 16, 2026cs.LG

Python library supporting Discrete Variational Formulations and training solutions with Collocation-based Robust Variational Physics Informed Neural Networks (DVF-CRVPINN)

We explore the possibility of solving Partial Differential Equations (PDEs) using discrete weak formulations. We propose a programming environment for defining a discrete computational domain, introducing discrete functions defined over a set of points, constructing discrete inner products, and introducing discrete weak formulations employing Kronecker delta test functions. Building on this setup, we propose a discrete neural network representation, training the solution function defined over a discrete set of points and employing discrete finite difference derivatives in the automatic differentiation procedures. As a challenging computational model example, we focus on Stokes equations in two-dimensions, defined over a discrete set of points. We train the solution using the discrete weak residual and the Adamax algorithm with discrete automatic differentiation of the discrete gradients. Despite introducing the python environment, we also provide a rigorous mathematical formulation based on discrete weak formulations, proving the well-posedness and robustness of the loss function. The solution of the discrete weak formulations is based on neural network training employing a robust loss function that is related to the true error. In this way, we have a robust control of the numerical error during the training of the neural networks. Besides the Stokes formulation, we also explain the functionality of the proposed library using the Laplace problem formulation.

Tomasz Służalec, Marcin Łoś, Askold Vilkha +1

AGH University of Krakow, Poland

Variational Formulation Neural Network Neural Network Representations Implicit Differentiation Python

Ask PDF