Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs

Zhangyong Liang; Huanhuan Gao

cs.LGMar 31, 2026

Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNs

Authors: Zhangyong Liang, Huanhuan Gao

Organizations: National Center for Applied Mathematics, Tianjin University, Tianjin, 300072, China · School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, China

Abstract

Enforcing prescribed global integral constraints in mesh-free neural PDE solvers is challenging in high-dimensional domains. Existing projection methods for spatial integrals are often tied to fixed grids or uniform quadrature, which can conflict with randomly sampled physics-informed neural networks (PINNs) and scale poorly with dimension. High-order differential operators also increase reverse-mode automatic differentiation memory costs. We propose Stochastic Dimension Implicit Functional Projection (SDIFP), a quadrature-level framework for enforcing prescribed first and second spatial moments. SDIFP replaces tensor-product nodal projection by a global affine correction of the neural-network output, with two scalar coefficients determined from a weighted quadrature rule. Under positive target variance and nonzero empirical raw variance, this correction is the nearest-point projection, in the weighted quadrature norm, onto the empirical two-moment constraint set. Thus, the prescribed moments are exact for the selected quadrature rule, while continuum errors are quadrature errors of the corrected field. For decomposable high-dimensional linear operators, SDIFP combines affine moment correction with stochastic operator-subset sampling. With independent residual and derivative sampling and conditionally unbiased coefficient-gradient estimation, the resulting estimator is unbiased for the specified quadrature-based residual objective; the shared-subset fast mode is biased in general. SDIFP avoids tensor-product quadrature for moment enforcement, separates forward quadrature evaluation from the reverse-mode graph, and retains pointwise inference efficiency once the affine coefficients are fixed or precomputed.

Explore similar work

Mar 25, 2026cs.LG

Stochastic Dimension Zeroth-Order Estimator: Stable and Memory-Efficient Training of PINNs

Physics-Informed Neural Networks (PINNs) for high-dimensional and high-order partial differential equations (PDEs) are primarily constrained by the

\mathcal{O}(d^k)

spatial derivative complexity and the

\mathcal{O}(P)

memory overhead of backpropagation (BP). While randomized spatial estimators successfully reduce the spatial complexity to

\mathcal{O}(1)

, their reliance on first-order optimization still leads to prohibitive memory consumption at scale. Zeroth-order (ZO) optimization offers a BP-free alternative; however, naively combining randomized spatial operators with ZO perturbations triggers a variance explosion of

\mathcal{O}(1/\varepsilon^2)

, leading to numerical divergence. To address these challenges, we propose the \textbf{S}tochastic \textbf{D}imension-free \textbf{Z}eroth-order \textbf{E}stimator (\textbf{SDZE}), a unified framework that achieves dimension-independent complexity in both space and memory. Specifically, SDZE leverages \emph{Common Random Numbers Synchronization (CRNS)} to algebraically cancel the

\mathcal{O}(1/\varepsilon^2)

variance by locking spatial random seeds across perturbations. Furthermore, an \emph{implicit matrix-free subspace projection} is introduced to reduce parameter exploration variance from

\mathcal{O}(P)

\mathcal{O}(r)

while maintaining an

\mathcal{O}(1)

optimizer memory footprint. Empirical results demonstrate that SDZE enables the training of 10-million-dimensional PINNs on a single NVIDIA A100 GPU, delivering significant improvements in speed and memory efficiency over state-of-the-art baselines.

Zhangyong Liang, Huanhuan Gao

Zeroth-Order Optimization Partial Differential Equations Automatic Differentiation

Ask PDF

May 7, 2026cs.LG

INEUS: Iterative Neural Solver for High-Dimensional PIDEs

In this paper, we introduce INEUS, a meshfree iterative neural solver for partial integro-differential equations (PIDEs). The method replaces the explicit evaluation of nonlocal jump integrals with single-jump sampling and reformulates PIDE solving as a sequence of recursive regression problems. Like Physics-Informed Neural Networks (PINNs), INEUS learns global solutions over the entire space-time domain, yet it offers a more efficient treatment of nonlocal terms and avoids the computationally expensive differentiation of full PIDE residuals. These features make INEUS particularly well suited for high-dimensional PDEs and PIDEs. Supported by a contraction-based convergence proof for linear PIDEs, our numerical experiments show that INEUS delivers accurate and scalable solutions for various high-dimensional linear and nonlinear examples.

Jean-Loup Dupret, Davide Gallon, Patrick Cheridito

Partial Differential Equations Neural Solvers

Ask PDF

Jun 4, 2026math.NA

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains. In this work, a deep adaptive sampling framework for PINNs is extended to the time-dependent setting by treating space and time as a unified domain without any explicit time marching. A normalising flow neural network model effectively learns the distribution induced by the PDE residual and generates new collocation points concentrated in regions where the solution is most difficult to learn. Unlike conventional adaptive strategies that require explicit time stepping or moving meshes, high-residual regions are automatically identified and tracked across both space and time, driven purely by the PDE residual distribution. The effectiveness of the proposed strategy is assessed on a range of benchmark problems, from sharp and moving features in two spatial dimensions to localised structures in up to eight spatial dimensions.

Anshima Singh, David J. Silvester

Partial Differential Equations Classical Numerical Solver Adaptive Sampling

Ask PDF