Ermakov--Based embeddings as a parameter-free layer for physics-informed neural networks
Authors: Roberto Bernal-Jaquez, Roberto Romero Huerta, Alexander Schaum
Organizations: Departamento de Matemáticas Aplicadas y Sistemas, Universidad Autónoma Metropolitana–Cuajimalpa, Mexico City, Mexico · Posgrado en Ciencias Naturales e Ingeniería, Universidad Autónoma Metropolitana–Cuajimalpa, Mexico City, Mexico · Department for Process Analytics, University of Hohenheim, Stuttgart, Germany
Evolution equations with time-dependent coefficients force a neural solver to spend capacity on a scale that the equation already determines. We show that a single classical object, the Ermakov-Pinney scale function ρ, can be written into the input layer of a physics-informed neural network as a parameter-free embedding. One construction covers three problem families through the two signs of the Pinney constant c: linear diffusion (c=−1, ω≡0), diffusion in a time-dependent confining trap (c=−1, ω=0), and the quantum oscillator with time-dependent frequency (c=+1). An exact moment identity gives ρ in closed form from the initial data alone, so no auxiliary network or knowledge of the solution is needed. Our central finding is comparative: each family requires a different component of the construction and has a different honest control. In family I, the diffusive-time reparametrisation alone reproduces most of the gain, so that family is a poor test of the idea and shows why an ablation is mandatory. In family II no time reparametrisation exists, the control degrades accuracy, and the amplitude prefactor becomes essential (40.8x). In family III the adiabatic approximation also fails, and the quadratic phase factor is what matters (5.7x). The method does not help when the reduced solution does not converge to a fixed point: its extrapolation advantage then disappears. The reduction itself is the classical scaling-variable change of the self-similar-asymptotics literature; what is new is the moment identity that makes it architectural for time-dependent coefficients, and the comparative anatomy above.
Figures & tables
Model
Input
Relative L2 error
Gaussian
bimodal
PINN
(x,t)
3.84×10−3±4.7×10−4
9.63×10−3±2.6×10−3
PINN- τ
(x,τ)
1.76×10−3±2.5×10−4
8.16×10−3±2.7×10−3
EB-coords
(ξ,θ)
9.56×10−4±1.4×10−4
6.33×10−3±3.3×10−3
EB-Net
(ξ,θ) , ρ−αv
1.16×10−3±4.7×10−4
5.53×10−3±2.0×10−3
Table 1: Family I. Mean ± s.d. over five seeds; all models have 12737 parameters. The gain from PINN to PINN- τ uses no Ermakov structure. The amplitude prefactor is neutral here (EB-coords vs EB-Net ).
Evaluation D(t)
PINN
PINN- τ
EB-Net
D1 (training)
3.32×10−3
1.83×10−3
1.05×10−3
D2 (phase +π/2 )
6.12×10−2
1.83×10−3
8.84×10−4
D3 ( f=3 , a=0.9 )
5.35×10−2
1.85×10−3
1.01×10−3
D4 ( D0=1.6 , fast) †
1.26×10−1
1.14×10−2
1.81×10−3
D5 ( D0=0.4 , slow)
2.26×10−1
1.62×10−3
6.56×10−4
† only D4 leaves the trained diffusive-time interval.
Table 2: Zero-shot transfer in family I: the same weights evaluated under a different D(t) , with no retraining. PINN- τ matches EB-Net on the schedules that stay inside the trained diffusive-time interval.
Model
physical residual
reduced residual
PINN
3.84×10−3
3.64×10−2
PINN- τ
1.76×10−3
1.99×10−2
EB-coords
9.56×10−4
1.05×10−2
EB-Net
1.16×10−3
1.04×10−2
Table 3: Family I with the residual reweighted by ρ3/D (normalised to unit mean), applied identically to all four models. Absolute accuracy worsens by about an order of magnitude; the ordering survives.
Model
Relative L2 error
Extrapolation t∈[T,2T]
Gaussian
bimodal
Gaussian
bimodal
PINN
1.19×10−2±1.2×10−3
1.74×10−2±1.4×10−3
1.21×100
1.14×100
PINN- τ
8.00×10−2±4.5×10−2
1.13×10−1±3.6×10−2
4.16×10−1
2.89×10−1
EB-coords
4.13×10−2±4.9×10−2
9.80×10−2±2.0×10−2
6.31×10−1
2.86×100
EB-Net
1.01×10−3±9.7×10−5
6.32×10−3±6.3×10−4
6.81×10−3
7.09×10−2
Table 4: Family II. Mean ± s.d. over three seeds. PINN- τ is now worse than the plain baseline; the amplitude prefactor separates EB-coords from EB-Net by 40.8× .
Figure 1: Family II. (a,b) in-domain error; (c,d) instantaneous error for weights trained only on t∈[0,T] , shaded region extrapolated.
Model
Input
Relative L2 error
ground state
superposition
PINN
(x,t)
9.99×10−2±6.1×10−3
3.40×10−1±9.5×10−2
PINN-ad
(xω,∫ω)
1.16×10−1±2.1×10−2
4.19×10−1±9.5×10−2
EB-coords
(ξ,θ)
4.10×10−2±3.2×10−3
1.14×10−1±1.7×10−2
EB no-phase
(ξ,θ) , ρ−1/2φ
4.19×10−2±3.9×10−3
9.16×10−2±2.7×10−2
EB-Net
(ξ,θ) , ρ−1/2eiSφ
7.31×10−3±1.0×10−3
6.07×10−2±1.6×10−2
Table 5: Family III. Mean ± s.d. over three seeds. The adiabatic control is worse than the plain PINN in both cases. Omitting the quadratic phase costs 5.7× on the ground state.
Figure 2: Family III. Note in (b,d) that no model extrapolates when the reduced solution beats rather than converging (remark 5 ).
Family
Pinney branch
Trivial control
Load-bearing piece
vs PINN
piece’s share
I. diffusion ut=Duxx
c=−1 , ω=0
τ (suffices)
coordinates only
4.0
prefactor: 1.22 ×
II. trap +γ(xu)x
c=−1 , ω=0
τ (fails)
+ prefactor ρ−α
11.7
prefactor: 41 ×
III. quantum ω(t)
c=+1
adiabatic (fails)
+ phase eiS
13.7
phase: 5.7 ×
Table 6: The central result. One construction, three families, and a different component load-bearing in each. Gains are EB-Net versus the plain PINN on the favourable initial condition of each family; the last column isolates the named component.
Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.
Physics-Informed Neural Networks (PINNs) embed governing equations into deep learning, but enforce them only through loss residuals, leaving highly oscillatory wave behavior to be discovered by optimization. As a result, methods that achieve relative L2 errors below 10−3 on standard manufactured Helmholtz benchmarks can fail on practical radiation problems involving singular excitations, absorbing boundaries, and wave fields spanning tens of wavelengths. Architectural physics embedding addresses this limitation by factorizing the field into analytically derived oscillatory kernels and learnable envelopes. However, the kernel dictionary must be manually constructed and scales with the number of elementary units, growing exponentially with the depth of hierarchically structured systems such as antenna arrays and metasurfaces. We propose PE-EK-PINN (Physics Embedded with Evolving Kernels), which treats physics kernels as reusable learned representations rather than fixed analytical inputs. A converged subsystem field is frozen and promoted to an evolved kernel, whose transformed copies are reused to represent higher-level configurations without deriving new governing equations. The resulting hierarchy makes the peak number of active kernels independent of system size and reduces cumulative training cost from O(N) to O(logN). Experiments on dipole arrays, composite line-source geometries, and cross arrays demonstrate the dramatic training cost reduction, while achieving a reduced or comparable relative L2 error. One notable example is PE-EK-PINN solves a 256-dipole array more than 30 times faster than direct PE-PINN.
Huiwen Zhang, Feng Ye, Chu Ma
Department of Electrical & Computer Engineering Madison, WI, 53706, USA
Amortizing physics-informed neural networks (PINNs) across related PDEs requires describing each equation to a reusable solver. Coefficient vectors encode numerical parameters in predefined slots, leaving operator and cross-field assignments implicit. We make these relationships explicit in an operator graph, with nodes for fields, derivatives, terms, and residuals and coefficients retained as term attributes. A graph hypernetwork generates diagonal codes that initialize a meta-trained factorized PINN for each target equation. Meta-training and target-specific adaptation use governing equations and prescribed conditions without solution labels. We compare coefficient-vector, DeepSets-based term-set, and graph conditioning by solution accuracy within a fixed adaptation budget. In scalar convection-diffusion-reaction problems, both term-based descriptors improve high-reaction accuracy, with similar performance. In two-field Fisher-KPP, meta-training sees uncoupled and one-way systems; after 3,000 adaptation steps on unseen two-way coupling, the graph's mean final error is 35.7% below the term set and 67.7% below the coefficient vector. In a fixed-structure capacitively coupled plasma model, the coefficient vector performs best. These results support extending coefficient conditioning with explicit equation relationships for physics-based solver adaptation.
We present an improved neural field architecture for solving partial differential equations (PDEs). Current physics-informed neural networks (PINNs) provide a flexible framework for solving PDEs, but they struggle to achieve highly accurate solutions and require computation that scales poorly with parameter count. Our model, which we call beignet (Bandlimited Embedding with Interpolated Grid Network), replaces the random Fourier feature embedding used by existing PINN models with a trainable multi-resolution Fourier feature pyramid. To query beignet at a continuous coordinate, we use Fourier interpolation at each level of the pyramid to return features at the input coordinate, and then decode this vector with a fully-connected neural network trunk. Our model provides multiple benefits: 1) Spatial derivatives can be computed efficiently by using the chain rule to compose derivatives of the neural network computed with automatic differentiation with derivatives of the feature grid computed spectrally by the Fast Fourier transform (FFT). 2) beignet can achieve higher accuracy in a compute-efficient manner by scaling the parameter count of this Fourier feature pyramid, instead of the less-efficient strategy of scaling the neural network architecture. 3) beignet can directly control the representation bandlimit, resulting in more stable optimization for difficult PDEs. We demonstrate that beignet finds significantly more accurate solutions on PDE benchmarks using fewer parameters than state-of-the-art PINN methods. We further evaluate beignet on the self-similar inviscid Burgers blowup problem and show that it can minimize residuals to near machine precision using Adam, an accuracy regime previously attained only by using computationally expensive higher-order optimizers.
Brandon Zhao, Yixuan Wang, Jonathan T. Barron +3
Department of Computing and Mathematical Sciences, California Institute of Technology · Google DeepMind · Cahill Center for Astronomy and Astrophysics, California Institute of Technology