Operator eigenvalue problems appear throughout science. Classical methods usually discretize the operator into a matrix and then solve the resulting matrix eigenvalue problem. This works well in low dimensions, but fine grids quickly become expensive in both memory and computation as the dimension grows. Neural network based solvers avoid storing these large grids, but recent state of the art neural methods can require hundreds of thousands of training steps and may struggle to find the desired eigenvalues. We show that the two approaches can help each other. A coarse finite difference method (FDM) calculation acts as a cheap numerical model of the operator spectrum. We use the approximate eigenvalues as fixed shifts during the training of the neural solver, as they only need to locate the relevant part of the spectrum. We also introduce Stabilized Inverse Power Method Neural Network (SIPMNN), a more stable training procedure for higher-dimensional problems. Across five test problems at d=10, the combined approach is more accurate overall than the tested fully neural alternatives while using eight to ten times fewer iterations.
Figures & tables
Figure 1: SIPMNN first obtains rough eigenvalues from one coarse FDM solve. Their shifts stay fixed while the neural networks refine each mode. Learned deflation passes each lower-mode estimate to the higher-mode updates.
IPMNN
SIPMNN
Benchmark
Reference λ1/λ2
Adaptive shift
Fixed FDM shift
Adaptive shift
Fixed FDM shift
d=5
Anharmonic
2.7957/4.0060
0.26 /1.17
0.43/4.02
2.20/12.21
0.78/ 0.77
Coupled oscillator
2.4597/3.3541
3.70/2.92
3.12 /238.40
137.61/60.42
8.00/ 1.10
Double well
4.3482/5.1403
0.10/ 0.11
0.11/ 0.11
0.05 /0.26
0.07/0.13
Quantum harmonic
2.5000/3.5000
3.14/5.63
2.45 /429.46
218.16/61.66
8.08/ 1.80
Table 1: Relative eigenvalue error (%) after 50,000 neural optimization steps, reported as λ1/λ2 . The lowest error for each mode is bold. The four configurations compare two training procedures and two choices of spectral shift.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Domain
Potential V(x)
Anharmonic
[−4,4]d
21∑jxj2+0.1∑jxj4
Coupled oscillator
[−5,5]d
21[0.8∥x∥2+0.2(1⊤x)2]
Double well
[−2.5,2.5]d
∑j(xj2−1)2
Quantum harmonic
[−5,5]d
21∥x∥2
Random Gaussian
[0,1]d
Sum of eight Gaussian wells
Appendix
Table 2: Schrödinger benchmark operators and domains.
Benchmark
Coarse FDM
Fixed FDM SIPMNN
Anharmonic
20.90/15.65
1.38/1.81
Coupled oscillator
35.38/21.89
28.06/5.10
Double well
10.87/12.01
0.53/1.50
Quantum harmonic
37.07/23.79
26.23/12.34
Random Gaussian
2.26/4.28
0.00/0.01
Appendix
Table 3: Error of the coarse n=5 FDM estimate and the final SIPMNN result at d=10 , reported as λ1/λ2 in percent.
Benchmark
Full
No smooth target
No delayed update
No sigma penalty
Anharmonic
1.38 / 1.81
5.99/1.82
8.09/5.02
161.21/24.88
Coupled oscillator
28.06/ 5.10
25.99/9.44
25.63 /10.61
124.92/5.14
Double well
0.53/1.50
1.21/ 0.46
0.10 /1.46
66.71/98.45
Quantum harmonic
26.23 /12.34
29.67/13.93
30.05/ 10.59
139.69/152.31
Random Gaussian
0.00 / 0.01
0.00 / 0.01
0.00 /23.07
0.01/0.02
Appendix
Table 4: Fixed FDM SIPMNN ablation at d=10 . Entries are relative errors (%) for λ1/λ2 . The lowest error for each mode is bold.
Benchmark
Residual r1/r2
Anharmonic
0.078/0.124
Coupled oscillator
0.046/0.318
Double well
0.057/0.209
Quantum harmonic
0.064/0.057
Random Gaussian
0.032/0.395
Appendix
Table 5: Normalized residuals r1/r2 for Fixed FDM SIPMNN at d=10 .
Component
Configuration
Purpose
Coarse FDM
Tensor grid with n=5 interior points per coordinate and h=(b−a)/(n+1) on a nonperiodic interval [a,b]
Fixed shifts
Neural collocation
50,000 static Latin hypercube points sampled separately from the FDM grid
Training
Anharmonic and Double Well references
Independent one-dimensional FDM with 800 interior points and a second-order central difference scheme. The d -dimensional values are sums of one-dimensional eigenvalues
Evaluation
Coupled Oscillator and Quantum Harmonic references
Analytic eigenvalues
Evaluation
Random Gaussian references
Independent Galerkin calculation in a sine basis
Evaluation
Appendix
Table 6: Numerical calculations used for shifts, training, and evaluation.
Setting
Value
Coarse operator
Second-order central-difference stencil
Coarse eigensolver
LOBPCG [ Knyazev, 2001 ] , tolerance 10−7 , at most 500 iterations
Random seed
0 for all random number generators
Random Gaussian potential
V(x)=∑ℓ=18aℓexp(−∥x−cℓ∥22/0.152) , with cℓ∼Uniform([0,1]d) and aℓ∼Uniform([−5,5])
Boundary condition
Map coordinates to x∈[−1,1]d and multiply the raw output by ∏j(1−xj2)
Neural network architecture
[d,40,40,40,40,1] at d=5 and
Appendix
Table 7: Implementation and training configuration.
Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.
Given a functional dependent on the spectrum of a differential operator, we address the problem of finding a domain which optimizes this functional. PDE solvers might be used to tackle this optimization. It is however computationally expensive. We propose two neural network models which learn the spectrum directly from the geometry of the domain and can be used to optimize the domain from one or more eigenvalues. We investigate two representations. The first encodes the domain through Fourier coefficients and a light MLP, which is efficient on star-shaped geometries, achieving a precision of 0.2%. Through a rescaling of the coefficients the designed models satisfy the scaling law of the eigenvalues. Additionally, averaging the outputs of the trained surrogates over rotations and reflections induces invariance for these transformations. The second is a model that takes the landscape function, the indicator function and the gradient of the landscape function. A Gram-Schmidt process produces orthogonal eigenfunctions as output of the model along with the associated eigenvalues. The landscape model reaches 1% mean relative error on the first ten eigenvalues, compared with 4% for an FNO model. Replacing the landscape by an SDF worsened both prediction and optimization errors. The trained model also generalizes from synthetic shapes to domains given as classical image dataset. The resulting surrogates of both approaches recover classical spectral optima such as the disk for the first eigenvalue or the conjectured minima of higher eigenvalues. This confirms that our models produce accurate differentiable estimates of eigenvalues, which can be used in shape optimization problems involving spectral quantities.
Alexis de Villeroché, Beniamin Bogosel, Stéphane Breuils +2
Partial differential equations (PDEs) govern complex systems, yet neural operators often struggle to efficiently capture the long-range, nonlocal interactions inherent in their solution maps. We introduce Spectral Filtering Operator (SFO), a neural operator that parameterizes integral kernels using the Universal Spectral Basis (USB), a fixed, global orthonormal basis derived from the eigenmodes of the Hilbert matrix in spectral filtering theory. Motivated by our theoretical finding that the discrete Green's functions of shift-invariant PDE discretizations exhibit spatial Linear Dynamical System (LDS) structure, we prove that these kernels admit compact approximations in the USB. By learning only the spectral coefficients of rapidly decaying eigenvalues, SFO achieves a highly efficient representation. Across six benchmarks, including reaction-diffusion, fluid dynamics, and 3D electromagnetics, SFO achieves state-of-the-art accuracy, reducing error by up to 40% relative to strong baselines while using substantially fewer parameters.
Noam Koren, Rafael Moschopoulos, Kira Radinsky +1
Department of Computer Science Technion - Israel Institute of Technology · Department of Computer Science Princeton University
Neural operators serve as fast, data-driven surrogates for scientific modeling but typically rely on a monolithic, single-pass inference procedure that struggles to resolve high-frequency details, a limitation known as spectral bias. We introduce the Iterative Refinement Neural Operator (IRNO), which augments pre-trained operators with a learned refinement module iteratively applied via fixed-point iteration. IRNO decomposes the prediction into a coarse initialization followed by successive residual corrections, paralleling classical numerical solvers. Under local assumptions, we establish contraction of the induced operator, ensuring convergence to a unique fixed point. To explicitly target high-frequency errors, we propose a progressive spectral loss that adaptively increases penalty on high-frequency components over refinement steps during training. Across physical systems, IRNO consistently lowers error, with up to 56.05% improvement on turbulent flow. On Active Matter, spectral analysis reveals that, relative to base operator, the normalized error ratios decrease to 27.72-36.10% in low-, 5.07-6.68% in mid-, and 1.48-2.04% in high-frequencies, remaining stable beyond the trained iteration count. Code is available at https://github.com/xiaotianliu-dartmouth/Iterative_Refinement_Neural_Operator
Xiaotian Liu, Shuyuan Shang, Xiaopeng Wang +2
1Dartmouth College · 2CUHK Shenzhen · 3Lawrence Berkeley National Lab