Neural networks for spectral optimization

Alexis de Villeroché; Beniamin Bogosel; Stéphane Breuils; Dorin Bucur; Jacques-Olivier Lachaud

cs.LGSep 28, 2026

Neural networks for spectral optimization

Authors: Alexis de Villeroché, Beniamin Bogosel, Stéphane Breuils, Dorin Bucur, Jacques-Olivier Lachaud

Abstract

Given a functional dependent on the spectrum of a differential operator, we address the problem of finding a domain which optimizes this functional. PDE solvers might be used to tackle this optimization. It is however computationally expensive. We propose two neural network models which learn the spectrum directly from the geometry of the domain and can be used to optimize the domain from one or more eigenvalues. We investigate two representations. The first encodes the domain through Fourier coefficients and a light MLP, which is efficient on star-shaped geometries, achieving a precision of 0.2%. Through a rescaling of the coefficients the designed models satisfy the scaling law of the eigenvalues. Additionally, averaging the outputs of the trained surrogates over rotations and reflections induces invariance for these transformations. The second is a model that takes the landscape function, the indicator function and the gradient of the landscape function. A Gram-Schmidt process produces orthogonal eigenfunctions as output of the model along with the associated eigenvalues. The landscape model reaches 1% mean relative error on the first ten eigenvalues, compared with 4% for an FNO model. Replacing the landscape by an SDF worsened both prediction and optimization errors. The trained model also generalizes from synthetic shapes to domains given as classical image dataset. The resulting surrogates of both approaches recover classical spectral optima such as the disk for the first eigenvalue or the conjectured minima of higher eigenvalues. This confirms that our models produce accurate differentiable estimates of eigenvalues, which can be used in shape optimization problems involving spectral quantities.

Figures & tables

Figure 1. Sample from the Fourier database.

\lambda_{5}=78.20

\lambda_{6}=88.52

\lambda_{7}=106.14

\lambda_{8}=118.92

\lambda_{9}=132.68

\lambda_{10}=142.72

Table 1. Optimal values for

\{\lambda_{5},\dots,\lambda_{10}\}

computed by Antunes and Freitas

	ReLU	SiLU	Tanh
width : time : $\lambda_{1}$ : $\lambda_{2}$ : $\lambda_{3}$ : $\lambda_{4}$ : $\lambda_{5}$ : $\lambda_{6}$ : $\lambda_{7}$ : $\lambda_{8}$ : $\lambda_{9}$ : $\lambda_{10}$ :	$90$ $192$ $300$ $556$ $1097$ $1684$ $0.144$ $0.089$ $0.073$ $0.168$ $0.087$ $0.071$ $0.249$ $0.130$ $0.100$ $0.229$ $0.140$ $0.119$ $0.275$ $0.173$ $0.150$ $0.285$ $0.177$ $0.155$ $0.347$ $0.187$ $0.151$ $0.378$ $0.204$ $0.166$ $0.315$ $0.194$ $0.169$ $0.373$ $0.252$ $0.223$	$90$ $192$ $300$ $542$ $1002$ $1716$ $0.099$ $0.054$ $0.044$ $0.132$ $0.072$ $0.054$ $0.180$ $0.096$ $0.075$ $0.214$ $0.128$ $0.104$ $0.262$ $0.154$ $0.121$ $0.225$ $0.147$ $0.118$ $0.277$ $0.172$ $0.142$ $0.314$ $0.189$ $0.153$ $0.276$ $0.184$ $0.157$ $0.340$ $0.237$ $0.198$	$90$ $192$ $300$ $842$ $997$ $1795$ $0.147$ $0.097$ $0.086$ $0.217$ $0.112$ $0.089$ $0.301$ $0.170$ $0.134$ $0.277$ $0.188$ $0.159$ $0.325$ $0.223$ $0.182$ $0.328$ $0.221$ $0.184$ $0.398$ $0.244$ $0.189$ $0.429$ $0.264$ $0.200$ $0.371$ $0.242$ $0.194$ $0.472$ $0.308$ $0.256$

Table 2. Mean relative error (%) by eigenvalue and training time (s) of the surrogate depending on the width of the hidden layers of the networks and their activation function.

Figure 2. Optimized shapes for

\{\lambda_{5},\dots,\lambda_{10}\}

with activation function ReLU (top) and SiLU (bottom) and associated eigenvalues computed with the PDE solver.

Figure 3. Optimal shapes for different functions

F

Figure 4. One representative domain for each different shape.

	$\lambda_{1}$	$\lambda_{2}$	$\lambda_{3}$	$\lambda_{4}$	$\lambda_{5}$	$\lambda_{6}$	$\lambda_{7}$	$\lambda_{8}$	$\lambda_{9}$	$\lambda_{10}$
our model	$0.24\%$	$1.08\%$	$1.07\%$	$1.07\%$	$0.98\%$	$1.05\%$	$1.01\%$	$0.99\%$	$0.98\%$	$0.85\%$

Table 3. Mean relative test error for each eigenvalue after validation.

	simple	star	polygon	triangle	non-convex	holed	several parts
Our model	$0.33\%$	$1.13\%$	$0.29\%$	$0.47\%$	$2.55\%$	$2.33\%$	$0.97\%$

Table 4. Mean relative test error for each shape after validation.

Architecture	landscape function	Indicator
our model	$\mathbf{0.93\%}$	$11.54\%$
FNO	$2.33\%$	$14.71\%$
DeepONet	$1.47\%$	$11.57\%$

Table 5. Mean relative error of

\{\lambda_{1},\dots,\lambda_{10}\}

, by architecture and input representation.

	$\lambda_{1}$	$\lambda_{2}$	$\lambda_{3}$	$\lambda_{4}$	$\lambda_{5}$	$\lambda_{6}$	$\lambda_{7}$	$\lambda_{8}$	$\lambda_{9}$	$\lambda_{10}$
mean	$2.9\%$	$8.4\%$	$8.0\%$	$7.6\%$	$7.2\%$	$7.5\%$	$6.7\%$	$6.5\%$	$6.7\%$	$7.5\%$
median	$1.5\%$	$7.4\%$	$6.9\%$	$5.5\%$	$5.4\%$	$5.3\%$	$4.6\%$	$4.3\%$	$4.2\%$	$5.0\%$

Table 6. Relative error per eigenvalue on MPEG-7, mean relative error

6.88\%

, median relative error

4.83\%

Figure 5. On the left, we show the starting domain’s boundary, on the right, the boundary of the domain after the gradient descent. On top, $\hat{\lambda}_{2}$ converges to the exact value for two equal disks ( $2.35\%$ of relative error). After 100 iterations $a_{1}/b_{1}=1.08$ , $a_{2}/b_{2}=1.05$ , $R_{1}/R_{2}=0.99$ . On bottom, $\hat{\lambda}_{6}$ does not hallucinate with respect to the eigenvalue computed with FreeFem ( $2.5\%$ of relative error).

Explore similar work

May 9, 2026cs.LG

Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates

Neural operators for time-dependent PDEs face a structural tension: spectral architectures (FNO and descendants) inherit exponential rollout-error growth from their one-step Lipschitz constant, while hierarchical U-Net operators trade resolution invariance for multi-scale detail. We introduce SpectraNet, an autoregressive neural operator that composes truncated spectral convolutions inside a U-Net hierarchy with a Residual-Target Spectral Block trained under a Semigroup-Consistency Loss. The residual-target parametrization replaces L^T stability blow-up with linear T*delta drift, and the spectral path's parameter count is Theta(L w^2 M^2), independent of grid N. Under a single unified protocol against 16 published neural-operator baselines on Navier-Stokes nu=1e-5 at 64x64, SpectraNet reaches test relative L2 = 0.0822 at 2.04M parameters -- 2.33x fewer than canonical FNO at ~20% lower error -- and wins five of six rows in a cross-PDE comparison against FNO (NS at nu in {1e-4, 1e-3}, PDEBench Shallow-Water 2D and Diffusion-Reaction, with the Active-Matter row going to FNO inside its seed spread). Trained from scratch at native 128^2 under the same protocol, SpectraNet improves to 0.0724 while FNO regresses to 0.3080. Free rollout stays bounded for T=100 where FNO diverges across all 200 test trajectories. On consumer CPU at B=1, SpectraNet runs sub-200ms while the full-attention Transformer that wins raw L2 pays ~60x latency; we do not claim to beat that Transformer on raw L2, only to dominate the lightweight (<=5M parameter, sub-200ms CPU) Pareto frontier. Source code: https://github.com/Enrikkk/spectranet

Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup +2

University of New Orleans, New Orleans, LA, USA · Naval Research Laboratory, Stennis Space Center, MS, USA

Neural Operators Partial Differential Equations Spectral Representation Method U-Net

Ask PDF

Aug 30, 2026cs.LG

Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains

Learning solution operators for partial differential equations (PDEs) on irregular and geometry-dependent domains remains a central challenge in scientific machine learning. While spectral methods provide strong inductive biases for modeling global interactions, they are typically limited to regular domains, and existing neural approaches often require domain warping, interpolation, or costly geometric embeddings. We introduce the \textbf{Graph Spectral Neural Operator (GSNO)}, a neural operator that combines spatial graph spectral decompositions with temporal Fourier transforms through a unified space--time spectral kernel. This formulation enables globally coherent operator learning on non-Cartesian discretizations without domain warping or autoregressive rollouts. By replacing learned geometric embeddings with a graph Laplacian spectral basis, GSNO provides geometry-aware spectral learning with low parameter complexity. Across steady and unsteady PDE benchmarks on irregular and geometry-dependent domains, GSNO achieves strong accuracy with reduced runtime and parameter counts, while demonstrating robust zero-shot generalization across mesh resolutions and geometry families.

Abdolmehdi Behroozi, Chaopeng Shen

Department of Civil and Environmental Engineering, Penn State University, University Park, PA 16802, USA

Neural Operators Partial Differential Equations Graph Laplacians

Ask PDF

Jan 23, 2026cs.LG

SFO: Learning PDE Operators via Spectral Filtering

Partial differential equations (PDEs) govern complex systems, yet neural operators often struggle to efficiently capture the long-range, nonlocal interactions inherent in their solution maps. We introduce Spectral Filtering Operator (SFO), a neural operator that parameterizes integral kernels using the Universal Spectral Basis (USB), a fixed, global orthonormal basis derived from the eigenmodes of the Hilbert matrix in spectral filtering theory. Motivated by our theoretical finding that the discrete Green's functions of shift-invariant PDE discretizations exhibit spatial Linear Dynamical System (LDS) structure, we prove that these kernels admit compact approximations in the USB. By learning only the spectral coefficients of rapidly decaying eigenvalues, SFO achieves a highly efficient representation. Across six benchmarks, including reaction-diffusion, fluid dynamics, and 3D electromagnetics, SFO achieves state-of-the-art accuracy, reducing error by up to 40% relative to strong baselines while using substantially fewer parameters.

Noam Koren, Rafael Moschopoulos, Kira Radinsky +1

Department of Computer Science Technion - Israel Institute of Technology · Department of Computer Science Princeton University

Neural Operators Partial Differential Equations Spectral Filtering

Ask PDF