Overview

Spectral methods study problems by expressing data or operators in terms of eigenvalues, eigenvectors, or frequency modes. Classical approaches include Fourier and graph-Laplacian transforms, PCA and low-rank approximation, kernel methods, and polynomial filters; they are used for tasks such as clustering networks, estimating dimensions, and solving or learning partial differential equations. Newer work also uses spectral structure to design neural operators and optimizers, diagnose model behavior, and speed up large matrix computations. A central challenge is balancing efficient, robust approximations with the assumptions and information lost when choosing a fixed basis, truncating modes, or relying on noisy spectra.

Latest papers

247 papers11 papers in the last four weeks, down 56% on the four weeks before. 0.1% of all new papers.

Feb 16, 2025

Physics-Informed Support Vector Kernels via Green-Function Analogies and Jackson-Chebyshev Spectral Design

Kernel selection for regression of physical observables is often heuristic. We investigate a physics-informed strategy in which functional forms and spectral structures associated with Green's functions motivate kernel selection without requiring an exact identification between a machine-learning kernel and a physical propagator. The principal construction is a Jackson-damped Chebyshev kernel inspired by the kernel polynomial method (KPM); its explicit feature map yields a positive-semidefinite Gram matrix by construction and provides an inspectable spectral prior for structured observables. We evaluate standard and custom SVR models on copper-conductivity proxies, local Dirac-like band dispersion, quartic-oscillator energy levels, photonic-crystal transmission, and Fibonacci-chain transmission using repeated nested validation, learning curves, random-forest and multilayer-perceptron baselines, and low-rank Nyström tests where relevant. The framework is intended for finite-data regression of precomputed observables while boundary conditions remain part of the physical model that generates those observables.
Jan 18, 2025

Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer

The principal objective of this work is twofold within nonparametric regression settings: (1) to establish the minimax optimal convergence rates for fixed-bandwidth Gaussian kernel spectral algorithms when the true regression function resides in a Sobolev space, and (2) to apply Gaussian spectral algorithms for achieving robust and adaptive transfer learning under concept shift. While minimax optimality of misspecified spectral algorithms has been established, existing guarantees are typically restricted to the non-saturation regime. We demonstrate that the infinite smoothness of fixed-bandwidth Gaussian kernels provides universal robustness to model misspecification by showing that this kernel choice enables any spectral algorithm to attain minimax optimal rates, provided the regularization parameter decays exponentially. This result effectively decouples optimality from the algorithm's inherent qualification. Building on this, we then advocate Gaussian spectral algorithms as powerful components in a learning framework for robust and adaptive transfer. Specifically, we derive the adaptive convergence rate of the excess risk for this framework and show that the rates are optimal up to logarithmic factors. Our results also reveal the impact of the magnitude of the concept shift and the sample size on the generalization error.
Jul 8, 2024

Adversarial dynamical systems characterize when data-driven learning succeeds or fails

Many systems resist analytical modeling, making data-driven inference of dynamics important. Yet data-driven methods can fail to converge or generalize, leaving open a central question: When can system behavior be learned reliably from data, and when is such learning impossible? We answer this question using adversarial dynamical systems to identify the boundary between accessible and inaccessible regimes. In Koopman operator learning, a leading framework for representing nonlinear dynamics through linear spectral objects, we design optimal data-driven spectral algorithms with convergence and certification guarantees under conditions arising broadly in physical systems. This yields a convergence theory for Koopman-operator approximations and resolves a longstanding open problem in Koopman spectral analysis. Conversely, by constructing adversarial systems, we prove matching impossibility results: without these conditions, no single-sequence limiting procedure can guarantee learning, regardless of data quality. These results sharply characterize when data-driven spectral learning can succeed and when it must fail. We validate the framework on oscillators, chaotic fluid flows and Arctic sea ice concentration forecasting. In the latter, we uncover hidden modes of Arctic sea ice decline, deliver long-range forecasts with geographic error bounds, and outperform state-of-the-art dynamical and deep learning models at substantially lower computational cost, enabling real-time deployment on standard CPUs.
Mar 25, 2024

Hearing the shape of an arena with spectral swarm robotics

Swarm robotics relies on robots with minimal sensing and computation that achieve self-organization at the level of the collective to address a variety of tasks. Yet swarm robotics still struggles with global tasks that require understanding the broader spatial context in which robots operate, such as identifying the shape of the arena in which they are embedded. Here, we introduce spectral swarm robotics, a framework in which robots recover global spectral descriptors of the swarm's geometric structure through strictly local interactions. To do so, robots emulate diffusion to locally estimate Laplacian spectral invariants, quantities normally obtained only through centralized computation. These spectral values serve as compact descriptors of the environment. In the present proof of concept, robots estimated the second Laplacian eigenvalue and used it to classify arenas from a predefined set of shapes with distinct spectral signatures. We show that estimating a single eigenvalue is sufficient for one-shot classification among selected confined environments with distinct spectral signatures. We uncovered a scaling law linking swarm size (a global parameter) and interaction radius (a local parameter) that ensures robust estimation. We validated the approach experimentally on two distinct hardware platforms, Kilobots and Pogobots, achieving reliable classification of arena shapes under realistic noise and sparse-connectivity conditions. Last, we discuss how locally recovering Laplacian spectral invariants opens avenues for swarm robotics in settings where producing efficient collective behaviors strongly depends on the global structure of the swarm.
Jan 16, 2024

Semidefinite programming relaxations and debiasing for MAXCUT-based clustering

In this paper, we consider the problem of partitioning a small data sample of size nn drawn from a mixture of 22 sub-gaussian distributions in Rp\mathbb{R}^p. We consider semidefinite programming relaxations of an integer quadratic program that is formulated essentially as finding the maximum cut on a graph, where edge weights in the cut represent dissimilarity scores between two nodes based on their pp features. We define the signal-to-noise ratio (SNR) as s2:=min⁡{npγ2,Δ2}s^2 := \min\{n p γ^2, Δ^2\}, where Δ2:=pγΔ^2 := p γ denotes the ℓ22\ell_2^2 distance between the two cluster centers. Our contributions are twofold. First, we provide a unified framework for analyzing three computationally efficient algorithms: SDP1, BalancedSDP, and Spectral clustering, yielding universal polynomial-rate misclassification guarantees for all three algorithms. Moreover, our theory allows for partial recovery (success rate <100%< 100\%) as long as s2s^2 is lower bounded by a constant. Second, we prove that the misclassification errors for SDP1 and BalancedSDP decay exponentially with respect to the SNR s2s^2 and the BalancedSDP requires no explicit debiasing when the two clusters have equal sizes. To our knowledge, this is the first time such results are obtained for semidefinite relaxations of MAX CUT in population clustering. We provide simulation evidence illuminating the theoretical predictions.
Nov 29, 2022

FC-PINO: High Precision Physics-Informed Neural Operators via Fourier Continuation

The physics-informed neural operator (PINO) is a machine learning paradigm that has demonstrated promising results for learning solutions to partial differential equations (PDEs). It leverages the Fourier Neural Operator to learn solution operators in function spaces and leverages physics losses during training to penalize deviations from known physics laws. Spectral differentiation provides an efficient way to compute derivatives for the physics losses, but it inherently assumes periodicity. When applied to non-periodic functions, this assumption can lead to significant errors, including Gibbs phenomena near domain boundaries which degrade the accuracy of both function representations and derivative computations. To overcome this limitation, we introduce the FC-PINO (Fourier-Continuation-based Physics-Informed Neural Operator) architecture which extends the accuracy and efficiency of PINO and spectral differentiation to non-periodic and non-smooth PDEs. In FC-PINO, we propose integrating Fourier continuation into the PINO framework, and test two different continuation approaches: FC-Legendre and FC-Gram. By transforming non-periodic signals into periodic functions on extended domains in a well-conditioned manner, Fourier continuation enables fast and accurate derivative computations. This approach avoids the discretization sensitivity of finite differences and the memory overhead of automatic differentiation. We demonstrate that standard PINO fails (without padding) or struggles (even with padding) to solve non-periodic and non-smooth PDEs with high precision, across challenging benchmarks. In contrast, the proposed FC-PINO provides accurate, robust, and scalable solutions, substantially outperforming PINO alternatives, and demonstrating that Fourier continuation is critical for extending PINO to a wider range of PDE problems when high-precision solutions are needed.
Feb 8, 2019

Robust Streaming PCA

We consider streaming principal component analysis when the stochastic data generating model is subject to perturbations. While existing models assume a fixed covariance, we adopt a robust perspective where the covariance matrix belongs to a temporal uncertainty set. Under this setting, we provide fundamental limits on convergence of any algorithm recovering principal components. We analyze the convergence of the noisy power method and Oja's algorithm, both studied for the stationary data generating model, and argue that the noisy power method is rate-optimal in our setting. Finally, we demonstrate the validity of our analysis through numerical experiments on synthetic and real-world datasets.