Spectral Methods

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 169

Sep 30, 2026cs.CV

Invariant Shape Analysis of Surfaces with Spherical Topology

Spherical harmonic descriptors of closed 3D shapes depend on the parameterization, the pose and the scale of the surface, and the standard rotation-invariant reductions, the power spectrum and the bispectrum, discard the relative orientation of the harmonic bands and cannot distinguish a shape from its mirror image. We construct a descriptor that removes all three dependencies exactly and loses nothing else: a conformal parameterization normalized by its conformal barycenter, followed by polynomial invariants of the rotation group. Identifying each harmonic band with a binary form turns the rotation quotient into classical invariant theory and makes reflections visible as the sign of an invariant, so chirality is recorded. The descriptor is complete for the truncated expansion, stable in the orbit distance, and comes with numerical diagnostics. Benchmarks confirm the guarantees, and on bilateral anatomical structures the descriptor separates mirror-image pairs from asymmetric pairs, which parity-blind descriptors cannot.

T. Shaska, M. -R. Siadat

Spherical Harmonics Spectral Methods

Sep 30, 2026cs.RO

Bounded Channel-Adaptive Spectral Learning for Forward-Consistent Inverse Flapping-Wing Aerodynamics

Flapping-wing vehicles regulate aerodynamic forces and moments through coordinated variations in stroke, deviation, and pitch motion. Because the resulting loads depend on both the instantaneous wing configuration and its preceding motion history, recovering suitable wing kinematics from a desired aerodynamic trajectory is a challenging inverse problem. Existing sequence models capture temporal dependencies, while spectral methods can exploit the periodic structure of flapping motion. However, unrestricted frequency-domain augmentation may interfere with temporal representations and produce inconsistent corrections across kinematic variables and prediction horizons. We propose the Bounded Channel-Adaptive Spectral Residual Gated Recurrent Unit (BCS-GRU), which retains recurrent temporal prediction as its primary representation and restricts spectral information to a controlled output-specific correction. We further introduce a causally aligned forward-consistency objective that evaluates predicted kinematics through a separately trained and frozen aerodynamic surrogate. Experiments under a unified episode-level protocol show that BCS-GRU improves inverse prediction over recurrent and adaptive-spectral baselines, with greater benefits at longer prediction horizons. Forward-consistent fine-tuning further improves surrogate-based aerodynamic consistency while maintaining mean kinematic accuracy. These results demonstrate that controlled spectral correction and forward-consistent learning provide an effective framework for history-aware inverse modelling of flapping-wing aerodynamics.

Haichuan Li

University of Turku

Recurrent Neural Networks Spectral Methods Inverse Dynamics

Sep 29, 2026cs.AI

Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders

Self-supervised pretraining reshaped prediction in language and vision, and brain foundation models (BFMs) inherited its promise. Representations learned from large unlabelled corpora should capture individual functional dynamics and generalise across cohorts. However, kernel ridge regression (KRR) fitted on functional connectivity (FC) matrices still predicts individual phenotypes more accurately than any BFM we tested. In this paper, we show that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction. We apply an efficient spectral filter to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance. Across the 5 datasets, 11 parcellations and 6 prediction targets we tested, we match or exceed the KRR baseline. Based on this finding, we then pretrain a small encoder model on about 4,000 hours of fMRI from 162 open datasets, whereby we align the pairwise similarities between the embeddings of recording snippets with those between the recalibrated connectomes. Our model performs on par with the best of the 6 published BFMs we tested while having an order of magnitude fewer parameters. Our encoder performs better than FC on short scans and in smaller cohorts, especially in fingerprinting. We release the pretrained model weights, the code and the pretraining data, preprocessed and parcellated.

Giovanni Marraffini, Victoria Shevchenko, Carlo Alberto Barbano +1

UNITO · MIND

Kernel Ridge Regression Medical Imaging Foundation Models Functional Connectivity Self-Supervised Pre-Training fMRI Foundation Models Spectral Methods

Sep 28, 2026stat.ML

Learning Conditional Expectation Operators via Functional Newton Updates

We introduce the Functional Spectral-Newton Method (FSNM) for learning the leading singular structure of a conditional expectation operator without fixing a basis or reproducing kernel Hilbert space. FSNM fits a low-rank representation of the centered joint-to-product density ratio kernel by alternating functional Newton updates. Each update reduces to a preconditioned regression, which we approximate with vector-valued regression trees in a stagewise boosting procedure. At the population level, we establish descent and an

O(1/T)

best-iterate block-stationarity rate under a relative weak-learner accuracy condition, and show that every nondegenerate local minimum over the full centered

L^2

spaces is a globally optimal rank-

d

approximation. Synthetic experiments show that FSNM recovers a low-rank density ratio and its leading spectral structure, and that the same learned kernel can answer multiple conditional queries without refitting.

Thiago Ramos, Alek Fröhlich, Daniel Perazzo +1

Federal University of São Carlos · CSML, Istituto Italiano di Tecnologia and University of Genoa · CSML, Istituto Italiano di Tecnologia and University College London

Density Ratio Estimation Low-Rank Approximation Spectral Methods Operator Learning

Sep 28, 2026eess.SP

Hierarchical Clustering and Signal Denoising on Digraphs

In this paper, we propose a representation of a digraph (directed graph) as a Hermitian matrix derived from its adjacency matrix. This representation characterizes both the connectivity and the edge orientation of the digraph. Based on the spectral decomposition of the Hermitian matrix, a digraph clustering algorithm with

k

-means is introduced to produce a partition on the graph. Applying this algorithm (bottom-up) recursively to a digraph with partially labeled vertices yields a spectral hierarchical digraph clustering (\myproj) algorithm that produces consistent nested partitions of the digraph, or equivalently, a tree structure. Furthermore, based on the in-degree and out-degree of each cluster in the digraph clustering, a pair of hierarchical interval partitions (filtrations) can be derived in a top-down manner to produce a pair of nested knot sequences. These knot sequences facilitate the construction of multilevel spline quasi-interpolants, enabling a noisy graph signal to be decomposed into a coarse approximation and inter-level details, followed by adaptive thresholding and reconstruction. Experiments on synthetic and real-world digraphs demonstrate the superiority of our {\myproj} algorithm for digraph clustering across diverse graph structural properties (homophily and heterophily) and supervision settings. Moreover, experiments on digraph signal processing using multilevel spline quasi-interpolants further demonstrate the effectiveness of signal recovery on digraphs in terms of RMSE and SNR.

Yi Wang, Sippanon Kitimoon, Hrushikesh N. Mhaskar +1

Department of Mathematics, City University of Hong Kong, Hong Kong, SAR China · Data Science Research Center, Faculty of Science, Chiang Mai University, Chiang Mai 50200, Thailand · Institute of Mathematical Sciences, Claremont Graduate University, Claremont, CA 91711, USA

Graph Signal Processing Hierarchical Clustering Graph Clustering Spectral Methods

Sep 27, 2026cs.LG

TNF based Spectral Embedding for Effective Application of Supervised Machine Learning Techniques in Automobile Insurance Fraud Detection

Fraud detection is an important area of research in the insurance business due to its financial implications. The primary aim of a fraud detection model is to identify fraud and non-fraud cases with high accuracy along with other important metrics such as Sensitivity, Specificity, Precision, F1-score, False Positive Rate, False Discovery Rate, AUC etc. To achieve this, we need to explore a suitable classification model to identify fraud and non-fraud cases. In this work, we have used auto insurance data set and explored classification models such as Decision Tree (DT), Random Forest (RF), XGBoost, LightGBM and Gradient Boosting Machine (GBM). To overcome the problem of data imbalance, we have employed MWMOTE and TGAN techniques. We have used Topological Node Feature(TNF) based spectral embedding for low dimensional data representation along with some popular embedding methods like MDS, Isomaps and t-SNE. After studying all the 65 possible combinations of these models, we have proposed an innovative method for effective automobile insurance fraud detection. For the given dataset, our results show that using a combination of MWMOTE as a data imbalance handling technique (Phase I), TNFSE2 as data embedding (Phase II) and Random Forest as classification (Phase III) provides the best result in comparison to all other combinations. This work also highlights the efficacy of TNF based spectral embedding in automobile insurance dataset

Rohan Yashraj Gupta, Lalith Srikanth Chintalapati, Satya Sai Mudigonda +2

Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning, Vidyagiri, Puttaparthi, Andhra Pradesh, India.

Financial Fraud Detection Spectral Methods

Sep 22, 2026cs.LG

Disentangling Heterogeneous Traffic Dynamics for Multi-Step Traffic Forecasting via Adaptive Spectral Decomposition

Accurate multi-step traffic forecasting remains challenging because observed traffic signals contain heterogeneous temporal dynamics with different characteristics and levels of predictability. Existing approaches typically model these dynamics within a unified representation or rely on predefined decomposition rules, which may limit their ability to flexibly separate persistent patterns from rapidly varying fluctuations. To address this issue, we propose the Adaptive Decomposition Network (ADNet), a component-specific forecasting framework that adaptively disentangles traffic dynamics into dominant and residual components. ADNet introduces a learnable complementary spectral decomposition mechanism that determines the contribution of each frequency bin to the two components. Unlike hard frequency partitioning, every frequency bin can contribute to both components with different learned proportions, allowing the decomposition to be optimized jointly with the forecasting objective. The reconstructed components are then modeled by two dedicated spatiotemporal forecasting branches, and their predictions are integrated to generate the final multi-step forecast. Experiments on the Alameda and Orange regions of the TraffiDent dataset show that ADNet achieves the best performance in 20 of the 24 reported region-horizon-metric comparisons, with particularly clear gains at longer forecasting horizons. Capacity-controlled ablation experiments further show that the learnable decomposition substantially outperforms a fixed decomposition and provides additional improvements beyond the dual-branch architecture alone. These results demonstrate the effectiveness of adaptive decomposition and component-specific modeling for multi-step traffic forecasting.

Zijun Huang, Chenrui Fu, Wenhao Wang +3

Beijing Normal-Hong Kong Baptist University · Dalian Maritime University · National Chung Hsing University

Time Series Decomposition Spatiotemporal Forecasting Spectral Methods Traffic Forecasting

Sep 21, 2026cs.LG

Topological Signal Processing With Unoriented Operators

Topological signal processing (TSP) processes signals on simplicial complexes with oriented boundary operators, which is the natural choice for flow signals or when the topological invariants play a role for the task at hand. However, many higher-order signals carry no orientation, and applying oriented operators to them is not well-defined since it introduces an arbitrary choice of simplex orientation. We study an unoriented TSP (UTSP) framework that replaces oriented boundaries with unoriented incidence matrices. First, we show that unoriented incidence and Laplacian matrices between arbitrary simplicial levels admit graph-like spectral properties. Second, since dropping orientation removes the Hodge decomposition, we introduce an unoriented counterpart, termed interaction-order decomposition, which quantifies how much of a higher-order signal is explained by aggregating lower-order signals. Third, we use this decomposition to derive regularizers for signal reconstruction that penalize each interaction order separately. Experiments on real-world data show that the order-aware regularizers outperform oriented baselines, with the largest gains when the signal energy is unevenly distributed across orders.

Andrea Cavallo, Varun Sarathchandran, Geert Leus +1

Delft University of Technology, Delft, Netherlands

Graph Signal Processing Spectral Methods Topological Data Analysis

Sep 20, 2026cs.CV

Training-Free Spectral Transductive Refinement for Cross-Domain Few-Shot Classification

Few-shot recognition with frozen visual features is especially fragile under domain shift and one-shot supervision, where a single labelled image is an unreliable estimate of its class. We ask how far this fragility can be reduced purely at test time, without retraining the encoder or augmenting the source domain. We present Spectral Transductive Refinement (STR), a training-free transductive inference rule that exploits the geometry of the complete support-query episode. Given frozen embeddings, STR builds a joint k-nearest-neighbour graph, maps the episode into a normalized-Laplacian spectral coordinate system, initializes class representatives from the labelled support, and iteratively refines them using pseudo-labelled queries. We evaluate STR under two protocols. A controlled component study with frozen ResNet-18 features shows that spectral refinement consistently improves over single-prototype spectral initialization across five shifted domains, with the largest gains in the one-shot regime where support estimates are weakest. We then benchmark STR against recent Cross-Domain Few-Shot Learning (CD-FSL) methods using the standard miniImageNet-pretrained ResNet-10 backbone over eight established target domains. Operating entirely at inference time, STR attains the highest 1-shot average among compared methods and remains competitive at 5-shot, rivalling approaches relying on heavy source-domain meta-training augmentations. Because STR is transductive, we report its setting explicitly. Diagnostics attribute its gains to iterative refinement in spectral coordinates rather than added prototype capacity, which remains inactive in our configuration.

Fahim Rahman, S. M. Tanjeeb Meheran Rohan, Md. Taimum Ibne Sayed +2

Department of Computer Science and Engineering Islamic University of Technology Gazipur, Bangladesh

Few-Shot Learning Spectral Methods Few-Shot Image Classification

Sep 16, 2026cs.LG

Provable Guarantees for Spectral Structured Prediction

Structured prediction is the simultaneous prediction of multiple labels, and is widely used in various fields, such as natural language processing and computer vision. In this paper, we study binary node label recovery on signed graphs with edge-flip noise, a model introduced by (Globerson et al., 2015), via a simple spectral method that decodes node labels from the signs of the principal eigenvector of the noisy signed adjacency matrix. We develop graph structure-agnostic theoretical guarantees for approximate inference of node labels as well as guarantees for maximum angle deviation with respect to the ground truth node labels. By leveraging tools from matrix concentration theory and eigenvector perturbation analysis, we derive new concentration inequalities that explicitly quantify the effect of the spectral gap of the adjacency matrix, number of nodes, degree distribution, and noise level. As a corollary, we relate our general results to the Cheeger constant and provide results for different classes of graphs. We perform several synthetic experiments to validate our theory. To the best of our knowledge, we are the first to provide theoretical guarantees for the spectral-based approach. As a byproduct of our analysis, we derive technical results that might be of independent interest and useful for other machine learning problems.

Violet Zheng, Jean Honorio

CIS, The University of Melbourne

Spectral Methods Node Classification Structured Prediction

Sep 14, 2026cs.RO

Geometry vs Structure: Graph-Based Diagnostics for LiDAR Point-Cloud Simulation Fidelity

Digital twins provide a scalable and cost-effective complement to real-world testing for validating autonomous-driving and advanced driver-assistance system (ADAS) sensor pipelines. However, quantifying their fidelity remains challenging, particularly for 3D LiDAR point clouds, where conventional geometric metrics may overlook important structural discrepancies. We present a graph-based framework for evaluating the structural fidelity of simulated LiDAR point clouds against real-world scans. While scan-level metrics such as Chamfer distance capture point-wise geometric similarity, they do not explicitly represent connectivity, topology, or object-level organization. Our framework constructs graphs from real and simulated point clouds, applies Louvain community detection to identify spatially coherent subgraphs, and matches corresponding communities using centroid proximity. For each matched pair, we compute

r_λ

, a bounded graph-spectral metric motivated by Weyl's inequality, and compare it with density-aware Chamfer distance (CDC) as a geometric baseline. Controlled perturbation experiments demonstrate that

r_λ

is invariant to rigid transformations and robust to sensor noise while remaining sensitive to structural deformation. We evaluate the framework on 50 paired real and simulated LiDAR scans acquired using a Velodyne VLP-32C sensor and CARLA, respectively. The dataset contains more than 1,000 matched communities across four representative classes: vehicles, vegetation, trees, and building walls. The results show that geometric and structural measures capture complementary aspects of simulation fidelity, supporting graph-spectral analysis as an additional diagnostic layer for validating digital twins in ADAS and autonomous-driving applications.

Ghazal Farhani, Taufiq Rahman

Automotive and Surface Transportation Research Centre, National Research Council Canada, London, Ontario, Canada.

Digital Twins Spectral Methods Autonomous Driving Simulation

Sep 14, 2026cs.LG

Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching

Flow-matching diffusion models have recently emerged as a strong paradigm for high-fidelity visual generation. However, their prohibitively high fine-tuning cost limits scalability to downstream tasks. While Low-Rank Adaptation (LoRA) combined with spectral initialization has demonstrated accelerated convergence and improved performance in autoregressive language models by better aligning gradient directions, we find that it fails to deliver similar gains in diffusion fine-tuning, often yielding marginal or even negative improvements over vanilla LoRA.We attribute this discrepancy to a fundamental mismatch between LoRA's low-rank parameterization and the intrinsically high-rank gradients induced by the flow-matching objective. In particular, stochastic timestep sampling introduces directionally heterogeneous gradient signals across training steps, leading to misaligned updates under low-rank constraints.To address this issue, we propose Prism-LoRA,a Principal-timestep Restricted Init via Sparse Matrix-decomposition framework that improves gradient alignment during fine-tuning. Our method consists of two key components: (i) principal timestep selection, which restricts initialization gradients to a subset of dominant timesteps to suppress effective gradient rank, and (ii) principal channel filtering, which removes task-irrelevant channels, enabling the one-step spectral initialization gradient to better align with the long-horizon optimization trajectory. Extensive experiments demonstrate that our method consistently improves both convergence speed and final performance across multiple diffusion fine-tuning benchmarks, including subject-driven generation, controllable generation, and deblurring, achieving not only performance improvement but also earlier stages of convergence over baseline LoRA and other spectral-init methods.

Jiayang Gu, Zheng Fang, Lichaun Xiang +3

University of Warwick · Bytedance

Flow Matching Diffusion Model Fine-Tuning Low-Rank Adaptation Spectral Methods

Sep 13, 2026cs.LG

Robust small-molecule identification from incomplete, degraded, and inconsistent spectra using multimodal mixed-condition training

Reliable small-molecule identification often requires complementary evidence from multiple spectroscopic measurements. In practice, however, spectra may be unavailable, degraded by measurement-related variations, or even incorrectly associated with a sample, thereby hindering accurate molecular identification. Herein, we propose a multimodal mixed-condition training strategy that accommodates missing, degraded, and mismatched measurements for small-molecule structure identification. The strategy incorporates chemical and spectroscopic knowledge through predefined missing-input configurations, modality-specific spectral perturbations, and chemically informed spectrum replacements. Models were trained on 635,441 samples comprising mass spectrometry (MS), infrared (IR), and nuclear magnetic resonance (NMR) simulated spectra from the Multimodal Spectroscopic Dataset (MSSD). They were then systematically evaluated on 79,462 held-out samples across 30 views designed to represent variations in spectra. A controlled comparison of complete-input and mixed-condition training under concatenation and mixture-of-experts (MoE) fusion showed that the training strategy was the principal source of improvement. For MoE, mixed-condition training increased the mean reciprocal rank (MRR) by 6.08% (from 0.9203 to 0.9763) and the top-1 molecular identification rate by 7.67% (from 89.50% to 96.36%). Notably, under single-modality inputs, IR MRR increased 2.15-fold (from 0.4337 to 0.9307), while MS MRR increased 2.31-fold (from 0.3711 to 0.8575). With the proposed strategy, complete-input performance remained high, while sample-level mismatch detection also improved. Together, these results highlight the potential of multimodal mixed-condition training for practical molecular identification by explicitly addressing incomplete, degraded, and mismatched measurements encountered in real-world analysis.

Bowen Gao, Lei Zhu, Yiying Wang +1

State Key Laboratory of Materials for Integrated Circuits, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, 865 Changning Road, Shanghai 200050, China

Missing-Modality Learning Multimodal Robustness Spectral Methods

Sep 11, 2026cs.LG

When does a spectral prior help graph learning? Connectivity-loss estimation under road-network disruptions

Rapid evaluation of many simultaneous road-link disruptions requires a practical compromise between exact spectral recomputation and local approximation. We estimate relative algebraic-connectivity loss after multi-edge deletion using graph neural networks (GNNs) that learn a bounded correction to a first-order Fiedler sensitivity. The study considers independent, spatially clustered, and edge-betweenness-targeted failures, with graph-disjoint synthetic splits and zero-shot transfer to 13 OpenStreetMap (OSM) areas in six countries. GCN, GraphSAGE, and edge-aware MPNN backbones are compared with analytical baselines. In expanded OSM tests, residual GCN improves spatial-failure MAE by 0.0391 (95% hierarchical interval 0.0151-0.0662), while residual GraphSAGE improves targeted-failure MAE by 0.0257 (0.0095-0.0446). Second-order perturbation improves first-order MAE by only 0.0028-0.0053. Correction slopes decrease under targeted transfer, indicating residual shrinkage around systematic prior error. Leave-one-country-out OSM-to-OSM transfer is mixed: residual GCN improves targeted-failure MAE by 0.0622 (0.0169-0.1153) but worsens the spatial point estimate. Sparse scaling extends to 20,000 nodes and separates one-time spectral setup from amortized screening cost. These results characterize the spectral residual as a useful but domain-sensitive inductive bias for structural connectivity screening. Code, cached networks, and reproducibility artifacts are archived at doi:10.5281/zenodo.22307723.

Van-Truong Le

Faculty of Information Technology, University of Science, Viet Nam National University Ho Chi Minh City, Viet Nam

Graph Neural Networks Spectral Methods

Sep 11, 2026quant-ph

Learning structural balance of graphs from quantum spectral features

We develop a quantum approach to spectral feature extraction from the density of states (DOS) of a problem-dependent Hamiltonian, and apply it to machine learning on signed graphs. We propose to embed a signed graph as an Ising model instance with positive and negative interactions, and use the standardized moments of the Ising DOS as features for learning. We show that these moments count signed closed walks, are switching-invariant, and are size-free by construction. As a benchmark, we target learning the frustration index, an NP-hard measure of structural balance that can be labeled exactly at moderate size. At zero field, the models can be sampled classically, allowing the quantum extraction procedure to be certified against exact ground truth. We propose DOS-QPE, a phase estimation on a purified maximally mixed probe, which samples the spectral density with orders of magnitude fewer shots than Hadamard test-based trace sampling and feeds the resulting features directly into classically trained models. On

1.4\times10^5

labeled graphs the exact DOS determines the frustration index, and five moments recover it with a mean error of 0.4, well below one sign flip. Beyond zero field, the underlying trace-estimation problem is DQC1-complete, providing access to spectral features for which no efficient classical sampling method is known. Our work opens routes towards quantum applications in social network balance analysis, spin-glass studies, correlation clustering, and protein-interaction networks.

Stefano Scali, Oleksandr Kyriienko

Department of Physics and Astronomy, University of Exeter, Exeter EX4 4QL, UK · School of Mathematical and Physical Sciences, University of Sheffield, Sheffield S10 2TN, UK

Quantum Machine Learning Spectral Methods

Sep 10, 2026cs.LG

Predicting Privacy Leakage from Weight Spectral Density

Membership inference attacks (MIAs) are widely used to audit the privacy disclosure risk of machine learning models, however current state-of-the-art attacks require training computationally expensive shadow models, making large-scale privacy evaluation impractical. In this work, we investigate whether inexpensive spectral metrics derived from the heavy-tailed self-regularisation framework can serve as proxies for MIA vulnerability. We evaluate several WeightWatcher spectral metrics on image and tabular classification tasks and compare their relationship with MIA privacy leakage against conventional measures of generalisation. Across datasets, stable rank exhibits a strong positive correlation with overall MIA success, while Log alpha-Norm shows a consistent negative correlation with MIA vulnerability at the low false-positive regime. These associations are observed to be stronger than those obtained using the generalisation gap. The results indicate that neural network spectra may contain information about privacy leakage that is not fully captured by conventional measures of overfitting, motivating spectral analysis as a promising direction for scalable privacy auditing.

Richard J. Preen, Jim Smith

Department of Computer Science and Creative Technologies University of the West of England Bristol, UK BS16 1QY

Privacy Auditing Membership Inference Attacks Spectral Methods

Sep 8, 2026math.ST

MiNCE: Nonparametric, Strongly Consistent Confidence Envelopes for Band-Limited Functions and their Smoothed Spectra

Minimum-norm confidence envelope strategies offer a nonparametric approach to constructing nonasymptotic, simultaneous confidence regions for band-limited functions, exploiting the theory of Reproducing Kernel Hilbert Spaces (RKHS). While the finite-sample coverage guarantees of these envelopes have been established, their consistency has not been analyzed so far. In this paper, we study this construction, here termed the Minimum-Norm Confidence Envelope (MiNCE) framework, and establish the strong uniform consistency of the resulting bands, both for noise-free and noisy observation models, under mild assumptions on the measurement noises. We further extend this formulation to the frequency domain, deriving nonasymptotic, simultaneous, strongly uniformly consistent confidence bands for the smoothed spectra. Numerical experiments in nonparametric regression and spectral estimation empirically confirm our theoretical results, illustrating the contraction of the confidence envelopes toward the target function as the sample size increases.

Balázs Csanád Csáji, Bálint Horváth

HUN-REN SZTAKI: Institute for Computer Science and Control, Budapest, Hungary · Institute of Mathematics, Eötvös Loránd University (ELTE), Budapest, Hungary

Confidence Region Estimation Nonparametric Regression Kernel Methods Spectral Methods

Sep 3, 2026cs.LG

Geometry-Aware Graph Construction via Adaptive Spectral Bandwidth Control

Kernelized graph methods - spectral clustering, diffusion maps, and sparse kernel -regression graphs - that use Gaussian kernels depend on the choice of Gaussian bandwidth sigma, which governs the spectral character of the local kernel operator. When sigma is too small, the kernel overestimates local complexity and treats each sample as an independent direction; when sigma is too large, the kernel collapses multiple directions together, the condition number diverges, and all geometric discrimination is lost. We propose a choice of scale to make the spectral complexity of the kernel consistent with the intrinsic complexity of the underlying manifold. We propose a per-node bandwidth criterion that operationalizes this principle by jointly matching the kernel's effective rank to the local intrinsic dimension estimated via minimum spanning tree, anchoring the search in the manifold-consistent log-log scaling regime. We evaluate SSL embeddings from six encoders on CIFAR-100, showing that adaptive bandwidth consistently improves leave-one-out (LOO) classification and label propagation (LP) accuracy over fixed-bandwidth methods and competing adaptive methods.

Ecem Bozkurt, Antonio Ortega

Department of Electrical and Computer Engineering University of Southern California, Los Angeles, CA, USA

Graph Structure Learning Graph-Based Semi-Supervised Learning Kernel Methods Spectral Methods

Sep 3, 2026math.RT

What is Smoothness?

Smoothness of a function on the real line is reflected in the decay of its Fourier transform, which suggests that smoothness of a function in

L^2(G)

for a group

G

should mean concentration of the Fourier coefficients at low frequency. Such a reading presupposes an ordering of the irreducible representations of

G

, but for non-abelian

G

, no ordering is canonical. Given a symmetric generating set

S

, the Laplacian of the associated Cayley graph is block diagonal over the dual, and we order the irreps by the mean of the eigenvalues in each block. This produces an ordering function

ω:\widehat{G}\to\mathbb{R}

that depends only on the pair

(G,S)

. This function is bounded between zero and two, vanishing only at the trivial representation and achieving the upper bound exactly when the Cayley graph is bipartite. We then ask how much freedom the construction has. Within the class of operators satisfying natural axioms, the induced orderings are exactly the real functions on the dual vanishing at the trivial representation and agreeing on conjugate pairs, and the orderings coming from inversion orbits of conjugacy classes form a basis for them. We cut the freedom down further by requiring two additional inputs: nonnegativity of the class weights and a declaration of which group elements count as uniform incremental changes, which pins the operator to the Cayley-Laplacian up to positive scale. We observe that the construction persists for compact groups even though the Cayley graph does not, and we extend the theory to finite sets carrying a transitive group action, where the acting group selects which frequencies exist and the generating set orders them. The answer to the title question is therefore that smoothness is a property of a function together with a choice of group and generating set, not of the function alone.

Zachary P Bradshaw

QodeX Quantum

Spectral Methods

Sep 2, 2026cs.LG

A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) commonly evaluate the spatial derivatives appearing in partial differential equation residuals using automatic differentiation (AD), whose computational and memory costs can become substantial when multiple or high-order derivatives are required. We perform a controlled comparison of spatial AD and Fourier spectral differentiation in periodic physical-space PINNs. Within each paired experiment, the neural representation, temporal differentiation, optimizer, sampling procedure, and training schedule are held fixed, so that the two cases differ only in the spatial differentiation procedure. For the Fourier variant, network outputs are evaluated on a uniform periodic grid and transformed to Fourier space, where spatial derivatives are obtained through spectral multiplication and the same Fourier coefficients are reused across derivative orders. We compare the two procedures in standard PINNs for the Allen--Cahn and Korteweg--de Vries equations and in Causal PINNs for the Allen--Cahn, Korteweg--de Vries, and Kuramoto--Sivashinsky equations. Across these five equation--framework settings, Fourier differentiation yields mean paired end-to-end training speedups ranging from

2.90\times

to

18.52\times

and reduces peak allocated graphics processing unit (GPU) memory by

68.7\%

--

94.1\%

. The final relative

L_2

errors remain of the same order, with neither differentiation procedure showing a consistent accuracy advantage. For the one-dimensional periodic benchmarks considered here, Fourier spectral differentiation therefore provides substantially lower training time and memory usage than spatial AD while retaining comparable solution error, at the cost of requiring a uniform structured spatial grid.

Xilai Liang, Zhao Zhang

Guangdong Technion–Israel Institute of Technology, Shantou, Guangdong, China · Research Centre for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong Province, 266237, China

Physics-Informed ML Automatic Differentiation Spectral Methods

Sep 2, 2026cs.CL

Selective Knowledge Edit Reversal via Gated Singular Vector Shrinkage

Knowledge editing provides an efficient way to update factual knowledge in large language models. However, malicious edits may introduce safety risks, making it necessary to reverse undesirable editing effects. Existing reversal methods for parameter-modifying edits mainly focus on global removal, which may also erase beneficial edits that should be preserved. In this paper, we study selective reversal of edited knowledge, where the goal is to reverse targeted edited facts while preserving the remaining edited facts. Based on the hypothesis that each edit is sparsely encoded within the dominant subspace of the edited matrix, we propose a spectral-based reversal framework that locates edit-sensitive components within the dominant singular subspace of edited weights. Experiments across multiple settings demonstrate the effectiveness of our method in reversing selected edits while preserving unrelated edited facts. These results suggest that different edits are sparsely encoded within dominant singular components and can be separable when the number of edits is moderate, making selective spectral reversal a promising direction for locating edit-specific components and repairing edited language models.

Weifeng Jiang, Ruirui Chen, Qianren Mao +3

College of Computing and Data Science, Nanyang Technological University, Singapore. · Digital Trust Centre, Nanyang Technological University, Singapore. · Institute of Advanced Intelligence and Computing, Agency for Science, Technology and Research, Singapore. +2

Knowledge Editing LLM Safety Spectral Methods

Aug 30, 2026cs.LG

Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains

Learning solution operators for partial differential equations (PDEs) on irregular and geometry-dependent domains remains a central challenge in scientific machine learning. While spectral methods provide strong inductive biases for modeling global interactions, they are typically limited to regular domains, and existing neural approaches often require domain warping, interpolation, or costly geometric embeddings. We introduce the \textbf{Graph Spectral Neural Operator (GSNO)}, a neural operator that combines spatial graph spectral decompositions with temporal Fourier transforms through a unified space--time spectral kernel. This formulation enables globally coherent operator learning on non-Cartesian discretizations without domain warping or autoregressive rollouts. By replacing learned geometric embeddings with a graph Laplacian spectral basis, GSNO provides geometry-aware spectral learning with low parameter complexity. Across steady and unsteady PDE benchmarks on irregular and geometry-dependent domains, GSNO achieves strong accuracy with reduced runtime and parameter counts, while demonstrating robust zero-shot generalization across mesh resolutions and geometry families.

Abdolmehdi Behroozi, Chaopeng Shen

Department of Civil and Environmental Engineering, Penn State University, University Park, PA 16802, USA

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Neural Operators

Aug 27, 2026cs.LG

Cone Rayleigh Levels: Finite Perturbations and Certified Control

We study the reuse of positive trial profiles under finite perturbations of nonsymmetric matrix pencils B-λG. The lower and upper cone Rayleigh levels need not coincide and are defined without requiring positive eigenvectors. In the positive orthant with positive diagonal

G

, we derive computable perturbation bounds and determine the exact worst-case trial gap over prescribed independent entrywise perturbation classes. This yields the largest uniform radius meeting a given trial-gap tolerance for fixed profiles, certifying trial-value accuracy without recomputation. Experiments on a nonnegative operator and five signed matrices compare sufficient and optimal radii, directional thresholds, and cold- and warm-start recomputation. Several signed cases exhibit severely limited uniform profile reuse despite the optimality of the radius. Exact rational checks verify the reported bounds and worst-case constructions for stored numerical inputs. A finite-budget linear program and differentiable constraints illustrate applications to verified control and learning.

Yavdat Sh. Il'yasov, Nur F. Valeev

Institute of Mathematics with Computing Centre, Ufa Federal Research Centre of the Russian Academy of Sciences, 112 Chernyshevsky St., Ufa 450008, Russia.

Robust Optimization Spectral Methods

Aug 27, 2026eess.SY

Data-driven Koopman mode approximation: A neural power iteration algorithm

This paper proposes a novel data-driven algorithm to approximate the dominant eigenfunctions (aka.~modes) of the Koopman operator of nonlinear dynamical systems using neural networks. The relevance of learning the dominant Koopman modes is to approximate nonlinear dynamics by linear ones in a lifted space, thereby enabling simplified control and analysis. To fight the curse of dimensionality arising from using expressive templates (here neural networks) for the mode approximation, the proposed method leverages a power-iteration scheme that directly learns the dominant Koopman modes without explicitly constructing the projection of the Koopman operator on the template of functions. Our approach connects to other approaches in the literature that avoid the curse of dimensionality by learning small dictionaries of functions, but differs from them in that we do not require ``anti-collapse mechanisms'' to ensure that the learned dictionary is expressive enough to approximate the Koopman operator since our power-iteration scheme is designed to converge toward the dominant modes of the projected Koopman operator. The approach is fully data-driven, requiring only sampled state transitions. Theoretical guarantees are provided, showing convergence under increasing sample size and network width (in connection with the neural tangent kernel theorem). Numerical experiments demonstrate that the method achieves accurate and smooth approximations of dominant modes while avoiding the limitations of traditional techniques such as extended dynamic mode decomposition.

Guillaume O. Berger, Raphaël M. Jungers

ICTEAM, UCLouvain, Louvain-la-Neuve, Belgium

Koopman Operator Learning Spectral Methods

Aug 20, 2026cs.LG

Decision Tree and K-Means Analysis of Raman Spectra for Edible Oils: A Physics-Informed AI Approach

Classification of edible oils in processed foods is important for food quality, fraud prevention, and regulatory compliance. This study develops a Mutually Exclusive, Collectively Exhaustive framework integrating spectral organization, interpretable classification, Physics-Informed Artificial Intelligence (PI-AI), and Frugal AI-based feature reduction. Five edible oils were analyzed in pure form and within a fried-potato-chip matrix using t-SNE, K-means clustering, Decision Trees, and Non-Negative Least Squares (NNLS)-based spectral decomposition. Unsupervised analyses showed stronger class organization and separability in pure oils, while food-matrix effects caused substantial spectral overlap. Decision Trees achieved 100% classification accuracy for pure oils using only four Raman variables from 1866 spectral features. These variables represented only 0.21% of the available spectral information while retaining perfect test-set performance. Two variables associated with lipid unsaturation (about 1650 cm-1) and hydrocarbon-chain organization (about 1127 cm-1) remained important after NNLS matrix correction. Their combined contribution increased from 50% in pure oils to about 62% and 89% in paper-subtracted and paper-plus-potato-subtracted datasets, respectively. NNLS-based PI-AI improved food-matrix classification by separating oil signatures from paper and potato contributions. Optimized post-pruned models achieved nearly 80% test accuracy using only five and four Raman variables, respectively. The four-feature representation reduced the data footprint by 99.44% without loss of accuracy. These findings demonstrate that Raman-based oil identification can use compact, physically meaningful, and interpretable spectral representations, supporting Frugal AI, Edge AI, portable sensing, and embedded food-quality monitoring.

Amrita Shaw, Chandrasekar S. N., Sai Muthukumar V. +2

Department of Food and Nutritional Sciences, Sri Sathya Sai Institute of Higher Learning, Vidyagiri, Prasanthi Nilayam, Sri Sathya Sai District, Andhra Pradesh, India – 515134. · Aix-Marseille University, Marseille, 13007, France. · Department of Physics, Sri Sathya Sai Institute of Higher Learning, Vidyagiri, Prasanthi Nilayam, Sri Sathya Sai District, Andhra Pradesh, India – 515134. +1

Physics-Informed ML Raman Spectroscopy Spectral Methods

Aug 14, 2026cs.AI

Attributing Preprocessing Invariance in Spectral Foundation Models

A spectral foundation model should remain useful when laboratories preprocess spectra differently. The standard test trains a classifier under one pipeline and evaluates under another, taking preserved accuracy as evidence of learned invariance. However, these models normalize each input before any learned parameter is applied. When normalization maps differently preprocessed spectra to the same vector, the encoder receives identical inputs and the measured invariance cannot be attributed to learning. We propose a normalization-only attribution control: compare the encoder against its normalization before interpreting transfer as learned invariance. On six Raman datasets the encoder does not measurably improve transfer over its normalization. A controlled experiment confirms invariance develops only when variation reaches the encoder past normalization. Across three systems, no encoder improves relative retention over its normalization. An audit of eighteen configurations across five modalities confirms the issue is widespread: the normalization-only control should be reported before crediting transfer to the encoder.

Dongjun Wei, Hongyi Wu

ESCP Business School, Paris · OpluxCare Tech Group Ltd. · The University of Hong Kong, Hong Kong

Transfer Learning Raman Spectroscopy Spectral Methods Invariant Representation Learning

Aug 11, 2026cs.SI

Spectral graph clustering with inhomogeneous latent geometry

We study spectral clustering in the presence of a confounding latent geometry. The leading eigenvectors may then be dominated by the latent geometry rather than by the communities. Nevertheless, we show in a block latent-space model that communities can be recovered from eigenvectors deeper in the spectrum. We analyze the spectral properties of the adjacency matrix through a limiting integral operator and use its structure to develop DBSPEC, a density-based spectral clustering algorithm that requires only approximate localization of the informative eigenvalue and is robust to poor eigenvalue separation. Crucially, this approach handles general latent geometries, overcoming restrictions to homogeneous toroidal models in prior works. Our theoretical predictions for the location of the informative eigenvalue notably align with observations in real-world experiments.

Konstantin Avrachenkov, Lucas S. Sibemberg, Alexander Van Werde

Inria Sophia Antipolis, France · Federal University of Rio Grande do Sul, Brazil · University of Münster, Germany

Spectral Clustering Graph Clustering Spectral Methods

Aug 11, 2026cs.RO

When Your State Estimator Has Lost The Plot: Detecting Estimator Failures Via Spectral Analysis

Reliable onboard state estimation is essential for safe robotic operation, yet unmodeled disturbances, such as sensor aliasing or out-of-distribution noise, still cause estimators to degrade or fail completely. While many methods aim to improve estimator robustness, only a few provide introspective mechanisms to assess estimate quality. Existing uncertainty measures, such as covariances, rely on idealized assumptions and tend to be overconfident, and more recent data-driven approaches are typically tied to their training data distributions. We propose a sensor-agnostic introspective method that assesses estimator health by analyzing the frequency-domain power distribution of recent velocity estimates. The method is evaluated using outdoor flight data from an aerial robot running visual-inertial, LiDAR-inertial, and radar-inertial odometry. The dataset includes multiple estimator failures, enabling analysis of several frequency-domain indicators, such as signal power, spectral bandwidth, and entropy. We observe consistent spectral power differences between healthy and degraded estimates, allowing detection of 51%-58% of labeled failures with 60%-84% precision across three fundamentally different state estimation frameworks. Our results show that even a simple frequency-domain analysis of a state estimator's output can serve as a lightweight introspective tool to complement existing robustness techniques in real-world robotic deployments, and opens promising avenues for future investigation.

Christian Lanegger, Helen Oleynikova, Roland Siegwart +1

Autonomous Systems Lab, ETH Zurich, Switzerland · Exclaim Robotics, Zurich, Switzerland

Spectral Methods Robot State Estimation Fault Diagnosis

Aug 10, 2026cs.IR

DualSpectralCF: Training-Free Sign-Aware Spectral Collaborative Filtering

Real-world recommendation platforms routinely collect explicit negative feedback such as 1-star reviews, hate-button clicks, distrust between users, and very-low watch-ratio videos. Learned sign-aware recommenders exploit this signal for clear accuracy gains, but only at the cost of gradient-based training. In parallel, a line of training-free spectral collaborative filtering methods matches or beats learned graph recommenders at a fraction of the cost, yet operates on positive interactions alone. We bridge these two lines with DualSpectralCF, a training-free framework of two components that attach to any spectral backbone of the form

\hat{\mathbf{r}}_u = F(\mathbf{M}) \mathbf{r}_u

: a signed input signal

\mathbf{r}_u^{\pm}

that encodes the user's explicit dislikes, and a signed item-item operator

\mathbf{M}^{\pm}

that blends like-together and dislike-together similarity. The framework is backbone-agnostic and adds just two scalar hyperparameters. We instantiate DualSpectralCF on ChebyCF, GF-CF, and Turbo-CF, and evaluate on five sign-aware benchmarks: every instance matches or beats its unsigned backbone on all 5 datasets, with Recall@20 lifts up to +32.6% with backbone-specific

(γ, κ)

tuning and +1.9% to +16.0% for DualSpectralCF-Cheby at the fixed default

(γ= -0.5, κ= 0.1)

, and the family runs 7.7 to 155.3

\times

faster than SIGformer while reaching 70.7% to 90.7% of its accuracy. Sign-awareness helps most for cold-start users, with up to +29.2% Recall@20 on Epinions users with 1 to 5 training items.

Guanqun Yang, Tong Qi, Xiaoxue Han

Stevens Institute of Technology Hoboken, NJ, USA · University of Maryland, College Park College Park, MD, USA

Recommender Systems Spectral Methods Collaborative Filtering

Aug 9, 2026stat.ML

Multi-kernel spectral clustering: Entrywise eigenvector perturbation bounds and exact recovery

Kernel spectral clustering with a single bandwidth can be inadequate for data exhibiting multiple characteristic pairwise-distance scales, a problem particularly prevalent in the high-dimensional regime. We address this issue through a multi-kernel formulation that aggregates kernels with different bandwidths. The bandwidths are selected as prescribed empirical quantiles of the pairwise squared distances, thereby capturing the relevant distance scales without requiring prior population-scale information. We develop a rigorous theoretical analysis of the resulting method under a general high-dimensional, multi-scale mixture model with heterogeneous cluster centers and covariance geometries. We construct a blockwise constant, low-rank informative approximation to the empirical multi-kernel matrix and establish row-wise

\ell_{2,\infty}

perturbation bounds for its leading spectral components, as well as for the associated normalized Laplacian matrix. These bounds yield observation-level control of the spectral embedding, which is more informative than conventional global eigenspace perturbation estimates. Under suitable eigen-gap and cluster-separation conditions, we show that approximate

K

-means applied to the multi-kernel spectral embedding achieves exact recovery with high probability.

Zeqin Lin, Guangming Pan, Zhixiang Zhang +1

1Nanyang Technological University · University of Macau · 3Dalian University of Technology

Clustering Unsupervised Clustering Spectral Clustering Kernel Methods Spectral Methods

Aug 7, 2026cs.AI

From Points to Edges: Edge-Conditioned Spectral Operators for Physics-Sensitive PDE Learning

Neural operators have become a central tool for solving partial differential equations (PDEs), with spectral operators offering efficient global mixing across spatial locations. However, many PDEs contain physics-sensitive local structures that are critical to the underlying physical behavior. For example, in Darcy flow, local material interfaces are often reflected by sharp changes in the permeability field and can strongly influence the solution. Existing spectral operators primarily adapt modal mixing based on center-point representations, making them insufficiently responsive to such localized structural variations. We propose the Edge-Conditioned Spectral Operator (ESO), a novel spectral operator framework that modulates global spectral mixing using local edge-wise variations. By incorporating the Pairwise-Variation Modal Mixer (PVMM) to inject local edge information into spectral mode selection, ESO preserves the global approximation capability of spectral neural operators while enabling the learned kernel to adapt to physics-sensitive local structures. Furthermore, we introduce a task-adaptive Physics-Aware Reweighting (PAR) that emphasizes physically important regions, identified by taskspecific physical quantities. Across nine PDE benchmarks, ESO consistently achieves state-of-the-art performance. Visual and region-wise analyses further demonstrate that ESO reduces solution errors near coefficient jumps, high-gradient flow structures, and other physically sensitive regions. The code is available at https://github.com/Tanpig-X/ESO.

Zhentao Tan, Ruijie Quan, Yi Yang

Collaborative Innovation Center of Artificial Intelligence (CCAI), Zhejiang University, China

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Neural Operators

Aug 6, 2026cs.SD

Explicit and Stable Pseudospectral Time-Domain Method for the Föppl-von Kármán Equations

Modal synthesis is a widely-used technique for simulation of musical instrument dynamics. In the linear case, a modal decomposition leads to an uncoupled system of damped and forced harmonic oscillators which can be efficiently solved by standard time-stepping methods. However, extensions to nonlinear problems are challenging due to the presence of products of modal expansions in the governing equations. In the case of the Föppl-von Kármán plate, the nonlinear coupling between the modes is described by a fourth-order tensor and is prohibitively expensive to evaluate in the modal domain. In this work, we propose a pseudospectral method in which the products are evaluated on a grid in the spatial domain while spatial derivatives are computed exactly in the modal domain. Discrete sine and cosine transforms between the modal and spatial domains are used to impose simply supported boundary conditions for the plate. Finally, we prove non-negativity of the nonlinear potential energy of the system and employ a scalar auxiliary variable technique for explicit and stable time integration in the modal domain. As a result, we reduce the computational cost of modal synthesis while preserving its advantages like a precise control over the simulated frequency range. Sound examples are presented.

Victor Zheleznov, Stefan Bilbao

Acoustics and Audio Group, University of Edinburgh, Edinburgh, UK · STMS (UMR9912), IRCAM, CNRS, Sorbonne Universit´e, Paris, France

PDE Solving Structural Mechanics Spectral Methods Physics-Based Simulation

Aug 5, 2026cs.LG

Spectral Distillation: From Nonlinear Dynamics to Linear State-Space Models

Can nonlinear dynamical systems be learned through a compact linear state-space representation, without directly solving a non-convex system-identification problem? We give a provable pipeline for doing so. Starting from observations of an unknown nonlinear dynamical system, we first learn an implicit spectral predictor using Observation Spectral Filtering (OSF), a convex method that competes with the best linear observer for the system. We then apply spectral-to-LDS distillation to convert this predictor into an explicit recurrent linear dynamical system. Our main theorem shows that the average prediction error of the distilled LDS decomposes into an exponentially-small distillation term and the OSF learning term governed by the Luenberger complexity of the best observer. The guarantee is dimension-free: it depends on observer complexity rather than on the latent dimension needed to represent the nonlinear system. To our knowledge, this yields the first end-to-end provable method for extracting a best-in-hindsight LDS representation of nonlinear dynamics through convex learning followed by provable distillation. Experiments on linear LDS benchmarks and MuJoCo behavior cloning show that the train-then-distill pipeline produces compact LDS predictors that match or outperform directly trained baselines.

Liane Galanti, Devan Shah, Shlomo Fortgang +1

Princeton University

Dynamical Systems Nonlinear System Identification Spectral Methods Knowledge Distillation

Aug 5, 2026physics.chem-ph

Physics-Based Molecular Fingerprints from Spectral Graph Theory Provide Efficient Geometry-Aware Measures of Chemical Similarity

Molecular representations are essential for the evaluation of molecular similarity and the development of structure-property relationships. Despite the known importance of 3D structure to determine chemical and physical properties, the most widely used molecular fingerprints encode only two-dimensional connectivity. Such representations fail to distinguish similar but distinct stereoisomers and conformers. Alternative 3D methods are typically defined pairwise, making their application to large chemical spaces prohibitive, while deep learning embeddings are expressive but uninterpretable and limited by their training data diversity. Here, we introduce novel physics-inspired molecular fingerprints based on principles from spectral graph theory. We represent molecules as a complete graph in 3D space, with edge weights encoding heuristic physical interactions. Eigenvalue decomposition of the resulting graph Laplacian matrix results in a computationally efficient fixed-length chemical fingerprint that encodes 3D structure while obeying necessary physical symmetries of permutation and E(3) invariance. Spectral fingerprints differentiate between unique molecular structures with identical 2D connectivity, overcoming a limitation of 2D descriptors, while maintaining the low computational cost needed for efficient screening of vast chemical spaces. We evaluate our fingerprints with community detection algorithms and observe strong performance against representative baselines across datasets from organic, inorganic, biological, reticular, and reaction chemistry. Nearest-neighbor property estimation and applicability domain analyses reveal the utility of our molecular representation in machine learning and cheminformatics. We anticipate that spectral fingerprints will serve as generalizable, interpretable, and efficient measures of chemical similarity that incorporate 3D information at minimal cost.

Jacob W. Toney, Ayleen Y. Farnood, Samir Darouich +1

Department of Chemical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Institute for Theoretical Chemistry, University of Stuttgart, 70569 Stuttgart, Germany +2

3D Representation Learning Spectral Methods Molecular Representation Learning

Aug 4, 2026cs.LG

Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices

For

n

unit vectors

x_1,\ldots,x_n \in \mathbb{R}^d

, we study the continuous ReLU derivative Gram matrix

H

, whose entries are obtained by averaging pairwise gated inner products over a standard Gaussian direction. Writing

Δ_\pm := \min_{i \neq j} \min\{ \|x_i-x_j\|_2, \|x_i+x_j\|_2 \}

for their projective separation, we prove the universal dimension-free lower bound

λ_{\min}(H) = Ω( Δ_\pm/\sqrt{\log n} )

. Conversely, we construct worst-case families satisfying the matching upper bound

λ_{\min}(H) = O( Δ_\pm/\sqrt{\log n} )

, showing that this rate is tight up to universal constants.

Zhao Song

ReLU Neural Networks Spectral Methods Neural Tangent Kernel

Aug 4, 2026cs.LG

Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon

Matrix spectral optimizers reshape weight-update spectra but usually delegate vector-valued biases to a separate optimizer. We study whether this separation is neutral. We formulate each affine layer as a joint momentum matrix

A=[M_W,αm_b]

and apply a capped regularized-inverse spectral map to the complete matrix, producing both the weight and physical bias updates. A strict five-seed ablation on a four-layer BERT-mini trained from scratch on IMDb compares exact-SVD Muon, weight-only inverse shaping, affine-probe inverse shaping, and the proposed joint regularized inverse (JRI). Weight-only inverse shaping raises validation-loss-selected test accuracy from

84.903\pm0.242\%

to

85.562\pm0.308\%

and lowers selected test loss from

0.3479

to

0.3345

. Allowing bias to alter the joint SVD while retaining an independent Adam bias update does not improve over weight-only inverse shaping. Using the transformed bias jointly raises selected test accuracy to

85.738\pm0.180\%

and lowers test loss to

0.3291

, with all five seeds improving relative to the probe baseline. During the peak-performance window, JRI preserves the eligible weight-update norm while reducing the bias-update norm from

0.02095

to

0.00301

, lowers boundary-function share from

86.58\%

to

78.97\%

, and changes the cosine between weight-induced boundary motion and explicit bias from

+0.030

to

-0.137

. An independent 22-seed replication yields

85.743\pm0.203\%

selected test accuracy. These results identify joint affine spectral allocation as a small but consistent extension to weight-only spectral optimization.

Gongyue Zhang, Honghai Liu

State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China.

Neural Network Optimization Muon Optimizer Matrix Optimization Spectral Methods

Aug 3, 2026cs.LG

RamanPFN: learning from Raman spectral structure with a tabular foundation model

Raman spectroscopy enables label-free molecular characterization across materials science, analytical chemistry, biomedicine, and industrial process monitoring. However, machine learning for high-dimensional spectroscopy remains constrained by limited labelled data and a mismatch between the physical organization of spectra and feature-agnostic models. Channel coverage alone does not ensure that related bands share a common inference context. Here we present RamanPFN, a general-purpose spectral foundation framework that enables unified in-context inference through physics-guided spectral learning. It captures full-spectrum compositional covariation via Global Compositional Unmixing (GCU), which decomposes distributed, multi-band mixture signatures into shared non-negative latent bases. Simultaneously, it resolves local vibrational structure through Local Vibrational Subspace Encoding (LVSE), which preserves fine-grained peak morphology, intensity fluctuations, and peak shifts within contiguous spectral neighborhoods. Extensive evaluation across 74 diverse public Raman datasets covered 129 regression targets and was further extended to 21 classification tasks. RamanPFN achieved state-of-the-art performance across all reported aggregate metrics against 28 independently reproduced methods spanning chemometrics, spectral neural networks, deep tabular learners and tabular foundation models. RamanPFN establishes a physics-guided paradigm for scientific spectroscopy, enabling data-efficient predictive learning across diverse chemical systems.

Xingyu Pan, Huan Wang, Jinjia Guo +3

Beihang University · Cleer Science

Tabular Foundation Models Representation Learning Physics-Informed ML Raman Spectroscopy Spectral Methods

Aug 2, 2026eess.SP

Sheaf-theoretic Signal Processing on Graphs: Spectral Theory, Filtering, and Sampling

Modern sensing, communication, and learning systems generate heterogeneous network signals, with local data differing in dimension, modality, and geometric structure. Processing such data requires a mathematical framework capable of simultaneously modeling heterogeneous local signal spaces and the transformations relating them. Network sheaves provide such a framework by associating local vector spaces with network entities and linear restriction maps with their interactions. This is the first paper to develop a unified sheaf signal processing (SSP) framework on network sheaves, extending the fundamental operations of signal processing, namely spectral analysis, filtering, and sampling, to heterogeneous local spaces. Unlike graph and topological signal processing, where signals are modeled over a common vector space, SSP jointly models heterogeneous local signal spaces and the linear transformations relating neighboring spaces through restriction maps. We define the Sheaf Fourier Transform (SFT), whose frequencies quantify signal inconsistency induced by the network topology, the restriction maps, and the local geometry. Building on this representation, we develop polynomial sheaf filters and formulate sampling as the joint selection of network nodes and intra-node components. We derive perfect recovery conditions for bandlimited sheaf signals and propose a greedy sampling-set design algorithm. To incorporate application-dependent signal models, including different bases, dictionaries, and learned embeddings, we introduce representation sheaves and characterize the natural transformations that preserve spectral properties and guarantee interoperability across representations. Experiments on synthetic, motion-capture, and financial datasets validate the proposed framework and demonstrate consistent improvements over canonical graph signal processing baselines.

Gabriele D'Acunto, Leonardo Di Nino, Paolo Di Lorenzo +1

Department of Information Engineering, Electronics, and Telecommunications, Sapienza University of Rome, 00184 Rome, Italy · National Inter-University Consortium for Telecommunications (CNIT), Parma, Italy

Graph Signal Processing Sheaf Theory Spectral Methods

Jul 30, 2026physics.data-an

Rethinking Total Absorption Gamma Spectroscopy Deconvolution: Supervised Machine Learning vs Response-Matrix Methods

The extraction of

β

-feeding distributions in Total Absorption

γ

-ray Spectroscopy constitutes a challenging inverse problem, particularly in nuclei with complex decay schemes involving a large number of excited states. In such cases, the measured spectrum arises from the superposition of many detector response functions, making the determination of the individual feedings intrinsically ill-posed and highly sensitive to the methodology employed. In this work, we present a systematic comparison between supervised Machine-Learning techniques and Response-Matrix methods using realistic Monte Carlo simulations of an experimental Total Absorption Spectrometer. Supervised Machine-Learning approaches construct a non-parametric estimator that infers level feedings from the measured spectrum after a training stage, whereas Response-Matrix methods determine the feeding distribution by directly minimizing the difference between measured and reconstructed spectra. Our results show that supervised Machine-Learning techniques achieve superior accuracy in the reconstruction of individual feeding intensities, whereas Response-Matrix methods provide robust and physically consistent initial solutions. These findings support a hybrid strategy in which a Response-Matrix method is first used to obtain an initial feeding estimate, which is then refined using a supervised Machine-Learning approach to achieve improved overall accuracy.

J. Balibrea-Correa, E. N{á}cher, C. Fonseca-Vargas +1

Instituto de Física Corpuscular, CSIC - Universidad de Valencia, Spain, 46980, Valencia, Spain

Spectral Methods Inverse Problems

Jul 30, 2026cs.AI

Dynamic Spectral Filtering for Temporal Graph Learning: Learning Evolving Propagation Operators

Temporal graph learning is commonly organized around the evolution of node states or the encoding of interaction histories. We study an underexplored, operator-centric question: should the graph propagation mechanism itself evolve over time? We introduce Dynamic Spectral Filtering (DSF), which represents propagation at snapshot t by a Chebyshev polynomial filter with vector-valued, time-dependent coefficients. DSF explicitly treats these compact multi-order coefficients as recurrent temporal states. A recurrent branch proposes updates, while multiplicative global and order-specific gates regulate their magnitude. The temporal state is independent of the number of nodes. On MOOC, Wikipedia, and Reddit temporal link-prediction benchmarks, converged DSF runs attain AP scores of 0.7851, 0.9088, and 0.9860, respectively, with 93K to 133K trainable parameters, 68 to 182 MB peak GPU memory, and 1.6 to 2.1 seconds of training per epoch. Against the closely related DEFT baseline, DSF is better on MOOC, within 0.001 AP on Reddit, and modestly lower on Wikipedia, while using 8.3 to 8.6 times fewer parameters, 25 to 33 times less GPU memory, and 5 to 19 times less time per epoch. Relative to all measured alternatives, it uses 3.3 to 38.6 times less GPU memory. These results support direct spectral-response evolution as a useful temporal inductive bias when computational efficiency is a first-class requirement.

Yan Kong

Nanjing University of Information Science and Technology, China

Temporal GNNs Temporal Link Prediction Dynamic Graph Learning Spectral Methods Spectral Graph Convolution

Jul 28, 2026cs.LG

Breaking the Periodicity Assumption: Robust Tensorial Multi-View Clustering via Graph-Spectral Low-Rank Learning

Tensorial multi-view clustering (TMC) has achieved strong performance due to its ability to capture high-order correlations across multiple views. Most existing t-SVD-based TMC frameworks apply the Fast Fourier Transform (FFT) along the sample mode to impose frequency-domain low-rank constraints. However, we reveal that this widely adopted design critically relies on an implicit ``periodicity assumption'' induced by the sample arrangement. When samples are ordered by class, neighboring indices tend to be semantically similar, creating artificial local continuity along the sample mode and a favorable spectral structure for FFT-based low-rank regularization. Once this ordering is removed by random permutation, existing t-SVD-based TMC methods suffer severe performance degradation. This strong sensitivity to class ordering conflicts with the permutation-invariant nature of clustering and indicates that part of the reported performance may be attributed to a privileged sample arrangement rather than genuine high-order structure modeling. In this paper, we systematically investigate this phenomenon and its underlying algebraic and spectral mechanisms. To address this fundamental flaw, we further propose a graph-spectral low-rank tensor learning framework based on the Graph Fourier Transform (GFT), which replaces the fixed Fourier basis along the sample mode with a data-driven graph spectral basis, thereby capturing the intrinsic manifold structure without relying on a particular sample ordering. Moreover, we develop an anchor-based variant to address large-scale datasets efficiently. Extensive experiments on various benchmarks validate our findings and demonstrate the competitive or superior performance of the proposed methods compared with state-of-the-art TMC approaches.

Jintian Ji, Xingsu Li, Songhe Feng

Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China · School of Computer and Information Technology Beijing Jiaotong University, Beijing, China

Representation Learning Multi-View Learning Tensor Decomposition Multi-View Clustering Permutation-Invariant Learning Spectral Methods

Jul 27, 2026stat.ME

Spectral Truncation in Synthetic Control

Synthetic control (SC) matches a treated unit's pre-treatment trajectory to a weighted combination of donor units. We study Spectral SC, which instead matches the treated unit in coordinates defined by the leading temporal singular vectors of the donor panel, and a hybrid estimator that places separately tunable weight on retained and discarded directions, nesting raw-path SC and truncated Spectral SC as endpoints. We prove that the family reduces exactly to raw-path SC at full rank, that exact balance on

K

retained dimensions with

N_0

donors is underdetermined whenever

N_0>K+1

, with an affine solution set of dimension

N_0-K-1

, and that spectral imbalance maps to treatment-effect bias through a finite-sample best-linear-predictor decomposition. We evaluate the estimators across eleven data-generating regimes, using

400

replications per regime and donor-only placebo validation to select regularization and the mixing weight. Truncated Spectral SC has significantly higher RMSE than tuned raw-path SC in every regime, with paired differences equal to

4

to

11

Monte Carlo standard errors. The hybrid estimator selects raw-path matching in most replications and is statistically indistinguishable from tuned SC in most regimes. The result is highly sensitive to preprocessing. With raw inputs, the performance gap is large; after removing unit and time fixed effects before spectral decomposition, as suggested by the assumptions behind our bound, the gap nearly disappears and placebo validation begins to favor truncation. We interpret these findings diagnostically rather than as evidence that Spectral SC should replace raw-path SC. Basis-estimation noise, balancing underdetermination, and fixed-effects contamination determine when spectral matching can help.

Mojtaba Eslami

University of Calgary

Spectral Regularization Causal Effect Estimation Spectral Methods

Jul 27, 2026cs.LG

Perturbative-NeuSA: A Structured Spectral Framework for Time-Dependent PDEs

Neural spectral PDE solvers often learn an entire unresolved vector field even when an inexpensive approximate model can already capture most of the trajectory. Here we introduce Perturbative-NeuSA, a residual formulation that decomposes the target solution into a low-fidelity background and a high-resolution perturbation, so that only the unresolved dynamics is learned. Starting from the exact perturbation equation, the method combines a fixed spectral operator, a background-dependent correction, the background defect in the target PDE, and an optional neural closure. This construction makes the roles of physical structure and neural closure separately measurable. Across 2D Burgers, Klein-Gordon, and heterogeneous 2D wave equations, the deterministic structured solver outperforms the trained NeuSA baseline while requiring no neural-network training. The largest gains occur on Burgers, where the deterministic correction reduces training and extrapolation errors by factors of 24 and 44, respectively. In addition, a Klein-Gordon sweep over seven background resolutions shows that the effect of the closure is conditional: it improves a poor background by 3.6 times, becomes neutral at intermediate resolutions, and degrades a well-resolved background. For the wave equation, however, the closure provides an additional 18% reduction when the remaining residual is interface-localized. Multi-initial-condition diagnostics further show that the useful closure regime depends on the initial-condition spectrum and can disappear in extrapolation when structured correction already captures the dominant Burgers dynamics. Perturbative-NeuSA therefore reframes neural closure as a conditional, diagnosable correction governed by background fidelity, residual organization, and compatibility with the closure model.

Xianli Zhu, Jia Yin

School of Mathematical Sciences, Fudan University Shanghai, China

Neural Surrogate Modeling Neural PDE Solvers PDE Solving Spectral Methods

Jul 24, 2026quant-ph

Quantum Spectral Model: Data Reuploading with Input-Conditioned Frequency Support

A central design principle in modern machine learning and artificial intelligence is to align a model's inductive bias with the structure of its input data. For matrix-valued inputs, relevant matrix-level relationships can be characterised through spectral values and spectral subspaces; however, common coordinate-wise rotation-gate data-encoding unitaries used in most quantum machine learning models do not explicitly construct such a matrix-level representation. We introduce Quantum Spectral Models (QSMs), in which we construct the generator of the data-encoding unitary directly from each input matrix. We study three QSM variants based on symmetric, global block, and non-overlapping patch-local block Hamiltonians. Their outputs admit truncated Fourier representations in which input-dependent spectral gaps supply candidate phase carriers, while spectral subspaces help determine their coefficients. We evaluate the QSMs and comparison quantum models on two matrix representations of Pendigits and two controlled synthetic tasks defined by spectral statistics. At the largest evaluated circuit depth, QSM variants lead the tested quantum models in mean test accuracy across all four benchmarks. The patch-local QSM leads on Pendigits, whereas the global block-Hamiltonian QSM leads on the controlled spectral tasks. Ablations show a task-dependent reversal: subspace-preserving controls perform better on Pendigits, whereas spectral-value-only controls lead among the tested ablations on the synthetic tasks. Together, these results shed new light on quantum machine-learning model design by showing how input-conditioned spectral representations can provide an analysable inductive bias, while offering a broader perspective on structure-aware model design in machine learning and artificial intelligence.

Peiyong Wang, Udaya Parampalli, Casey R. Myers

CSIRO Technology · Research Way, Clayton VIC 3168, Australia · School of Computing and Information Systems +6

Quantum Machine Learning Spectral Methods

Jul 23, 2026cs.MA

Spectral Dynamics of Semantic Drift in Clinical Multi-Agent Language Model Networks

The integration of iterative LLMs within multi-agent diagnostic frameworks requires a rigorous quantitative reevaluation of underlying communication topologies. Frequently used architectural paradigms depend on scale-free or small-world networks, assuming optimal communication efficiency. Our study mathematically dismantles that assumption for semantic data. By mapping multi-agent communication uncertainty trajectories onto a 768-dimensional Bio_ClinicalBERT embedding space via an analytical isotropic variance proxy using Barab'asi--Albert (BA) and Watts--Strogatz (WS) networks, we prove that structural bottlenecks compromise diagnostic safety. Our phase transition matrices illustrate that localized dense cliques confine hallucinated data, preventing global consensus and forcing the system toward a permanent entropy saturation threshold of

H_{\infty} \approx 5.947

. As a result, we measure a severe terminal cosine similarity degradation of 53.29%, completely overwriting the original ground-truth. Moreover, the terminal semantic drift reveals a catastrophic variance amplification of 51.81% (

ρ= 1.5181

) in highly clustered architectures, proving total system unpredictability when compared to Erdős--R'enyi configurations (

ρ= 1.0766

). Instead of reducing errors, hub-centric systems autonomously compound localized hallucinations. By introducing dynamic spectral monitoring operating at an

\mathcal{O}(N^3)

time complexity and imposing a strict lower bound on algebraic connectivity (

λ_{2_{min}}

) via the continuous eigen-decomposition of the graph Laplacian, we present a mathematically rigorous technique to ensure global state diffusion. Securing the reliability of autonomous medical diagnostics necessitates treating topological stability as a non-negotiable quantitative imperative.

Amritesh Banerjee

University of Massachusetts Amherst, USA

Language Modeling Spectral Methods Multi-Agent Communication Multi-Agent LLMs Medical LLMs LLM Agent Reliability

Jul 23, 2026cs.LG

Filter Learning for Subgraphs: Algebras and Performance Risk Bounds

Graph signal processing tasks that leverage spectral information typically assume access to the complete graph topology, which is often unavailable in practice. We propose a systematic framework for subgraph filter learning (SFL), where subgraph-supported operators approximate ambient graph filters under partial observations. We formulate SFL as a statistical learning problem in which optimal subgraph operators are inherently data-dependent. To address the difficulty of directly estimating such operators, we develop a subgraph filter algebra based on distance-aware Laplacian constructions, defining a structured and controllable class of filters for effective approximation. We further establish performance risk bounds under the least squares loss, quantifying how well the learned operator approximates the restricted ambient mapping. Experiments real-world datasets show that, for SFL tasks, the proposed algebraic models consistently outperform polynomial filters, distribution-agnostic operators, and direct numerical filter learning baselines that attempt to recover the underlying structure from data.

Purui Zhang, Feng Ji, Yanan Zhao +2

Graph Signal Processing Spectral Methods

Jul 23, 2026cs.LG

Regularized Optimization on Grassmann Manifold: Theory, Algorithm and Applications

Spectral methods are among the most widely used techniques for community detection, clustering, and graph learning. Their performance, however, critically depends on the accurate estimation of the underlying spectral subspace and can deteriorate substantially in the presence of noise, outliers, or model perturbations. To address this limitation, we propose a Regularized Projection Matrix Approximation (RPMA) framework for robust estimation of rank-

K

projection matrices. RPMA extends classical spectral projection by incorporating a regularization term, producing projection estimates that are more robust, sparse, and interpretable. We formulate the proposed model as an optimization problem on the manifold of rank-

K

projection matrices and exploit its geometric equivalence to the Grassmann manifold. Based on this manifold characterization, we derive the first- and second-order optimality conditions, establish the local stability of the regularized leading eigenspace, and characterize the stability of the critical-point landscape under sufficiently small regularization. To efficiently solve the resulting nonconvex optimization problem, we develop a Riemannian gradient projection algorithm with backtracking line search, together with a more efficient Cayley--Sherman--Morrison--Woodbury (Cayley--SMW) gradient method that avoids repeated eigendecompositions. Extensive experiments on both synthetic and real-world datasets demonstrate that RPMA substantially improves the recovery accuracy of projection matrices and consistently outperforms conventional spectral projection methods for community detection and clustering under noisy environments.

Zhuan Liang, Zheng Zhai

Department of Statistics, Faculty of Arts and Sciences at Beijing Normal University, Zhuhai

Spectral Regularization Riemannian Optimization Nonconvex Optimization Graph Clustering Spectral Methods

Jul 15, 2026stat.ML

Spectral Concentration and Recovery in Sparse High-Dimensional Random Geometric Graphs

We study sparse random geometric graphs generated by connecting pairs of high-dimensional vectors whose inner product exceeds a threshold. The latent vectors are sampled either uniformly from the sphere or from a standard Gaussian distribution. Although every edge appears with probability

p

, the edges are dependent through their shared latent vectors. For the spherical model, at the connectivity scale

np=Ω(\log n)

, we prove

\|A-\mathbb E A\|=O\left(\sqrt{np\log n}+npτ\right)

, with high probability, where

τ

is the cap threshold. This sharpens the spectral norm bound of Liu, Mohanty, Schramm, and Yang (2023) under weaker assumptions. An analogous result holds for the Gaussian model after removing the fluctuations of the vector norms, yielding improved global synchronization guarantees for the homogeneous Kuramoto model. We then recover the latent geometry from the leading eigenspace. When

np\gg\log n

, both the latent vector and relative Gram matrix errors vanish provided

d\ll np\log(1/p)/\log n

. The required lower dimension is only

d\gg\log(1/p)

for the spherical model and

d\gg\log^2(1/p)\log n

for the Gaussian model, improving the recovery guarantees of Li and Schramm (2023). Finally, we prove the first exact recovery result for the Gaussian mixture block model of Li and Schramm (2023). At the optimal connectivity scale

np=Ω(\log n)

, a polynomial-time semidefinite program exactly recovers all labels in a moderate-separation regime, whereas larger separation makes exact recovery impossible because isolated vertices appear with high probability. Our proofs combine orthogonal polynomial expansions, decoupling, and matrix concentration, avoiding the trace-moment arguments used in previous work.

Manuel Fernandez, Yizhe Zhu

Department of Mathematics, University of Southern California

Graph Clustering Spectral Methods

Jul 15, 2026math.NA

Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs

For low-dimensional problems (

d\leq3

), spectral methods can achieve exceptionally high accuracy. For middle-dimensional problems (

4 \leq d \lesssim 10

), spectral methods remain feasible through specific techniques such as sparse grids or hyperbolic cross. However, for high-dimensional problems (

d\gg 10

), spectral methods suffer frome the curse of dimensionality. Physics-informed neural networks (PINNs) have emerged as a promising approach to overcome this challenge, offering scalability to high dimensions, but often suffer from limited accuracy and efficiency. Recently proposed spectral-informed neural networks (SINNs) combine spectral methods with PINNs, operating directly in the spectral domain to avoid spatial derivative computations and to reduce memory consumption. In this work, we introduce Modified SINNs, which integrate coefficient decay scaling and basis embeddings motivated by harmonic analysis to enhance accuracy in high-dimensional problems and enable accurate approximation of unknown spectral coefficients. Numerical experiments on steady and time-dependent partial differential equations demonstrate that Modified SINNs outperform sparse grid spectral methods on middle-dimensional problems with incomplete spectral information and achieve superior accuracy compared to PINNs on high-dimensional problems.

Tianchi Yu, Ivan Oseledets

1AXXX · 2Applied AI Institute · 3INM

PDE Solving Physics-Informed ML Spectral Methods

Jul 12, 2026cs.LG

The Spectral Structure of Latent Treatment Effects

Identifying heterogeneous treatment effects under unobserved confounding is central in observational causal inference. In proxy models with a discrete latent confounder, prior Synthetic Potential Outcomes (SPO) [Mazaheri-Squires-Uhler '25] recover the mixture of treatment effects through recursively constructed scalar moments. We show that this sequence is one projection of a more fundamental object. Under the same population factorization assumptions, there is an exact compressed observable operator: after projecting onto the shared proxy signal subspace, the difference of two treatment-arm quotient operators is similar to the diagonal matrix of latent treatment effects. Its eigenvalues are the latent effects; its lifted left eigenvectors, after anchor normalization, recover the target-proxy feature matrix and then the latent mixture proportions. Every scalar SPO moment is a bilinear functional of a power of this operator. The resulting estimator handles overcomplete proxy systems, replaces high-order scalar inversion with finite-dimensional spectral analysis, and admits high-probability first-order perturbation bounds for treatment effects, feature rows, and simplex-projected mixture weights.

Hamza Virk, Bijan Mazaheri, Yihren Wu

1Dartmouth College · 2Broad Institute of MIT and Harvard · 3Hofstra University

Causal Effect Estimation Latent Variable Models Conditional Average Treatment Effect Estimation Spectral Methods

Jul 10, 2026cs.CL

Complexity-Guided Component-wise Initialization for Language Model Pretraining

Pretrained language models often exhibit structured weight spectra, suggesting that training may repeatedly produce similar layerwise and component-wise organization. We ask whether these recurring spectral patterns can be reused as an initialization signal for GPT-2-style language-model pretraining. First, we analyze eleven pretrained GPT-2-style checkpoints that vary in size, language, tokenizer, and training corpus, measuring Frobenius norm and effective-rank entropy across layers and Transformer subcomponents. The checkpoints show shared depth trends, especially increasing scale and stronger spectral concentration in residual-writing matrices. We then construct initialization schemes that imitate the component-wise magnitudes and spectral profiles of pretrained models, and compare them with several weight initialization methods. These initializers visibly change the model's structural spectral patterns, but the evaluation results do not show a corresponding performance advantage. Pretrained-weight reuse remains competitive, while coarse spectral matching alone is not a reliable optimization strategy. Our results suggest that pretrained spectra are useful diagnostics of trained model structure, but that effective reuse likely requires preserving richer information than component-wise scale and singular-value shape.

Konstantin Garbers, Nicholas Oh

Peking University Beijing, China

Language Model Pretraining Neural Network Initialization Spectral Methods

Jul 9, 2026cs.LG

Group Invariant Spectral Embedding

Spectral embedding methods are widely used for dimensionality reduction and clustering of high-dimensional datasets with intrinsic low-dimensional structures. Although many datasets of practical interest exhibit invariance under symmetries such as rotations, standard spectral embedding methods do not account for this, treating symmetry-related data points as unrelated. Our approach to this problem is to incorporate the symmetries directly into the affinity kernels used for spectral embedding. We analyze the case of a Riemannian data manifold

M

with symmetries given by a compact Lie group~

G

and prove that, under suitable conditions, graph Laplacians constructed from three types of invariant kernels converge pointwise to explicit second-order differential operators on the quotient space

M/G

. Our analysis implies improved convergence rates, as the effective dimension drops according to the dimension of the group. We validate our approach on datasets with

\mathrm{SO}(2)

or

\mathrm{SO}(3)

symmetry, and show that

G

-invariant spectral embedding recovers the intrinsic geometry of the data, in contrast to standard spectral embedding, which fails to do so even in the limit of infinite data.

Yeari Vigder, Paulina Hoyos, David Thong +3

Department of Statistics and Operations Research, Tel Aviv University, Tel Aviv, Israel · Department of Mathematics, University of Texas at Austin, Austin, TX, USA · Department of Mathematics, KTH Royal Institute of Technology, Stockholm, Sweden

Manifold Learning Dimensionality Reduction Spectral Methods Invariant Representation Learning

Jul 8, 2026cs.LG

Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces

Spectral positional encodings (PEs) for \emph{directed} graphs face two obstacles: magnetic Laplacians require an

O(n^3)

Hermitian eigendecomposition per potential, and their complex eigenvectors are defined only up to unitary gauge, which prior work handles with basis-invariant architectures. We propose learnable spectral PEs of the form

h_θ(A_q)\,R

, where

A_q

is a normalized magnetic operator,

h_θ

a learnable scalar spectral response, and

R

a block of random probes. Because the PE is a \emph{matrix function} of the operator, it is gauge-invariant by construction. We compute it in a Hermitian block Krylov subspace from sparse matrix--vector products only, prove that

k = O(\log(1/\varepsilon))

block steps suffice uniformly over heat--resolvent response families, and give a covering-number argument for why low-dimensional structured families generalize where free per-eigenvalue weights overfit. On a directed SBM whose symmetrization is uninformative by construction, direction-blind PEs stay at chance while magnetic Krylov PEs converge to the exact-eigendecomposition oracle as the depth grows. The same probes yield gauge-invariant pairwise features with

1/\sqrt{s}

Monte-Carlo error, and the undirected

q{=}0

case improves heterophilous benchmarks over no-PE and polynomial baselines.

Jiaqing Xie, Yuxin Wang

Fudan University

Positional Encoding Spectral Methods Invariant Representation Learning Krylov Subspace Methods

Jul 6, 2026quant-ph

Quantum Spectral Anomaly Detection

A core task in quantum anomaly detection is to compute an anomaly score that quantifies how strongly a test quantum state deviates from a given quantum dataset assumed to be normal. Classically, principal component analysis (PCA) for centered data computes the anomaly score by evaluating the test sample relative to the subspace spanned by the selected leading eigenvectors. However, for quantum data that lack a standard centering, explicitly recovering principal eigenvectors, constructing full Gram matrices, or loading quantum-random-access-memory-style data can be more costly than estimating the anomaly score itself. To avoid these costs, we propose Quantum Spectral Anomaly Detection (QSPADE), which computes PCA-like anomaly scores directly from the spectrum of the average state of the normal dataset. By replacing hard PCA rank selection with a smooth, temperature-controlled spectral threshold, QSPADE makes near-threshold spectral components contribute partially to the anomaly score. This makes the score vary continuously rather than jump when a borderline component is included or excluded, and makes it less sensitive to noise or arbitrary hard cutoffs near the threshold. In the zero-temperature limit, QSPADE recovers the hard-projector PCA score. The proposed measurement-based quantum detector can be calibrated with a sample complexity independent of the data dimension. Numerical simulations show that QSPADE behaves like kernel-PCA on encoded classical data and detects changes across a transverse-field Ising transition without predefined order parameters. Consequently, QSPADE gives an efficient framework for both quantum-kernel anomaly detection on encoded classical data and the monitoring of quantum-native systems where diagnostic observables are unknown.

Yewei Yuan, Michele Minervini, Mark M. Wilde +1

Global College, Shanghai Jiao Tong University, Shanghai 200240, China · School of Electrical and Computer Engineering, Cornell University, Ithaca, New York 14850, United States · Shanghai Jiao Tong University, Shanghai 200240, China +1

Spectral Methods Anomaly Detection

Jul 4, 2026cs.LG

PIEFS: Physics-Informed Eigenfunction Features with Learnable Scaling

Spectral methods are widely used to construct representations from the geometry of data, but they often rely on a fixed kernel, graph Laplacian, or manually selected feature scaling. We propose Physics-Informed Eigenfunction Features with Learnable Scaling (PIEFS), a supervised neural representation-learning framework with a spectral inductive bias, based on a modified Dirichlet energy. In PIEFS, scalar coordinate maps are trained under empirical Gram orthogonality, a supervised linear readout, and a Dirichlet penalty in which the input gradient is transformed by a learnable metric

A(x)=Λ(x)U(x)

. The diagonal factor

Λ(x)

controls anisotropic scaling, while the orthogonal factor

U(x)

is parameterized by a structured product of Givens rotations. This construction yields task-adaptive Dirichlet-regularized coordinates rather than eigenfunctions of a fixed supervision-independent operator. Experiments on synthetic, tabular, and image-based benchmarks study the effect of identity, diagonal, and rotation-scaling metrics, and compare the resulting coordinates with classical baselines and NeuralEF. The results support PIEFS as a compact supervised spectral representation method and identify optimization stability, validation on explicit operator eigenproblems, and richer metric parameterizations as the main directions for future work.

Varvara Nazarenko, Timur Lidzhiev, Alexander Tarakanov

Faculty of Computer Science, HSE University, Moscow, Russia · AI VK, Moscow, Russia

Spectral Regularization Representation Learning Metric Learning Spectral Methods

Jul 3, 2026cs.CL

Spectral Signatures of Large Language Models

The rapidly growing repository of publicly available large language models (LLMs) presents significant challenges for systematic management and quantification at scale, such as model lineage tracing, licensing, and evaluation. However, task-specific benchmarks are insufficient for this setting, as LLMs differ widely in architectures, scales, and training procedures. To address this challenge, we adopt spectral shape-based metrics for managing and quantifying LLMs based on Heavy-Tailed Self-Regularization theory. Our approach uses the shape information of the weight empirical spectral density as a compact spectral signature of each model. This signature captures intrinsic properties of pretrained models and remains robust during post-training, making it suitable for model-level analysis. In addition, this metric is data-free, computationally-efficient, and scale-invariant, enabling large-scale analysis in practice. Moreover, we curate a large and diverse model corpus consisting of major open-source LLM families, and use it to systematically benchmark spectral and non-spectral metrics across models and downstream tasks. We show that our spectral signature supports the tracking of the model lineage, the unsupervised clustering of similar models, and the quantification of the model performance. Overall, the proposed spectral signature provides a meaningful proxy for broad performance trends across LLMs, enabling efficient organization, comparison, and analysis of large model collections.

Zhuoying Zhang, Ishan V. Prasad, Yuanzhe Hu +4

Dartmouth College Hanover, NH, USA · University of California, Berkeley Berkeley, CA, USA · Lawrence Berkeley National Laboratory Berkeley, CA, USA

LLM Evaluation Language Model Fingerprinting Spectral Methods

Jul 3, 2026quant-ph

Complexity of Normalized Persistence Problems for Topological Data Analysis and Local Hamiltonians

Topological data analysis (TDA) is a machine learning technique that uses topology to extract patterns from data and has shown the potential to exhibit quantum advantage. A key concept in TDA is persistent homology, which measures the robustness of topological information at different lengthscales. In this paper, we introduce and study the problem of normalized persistence, a practically motivated and easily interpretable version of persistent homology that counts the fraction of holes that persist at different lengthscales. We prove that a variant of normalized persistence is

\mathsf{DQC}_1

-hard and contained in

\mathsf{BQP}

, giving evidence of an exponential quantum speedup for TDA under the standard assumption that

\mathsf{DQC}_1 \not\subseteq \mathsf{BPP}

. These are the first

\mathsf{DQC}_1

-hardness results that are directly applicable to TDA instances. We also find a close connection between normalized persistence and the complexity of estimating spectral quantities in the low-energy subspace of local Hamiltonians. We study a family of such problems, including a low-energy normalized subtrace and spectral density. We show that these are

\mathsf{DQC}_1

-hard for

O(1)

-local Hamiltonians, strengthening previous results that required log-local interactions. We also introduce a variant of

\mathsf{DQC}_1

with perfect completeness (

\mathsf{SDQC}_1

) to characterize the hardness of problems normalized by an exact kernel. This includes normalized persistence for

O(1)

-local Hamiltonians, which we show is

\mathsf{SDQC}_1

-hard.

Dominic Lowe, M. S. Kim, Roberto Bondesan +1

Blackett Laboratory, Imperial College London, SW7 2AZ, United Kingdom · Department of Computing, Imperial College London, SW7 2AZ, United Kingdom · Yukawa Institute for Theoretical Physics & The Hakubi Center, Kyoto University, Japan

Spectral Methods Persistent Homology

Jul 3, 2026q-bio.QM

Recovering Candidate Circadian Regulators of Arrhythmic Pituitary Hormone Genes Using Reliability-Weighted Magnetic Laplacian with rwMagLap

We study how to recover candidate circadian-clock regulators of pituitary hormone genes that are important for women's health but do not show a clear 24-hour rhythm in bulk tissue, aiming to nominate clock-linked regulatory targets that could inform future chronopharmacologic and chronotherapeutic strategies. We propose \textbf{rwMagLap}, which builds a graph on rhythmic backbone genes. For each edge, we combine 24-hour fit quality with peak-time phase, represented as a complex unit-circle value, yielding a Hermitian adjacency matrix and a magnetic Laplacian. We insert arrhythmic hormone genes, treated as anchors, by a reliability-weighted nearest-neighbor projection. The projected anchor-neighbor weights are pooled into a soft teleport distribution, and complex personalized PageRank then ranks rhythmic backbone genes by the magnitude of their PageRank scores. In pituitary data, we find that all 11 women's-health anchors are arrhythmic. Even so, we find that the top-50 list is

7.95\times

enriched for the 13-gene KEGG circadian set (7 of the 8 set genes in the 454-gene backbone; corrected Benjamini-Hochberg (BH)

p_{\mathrm{BH}}=4\times10^{-6}

) and

4.54\times

enriched for the 111-gene Reactome set (8 of 16 genes;

p_{\mathrm{BH}}=1.6\times10^{-4}

), while a phase-blind real-valued baseline recovers none. We recover candidates through reliability weighting and phase-aware seeding rather than through magnetic propagation. The magnetic phase adds a different capability: it represents temporal order. On pituitary backbone, the magnetic embedding recovers measured peak-time order of connected pituitary genes with accuracy

0.971

, while

q{=}0

, i.e., no magnetic charge, is at chance.

Shabnam Sodagari, Nick Jasperson

Graph Signal Processing Graph-Based Retrieval Gene Regulatory Network Inference Spectral Methods

Jul 2, 2026cs.AI

Repair the Amplifier, Not the Symptom: Stable World-Model Correction for Agent Rollouts

Long-horizon language agents increasingly maintain executable world models in the form of planning graphs, where tool calls, validators, memory updates, recovery branches, and final answers are connected by typed dependencies. When a rollout fails, repairing the most visible error can leave the underlying error-amplification path intact, while replaying the full graph is expensive and difficult for long-context models to use reliably. We study world-model correction: selecting a compact subgraph of a failed planning graph whose repair stabilizes subsequent rollouts. We first instantiate a strong family of engineering correctors, including pointwise error scans, TopK and window selection, local graph expansion, cascade repair, and full-context LLM repair. We then propose WM-SAR, a spectral subgraph repair method that estimates node-edge amplification, greedily grows a connected repair region by marginal residual-spectral relief, and sends only this region to an LLM for root-cause repair. Theoretically, we connect residual spectral radius to rollout error and planning regret, motivating repair as stabilization rather than attribution alone. Across synthetic calling-tree graphs, benchmark-inspired agent topologies, and cross-model LLM repair experiments, WM-SAR achieves stronger long-horizon stabilization and root-cause recovery under compact token budgets, matching much larger repair contexts while exposing the LLM to a cleaner causal subgraph.

Xinyuan Song, Zekun Cai

Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

Spectral Methods Long-Horizon LLM Agents

Jun 29, 2026cs.LG

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

Hessian spectral properties are a standard tool in analysing neural-network training, with eigenvalues linked to sharpness, generalization, and optimization dynamics. Eigenvalues quantify curvature magnitude, while eigenvectors identify which parameters generate that curvature. In this work, we study how the leading Hessian eigenvectors evolve during training and how they affect the learning trajectories. We track the training dynamics of multilayer perceptrons on a classification problem and measure eigenvector dynamics through two complementary statistics: (i) displacement over time, inspired by analyses of glassy systems, and (ii) localization via the inverse participation ratio. The metrics are compared against a random null model of the Hessian induced by the architecture. Our results reveal clear optimizer-dependent behaviour. SGD leads to progressively more stable leading curvature directions, while Adam exhibits substantially stronger reorganization of eigenvectors throughout training. We also observe a localization phenomenon under Adam, where a small subset of parameters contributes disproportionately to the leading curvature directions. These results suggest that Hessian eigenvector dynamics capture key differences in optimizer behaviour and the resulting training trajectories.

Marcelina Marjankowska, Valerio Modugno, Paolo Barucca

Financial Computing and Analytics Group, University College London · Robot Perception and Learning Lab, University College London

Spectral Methods Adaptive Gradient Methods Neural Network Training Dynamics Stochastic Gradient Descent