Spectral Regularization

Follow this topic Ask about this topic

Spectral regularization modifies learning by shrinking, filtering, or constraining directions identified through eigenvalues, singular values, or frequency components. Common approaches include ridge and Tikhonov filters, principal-component truncation, early stopping, spectral- or nuclear-norm penalties, and frequency-domain losses. These methods are studied in kernel learning and inverse problems as well as neural-network training, graph learning, and generative models, where they can improve generalization, stability, or representation quality. Theory provides adaptive and minimax guarantees in some settings, but a central challenge is choosing which directions to suppress: uniform shrinkage may discard useful structure, while selective control must avoid leaving unstable or collapsed modes unchecked.

Latest papers

43 papers5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Regularized free deconvolution for spectral repair of resampling distortion in high-dimensional particle filters
Oct 16, 2026Deming Song, Chi-Kin Lam, Kefeng Li +3Particle Filtering Spectral Regularization
Macao Polytechnic University, Macao · College of Textile Science and Engineering, Wuyi University, Jiangmen, China · Guangdong Provincial Key Laboratory of Novel Polyamide 6 Functional Fiber Materials Research and Application, Jiangmen, China
ORCA: The Annealed Spectral Conditioning Optimizer for Faster, Better LLM Training
Oct 5, 2026Yuanshi Liu, Boyuan Jiang, Liang Hou +4Spectral Regularization Deep Learning Optimization
School of Intelligence Science and Technology, Peking University · Kling Team
Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression
Sep 30, 2026Juno Kim, Hengyu Fu, Peter Bartlett +2Spectral Regularization Linear Regression
University of California, Berkeley · Google DeepMind
$λ$ -JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning
Sep 28, 2026Berker Demirel, Clémentine Dominé, Valentino Maiorca +3Spectral Regularization Joint-Embedding Predictive Architecture
Institute of Science and Technology Austria (ISTA) Am Campus 1, 3400 Klosterneuburg, Austria
Prox-Friendly Log-Magnitude Prior on Complex-Valued Signal
Sep 28, 2026Kazuki Matsumoto, Keidai Arai, Kohei YatabeSpectral Regularization Speech Enhancement
Tokyo University of Agriculture and Technology (TUAT), Tokyo, Japan
Graph Learning with Spectral Connectivity Priors for Scarce Data
Sep 23, 2026Mingxiao Liu, Bahar Oveisgharan, Bingyan Zou +3Graph Signal Processing Graph Structure Learning
Tsinghua University, China · York University, Canada
A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks
Sep 17, 2026Mike Nguyen, Nicole MückeEarly Stopping Nonparametric Regression
Technical University of Braunschweig
Why Learning Rediscovers the Closed-Form Diagonal Regularizer
Sep 9, 2026Jeahn Han, Pyojin KimSpectral Regularization PDE Inverse Problems
1GIST
Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning
Sep 7, 2026Yuwen Hao, Menglin YangSpectral Regularization Continuous Latent Reasoning
The Hong Kong University of Science and Technology (Guangzhou)
Spectral Initialization and Scheduled Graph Smoothness for Uncertain Knowledge Graph Completion
Sep 2, 2026Md Abrar Jahin, Taufikur Rahman Fuad, Jay Pujara +1Spectral Graph Embedding KG Completion
University of Southern California · Islamic University of Technology
Classical Limits of Spectral Filtering in Quantum Generative Models
Aug 14, 2026Marco RothQuantum Circuit Born Machines Quantum Machine Learning
Fraunhofer Institute for Industrial Engineering IAO, Nobelstr. 12, D-70569 Stuttgart, Germany
Spectral Truncation in Synthetic Control
Jul 27, 2026Mojtaba EslamiSpectral Regularization Causal Effect Estimation
University of Calgary
Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
Jul 24, 2026Quyen Tran, Hai Nguyen, Quan Dao +4Class-Incremental Learning Spectral Regularization
1Rutgers University · 2Tuft University · 3HUST +1
Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent
Jul 24, 2026Peng ZhaoSpectral Regularization Early Stopping
Department of Applied Economics and Statistics · University of Delaware
Regularized Optimization on Grassmann Manifold: Theory, Algorithm and Applications
Jul 23, 2026Zhuan Liang, Zheng ZhaiSpectral Regularization Riemannian Optimization
Department of Statistics, Faculty of Arts and Sciences at Beijing Normal University, Zhuhai
An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
Jul 22, 2026Jiachun LiSpectral Regularization Muon Optimizer
How to Tame Grokking: Representation Geometry as a Control Signal
Jul 13, 2026Maksim A KazanskiiNeural Network Generalization Spectral Regularization
Independent Researcher
Spectrally Deconfounded Gradient Boosting
Jul 10, 2026Andrea Nava, Peter Bühlmann, Fabio SigristSpectral Regularization Gradient-Boosted Decision Trees
Seminar for Statistics, ETH Zürich, Zürich, Switzerland · IFZ, Lucerne University of Applied Sciences and Arts, Rotkreuz, Switzerland
PIEFS: Physics-Informed Eigenfunction Features with Learnable Scaling
Jul 4, 2026Varvara Nazarenko, Timur Lidzhiev, Alexander TarakanovSpectral Regularization Representation Learning
Faculty of Computer Science, HSE University, Moscow, Russia · AI VK, Moscow, Russia
Regularized Variational and Spectral Log-Density-Ratio Estimation in the Gaussian Location Model
Jul 2, 2026Francis BachSpectral Regularization Density Ratio Estimation
SIERRA · Inria - Ecole Normale Sup´erieure PSL Research University
Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation
Jun 27, 2026George Coote, Matthew J. ColbrookSpectral Regularization Koopman Operator Learning
Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, CB3 0WA, United Kingdom
Differential Spectral Damping Gap Adaptive Regularization for Ill-Conditioned Kernel Methods
Jun 22, 2026Praveg VashishthaSpectral Regularization Support Vector Machines
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · Sopra Banking Software India
The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders
Jun 20, 2026Weiqin Yang, Yue Pan, Chongming Gao +4Spectral Regularization Sequential Recommendation
Zhejiang University · State Key Laboratory of Blockchain and Data Security, Zhejiang University · College of Computer Science and Technology, Zhejiang University +2
Spectrally Safe Neural Operator Warm-Starts for Large-Scale Newton Solvers
Jun 20, 2026Jaemin Oh, Youngkyu Lee, Jerome Darbon +1Spectral Regularization PDE Solving
Division of Applied Mathematics, Brown University, Providence, RI 02912, USA
Lung-SRAD: Spectral-Aware Regularized Audio DASS with Dual-Axis Patch-Mix Contrastive Learning for Respiratory Sound Classification
Jun 10, 2026Hemansh Shridhar, Miika Toikkanen, June-Woo KimContrastive Learning Spectral Regularization
RSC LAB, MODULABS, Republic of Korea · Department of Electronic Engineering, Wonkwang University, Republic of Korea · AI Convergence Research Institute, Wonkwang University, Republic of Korea
Spectrally Regularized Latent Flow Matching for Turbulence Generation
Jun 10, 2026Khalid Rafiq, Aditya G. NairSpectral Regularization Flow Matching
Department of Mechanical Engineering University of Nevada, Reno, NV, USA
Closed-Form Spectral Regularization for Multi-Task Model Merging
Jun 5, 2026Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4Spectral Regularization Multi-Task Learning
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Sun Yat-sen University, Shenzhen 510275, China +1
Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View
Jun 3, 2026Mingyu LiSpectral Regularization Weight Decay
Beijing Normal University
Stochastic Rounding Increases Small Singular Values
May 29, 2026Linkai Ma, Tingzhou Yu, Petros DrineasSpectral Regularization Stochastic Rounding
Department of Mathematics University of Alberta
Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization
May 28, 2026Petar JolakoskiSpectral Regularization Linear Regression
Research Center for Computer Science and Information Technologies, Macedonian Academy of Sciences and Arts, Bul. Krste Misirkov 2, 1000 Skopje, Macedonia

Oct 16, 2026J/K move · Enter open · S save

Regularized free deconvolution for spectral repair of resampling distortion in high-dimensional particle filters

Deming Song, Chi-Kin Lam, Kefeng Li, Hui Yu+2

Macao Polytechnic University, Macao · College of Textile Science and Engineering, Wuyi University, Jiangmen, China · Guangdong Provincial Key Laboratory of Novel Polyamide 6 Functional Fiber Materials Research and Application, Jiangmen, China

Particle Filtering Spectral Regularization

Open paper AskPDF arXiv

Resampling is the standard remedy for weight degeneracy in particle filters, but in high dimensions it distorts the spectrum of the ensemble covariance in a way that is computable from the importance weights. This paper studies the inverse problem: recovering the pre-resampling spectrum from the observed weights and the empirical spectrum alone, and using it without biasing the filter. We first show that the forward spectral law extends to random importance weights once one conditions on the weights together with the realized offspring counts, rather than on the weights alone. We then construct a regularized free-deconvolution estimator, prove its consistency with an explicit error budget at a fixed resolution scale, and certify model mismatch through a computable forward residual. The repaired spectrum enters the filter only as the preconditioner of a Metropolis-adjusted rejuvenation move, so that spectral error affects mixing efficiency but not the invariant target. For the full filtering cycle, we derive a geometrically summable error recursion under stable scalar dynamics and an explicit law for the cost of abstaining from repair. Pre-registered simulations show that the inversion reduces the spectral error by an order of magnitude, that the corrected preconditioner recovers about 90% of the oracle's sampling efficiency in an anisotropic regime, and that the cost of abstention depends on when abstentions occur rather than on how many. The repair recovers eigenvalue laws, not eigenvectors, and is certified only within an explicitly stated exogenous-weight regime.