Spectral Methods

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 169

All topics

Aurora: A Leverage-Aware Spectral Optimizer
Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Neural Network Optimization Muon Optimizer
Tilde Research
Multi-Objective Molecular Generation with Frequency-Controlled Evolutionary Dynamics
Jun 25, 2026Elia Colleoni, Paolo Guida, Didier Barradas-Bautista +1Multi-Objective Evolutionary Optimization Molecular Generation
Clean Energy Research Center (CERP) Physical Science and Engineering (PSE) Division King Abdullah University of Science and Technology Thuwal, Saudi Arabia 23955 · Kaust Visualization Lab (KVL) Core Lab Division King Abdullah University of Science and Technology Thuwal, Saudi Arabia 23955
Transformer-Based Classification of Bacterial Raman Spectra with LOOCV
Jun 25, 2026Jamile Mohammad Jafari, Thomas BocklitzTransformer Raman Spectroscopy
Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Helmholtzweg 4, 07743, Jena, Germany
Real vs. Complex Spectral Bases for Neural Operators: The Role of Green's Function Alignment
Jun 23, 2026Jason Sulskis, Sathya RaviFourier Neural Operator PDE Operator Learning
Electronic Systems Laboratory, Applied Embedded Systems Division Georgia Tech Research Institute · Department of Computer Science University of Illinois at Chicago
Can Aggregate Invariants Accelerate Continuous Subgraph Matching? Limits, Laws, and a Dynamic Spectral Index
Jun 23, 2026Minghao Chen, Jiale ZhengGraph Matching Spectral Methods
Tencent Technology Shenzhen, China · HUAWEI Noah’s Ark Lab Shenzhen, China
Convex--Concave Quadratic Spectral Filtering for Graph Neural Networks
Jun 23, 2026Ranhui Yan, Jia Cai, Mengzhu Chen +1Graph Neural Networks Spectral Methods
School of Computing and Artificial Intelligence, Guangzhou Xinhua University, Guangzhou, Guangdong, China · School of Statistics and Data Science, Guangdong University of Finance and Economics,2026 Guangzhou, Guangdong, China
A Spectral Theory of Normalized Corrected GNN Propagation
Jun 22, 2026Qihan Chen, Wei Li, Meng Qin +1Graph Neural Networks Spectral Methods
Center for Discrete Mathematics, Fuzhou University · School of Computer and Information Science, Fujian Agriculture and Forestry University · Department of Strategic & Advanced Interdisciplinary Research, Pengcheng Laboratory (PCL), Shenzhen, Guangdong, China
Collapsed Effective Operators for Higher-order Structures
Jun 22, 2026Maximilian Krahn, Lennart Bastian, Vikas Garg +2Graph Clustering Spectral Methods
Department of Computing, Imperial College London, London, UK · Chair of Health Informatics, Technical University of Munich, Germany · Munich Center for Machine Learning, Germany +2
A Standard Processing Pipeline for High-accuracy Measurement of Few-shot Regression on Laser Induced Breakdown Spectroscopy
Jun 20, 2026Hao LiFew-Shot Learning Spectral Methods
Electrical and Computer Engineering University of Arizona Tucson, USA
Reconstructing Randomly Masked Spectra Helps DNNs Identify Discriminant Wavenumbers
Jun 19, 2026Yingying Wu, Jinchao Liu, Yan Wang +3Synthetic Data Augmentation Masked Autoencoders
Institute of Robotics and Automatic Information System (IRAIS), College of Artificial Intelligence, Nankai University, China · VisionMetric Ltd, Canterbury, Kent, UK · School of Physics and Astronomy, University of Kent, UK +1
TF-SNO: Time-Frequency Gated Spectral Neural Operators for Learning Non-Stationary Partial Differential Equations
Jun 19, 2026Yitian Zhou, Chaoning Zhang, Zhenzhen Huang +8PDE Solving PDE Operator Learning
University of Electronic Science and Technology of China Chengdu, China · Institute of Software, Chinese Academy of Sciences Beijing, China · Harbin Institute of Technology Shenzhen, China
Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity
Jun 19, 2026Tejas Pradeep Shirodkar, P. J. NarayananSingular Learning Theory Spectral Methods
IIIT, Hyderabad
Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection
Jun 17, 2026Richard Yi Da XuDeterminantal Point Process Training Data Selection
Hong Kong Baptist University · TadReamk Limited
Geometrical fairness in graph neural networks
Jun 16, 2026Arturo Pérez-Peralta, Sandra Benítez-Peña, Blas Kolic +1Graph Neural Networks Graph Representation Learning
Department of Statistics, University Carlos III of Madrid, Spain · uc3m-Santander Big Data Institute
Multi-Adapter PPO: A Cross-Attention Enhanced Wavelength Selection Framework for LIBS Quantitative Analysis
Jun 16, 2026Hao Li, Man Fung ZhuoReinforcement Learning Proximal Policy Optimization
Electrical and Computer Engineering · University of Arizona · Tucson, USA
The Discrete-Log Clock: How a Transformer Learns Modular Multiplication
Jun 16, 2026Huu Danh NguyenTransformer Interpretability Frequency-Domain Feature Learning
Stanford University
Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performance
Jun 15, 2026Jernej Grlj, Aaron D. LaudaDimensionality Reduction Spectral Methods
Department of Mathematics University of Southern California Los Angeles, CA 90089 · Department of Mathematics Department of Physics University of Southern California Los Angeles, CA 90089
Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data
Jun 14, 2026Jeffery Opoku, David BanaheneWeighted Conformal Prediction Conformal Prediction
aUniversity of Texas Rio Grande Valley, United States · bFlorida International University, United States
Machine Learning for Biomedical Raman Spectroscopy: From Spectral Acquisition to Clinical Translation
Jun 12, 2026Bogdan Oancea, Ana Maria Seciu-Grama, Nicoleta Siminea +10Interpretable ML Raman Spectroscopy
National Institute of Research and Development for Biological Sciences, Romania · University of Bucharest, Romania · University of Turku, Finland +1
Muon $^p$ : Muon with Fractional Spectral Powers
Jun 11, 2026Yihe Dong, Will SawinNeural Network Optimization Muon Optimizer
Princeton University
Adaptive-Frequency Resonate-and-Fire Neurons for Spectral Estimation of Streaming Radar Signals
Jun 11, 2026Stefano Chiavazza, Sen Yuan, Marc Geilen +2Spiking Neural Networks Neuromorphic Computing
Electronic Systems, Eindhoven University of Technology, The Netherlands · MS3 Group, Delft University of Technology, The Netherlands
Limits of spectral learning under noise
Jun 11, 2026Sabin Roman, Ljupco Todorovski, Saso Dzeroski +2Noisy-Label Learning Spectral Methods
Jožef Stefan Institute, Ljubljana, Slovenia · Faculty of Mathematics and Physics, University of Ljubljana, Slovenia · Department of Chemical Engineering, Universitat Rovira i Virgili, Tarragona, Catalonia +2
Enhancing Spectral Embedding through Robust and Flexible Knowledge Transfer in Electronic Health Records
Jun 10, 2026Feiqing Huang, Zongqi Xia, Rong Ma +1Unsupervised Representation Learning Transfer Learning
Department of Biostatistics, Harvard T.H. Chan School of Public Health · Department of Neurology, University of Pittsburgh · Department of Data Science, Dana-Farber Cancer Institute +1
Geometric bias in eigenspace perturbation under random heterogeneous noise
Jun 9, 2026Fengkai Liu, Ke Wang, Wanjie WangRandom Matrix Theory Spectral Methods
Department of Mathematics, Hong Kong University of Science and Technology · Department of Statistics and Data Science, National University of Singapore
The Spectral Dynamics and Noise Geometry of Muon
Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon Optimizer Matrix Optimization
Massachusetts Institute of Technology
Chiaroscuro Attention: Spending Compute in the Dark
Jun 6, 2026Prateek Kumar SikdarTransformer Transformer Attention
Accenture, India
Frequency-Scale Saliency for Spectral Descriptor Analysis in 3D Shape Retrieval
Jun 5, 2026Jianru ShenSpectral Methods
University of Montana, Missoula, MT 59812, USA
Closed-Form Spectral Regularization for Multi-Task Model Merging
Jun 5, 2026Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4Spectral Regularization Multi-Task Learning
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Sun Yat-sen University, Shenzhen 510275, China +1
Monte Carlo Steklov Operators for Large-Scale Geometry Processing in the Wild
Jun 4, 2026Arman Maesumi, Tanish Makadia, Aruna Anderson +33D Representation Learning Spectral Methods
Brown University, USA · Loyola Marymount University, USA · Massachusetts Institute of Technology, USA
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Deep Learning Optimization Momentum Methods
1The Institute of Statistical Mathematics · 2The Graduate Institute for Advanced Studies, SOKENDAI · 3National Institute of Informatics +3
Pseudospectral Bounds for Transient Amplification in Coupled Gradient Descent
Jun 1, 2026Ahanaf Hasan AriqStochastic Approximation Two-Timescale Stochastic Approximation
Ideal School and College
Localizing broadband noise sources using the Loève spectrum and a 2.5D approach
Jun 1, 2026Christian H. Kasess, Wolfgang Kreuzer, Holger WaubkeDirection-of-Arrival Estimation Spectral Methods
Acoustics Research Institute, Austrian Academy of Sciences, Dominikanerbastei 16, Vienna, A-1010, Austria
Easy, robust approximate message passing for planted spike models
May 30, 2026Misha Ivkov, Tselil SchrammRandom Matrix Theory Principal Component Analysis
*Stanford University. · †Stanford University.
Data-Driven Spectral Prediction for Accelerating Large-Scale Electronic Structure Calculations
May 29, 2026Abhiram Badrinarayanan, Davor Davidovic, Edoardo Di Napoli +4Quantum Chemistry Scientific ML
Jülich Supercomputing Centre, Forschungszentrum Jülich, Germany · CEA / IRIG / MEM / L_Sim, France
Modeling Spectral Energy Shifts in Spatio-Temporal Graph Anomaly Detection
May 29, 2026Yilin Liu, Hongchao Zhang, Taylor T. Johnson +2Spectral Methods Spatiotemporal GNNs
College of Connected Computing, Vanderbilt University, Nashville, USA · Department of Civil and Environmental Engineering, Electrical and Computer Engineering, Vanderbilt University, Nashville, USA
Spectral Anatomy of Quantum Gaussian Process Kernels
May 29, 2026Jian Xu, Chao Li, Guang Lin +4Quantum Machine Learning Gaussian Processes
1RIKEN iTHEMS · 2RIKEN AIP · 3South China University of Technology +1
Rotation-Invariant Spherical Watermarking via Third-Order SO(3) Representation Coupling
May 26, 2026Pengzhen Chen, Yanwei Liu, Xiaoyan Gu +3Robust Watermarking Spherical Harmonics
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense +2
MuCon: Clipped Muon Updates for LLM Training
May 26, 2026Albert YiMuon Optimizer Spectral Methods
Mean-Shift PCA by Knockoff Mean
May 25, 2026Mengda Li, Zeng Li, Jianfeng YaoDistribution Shift Robustness Principal Component Analysis
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen, China
Learning Laplacian Eigenspace with Mass-Aware Neural Operators on Point Clouds
May 23, 2026Zherui Yang, Tao Du, Ligang LiuPoint Cloud Learning Geometric Representation Learning
University of Science and Technology of China, China · Tsinghua University, China · Shanghai Qi Zhi Institute, China +1
Entrywise Error Bounds for Spectral Ranking with Semi-Random Adversaries
May 22, 2026Dongmin Lee, Anuran Makur, Japneet SinghLearning to Rank Pairwise Ranking
Department of Computer Science Purdue University West Lafayette, IN, USA · Department of Computer Science Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN, USA
The ASE-LSE Disagreement Landscape: An End-to-End Characterisation of Extremes and Structural Drivers
May 21, 2026Minh Triet Pham, Ian GallagherGraph Embedding Spectral Methods
School of Mathematics and Statistics The University of Melbourne, Australia
Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
May 19, 2026Aarash Abro, Muhammad TahirLLM Fine-Tuning Spectral Methods
Zeta Labs · Lahore University of Management Sciences
Neural Acceleration for Graph Partitioning
May 18, 2026Joshua Dennis Booth, Vishvam PatelGraph Clustering Spectral Methods
Department of Computer Science University of Alabama in Huntsville Huntsville AL, USA
A Practical Noise2Noise Denoising Pipeline for High-Throughput Raman Spectroscopy
May 18, 2026David Martin-Calle, Cesar Alvarez Llamas, Vincent Motto- Ros +3Denoising Autoencoders Convolutional Autoencoder
ILM,UCBL,CNRS · Universit´e Claude Bernard Lyon 1, CNRS, Institut Lumi`ere Mati`ere, F-69622 Villeurbanne, France · ILM,UCBL,CNRS,IUF +1
Prism: Structural Symmetry Scanning via Duality-Constrained Laplacian Projection
May 18, 2026Jiatong XieGraph Clustering Spectral Methods
Independent researcher.
Self-Distillation is Optimal Among Spectral Shrinkage Estimators in Spiked Covariance Models
May 18, 2026Radu Lecoiu, Debarghya Mukherjee, Pragya SurSelf-Distillation Spectral Methods
Department of Statistics, Harvard University · Department of Mathematics & Statistics, Boston University
CHASM: Cross-frequency Harmonized Axis-Separable Mixing for Spectral Token Operators
May 14, 2026Pengcheng Fang, Hongli Chen, Yuxia Chen +3Frequency-Domain Feature Learning Accelerated MRI Reconstruction
University of Southampton · University of Queensland · 3Chengdu University of Technology +1
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
May 14, 2026Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi +2Hierarchical Representation Learning Scaling Laws
Département d’Informatique, Ecole Normale Supérieure, PSL & CNRS · Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)
Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology
May 14, 2026Jesseba Fernando, Grigori GuitchountsDynamical Systems Transformer
Network Science Institute, Northeastern University · Flagship Pioneering
Reframing preprocessing selection as model-internal calibration in near-infrared spectroscopy: A large-scale benchmark of operator-adaptive PLS and Ridge models
May 13, 2026Gregory Beurier, Robin Reiter, Camille Noûs +2Linear Regression Spectral Methods
CIRAD, UMR AGAP Institut, Montpellier, France · UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France · CIRAD, UMR AGAP Institut, F-34398 Montpellier, France +1
Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
May 13, 2026Tien-Phat Nguyen, Truong Nguyen, Minh-Phuc Truong +3Momentum Methods Gradient Descent
Hanoi University of Science and Technology Hanoi, Vietnam · Indiana University 107 S. Indiana Ave, Bloomington, IN 47405, USA · Monash University Clayton, VIC 3800, Australia
Modeling Heterophily in Multiplex Graphs: An Adaptive Approach for Node Classification
May 12, 2026Kamel Abdous, Nairouz Mrabah, Mohamed BouguessaGraph Representation Learning Spectral Methods
Department of Computer Science, University of Quebec at Montreal, Montreal, QC, Canada
Minimax Rates and Spectral Distillation for Tree Ensembles
May 12, 2026Binh Duc Vu, David S. WatsonModel Compression Gradient-Boosted Decision Trees
King’s College London
Bin Latent Transformer (BiLT): A shift-invariant autoencoder for calibration-free spectral unmixing of turbid media
May 12, 2026Martin HohmannAutoencoders Transformer Attention
Institute of Photonic Technologies (LPT), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Konrad-Zuse-Straße 3/5, 91052 Erlangen, Germany
A Spectral Framework for Closed-Form Relative Density Estimation
May 11, 2026Francis BachDensity Ratio Estimation Spectral Methods
SIERRA · Inria - Ecole Normale Supérieure · PSL Research University
Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs
May 11, 2026Xuxiang Zhao, Angelica I. Aviles-RiveroFourier Neural Operator Representation Learning
Qiuzhen College, Tsinghua University · YMSC, Tsinghua University
Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators
May 11, 2026Xiaoxiao Lu, Ye Yuan, Jiahao ShiLatent Dynamics Modeling PDE Surrogate Modeling
School of AIA, Huazhong University of Science and Technology
Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
May 10, 2026Shabarish Chenakkod, Michał DerezińskiLow-Rank Matrix Decomposition Randomized Sketching
University of Michigan, Ann Arbor, MI 48105.
Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates
May 9, 2026Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup +2U-Net PDE Surrogate Modeling
University of New Orleans, New Orleans, LA, USA · Naval Research Laboratory, Stennis Space Center, MS, USA

Jun 26, 2026, cs.LGJ/K move · Enter open · S save

Aurora: A Leverage-Aware Spectral Optimizer

Alec Dewulf, Dhruv Pai, Li Yang, Ashley Zhang+1

Tilde Research

Neural Network Optimization Muon Optimizer Matrix Optimization Spectral Methods Neural Network Training Dynamics

Open paper AskPDF arXiv

We show that for tall matrix parameters, like projection matrices in the MLP layers, the Muon update can have row norms that are arbitrarily non-uniform. This can lead to a self-reinforcing feedback loop whereby neurons receive persistently small updates and eventually do not contribute meaningfully to network outputs. This problem is effectively mitigated by an additional row normalization step, but current methods do this in a way that moves the Muon update geometry away from the polar factor of the momentum matrix, which we find is undesirable. We propose Aurora, an optimizer that enforces row-uniformity of matrix parameter updates while respecting Muon's polar factor geometry. Aurora outperforms Muon in our pre-training experiments and, when combined with existing methods, achieves state-of-the-art performance among spectral optimizers on the optimizer track of the modded-nanoGPT speedrun. Additionally, we find that Aurora's empirical gains over Muon scale with the MLP expansion factor, suggesting that Aurora may allow for effective training of very wide MLP layers.

Momentum

Related topics

Latest papers 169