Spectral Methods

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 169

All topics

Aurora: A Leverage-Aware Spectral Optimizer
Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Neural Network Optimization Muon Optimizer
Tilde Research
Multi-Objective Molecular Generation with Frequency-Controlled Evolutionary Dynamics
Jun 25, 2026Elia Colleoni, Paolo Guida, Didier Barradas-Bautista +1Multi-Objective Evolutionary Optimization Molecular Generation
Clean Energy Research Center (CERP) Physical Science and Engineering (PSE) Division King Abdullah University of Science and Technology Thuwal, Saudi Arabia 23955 · Kaust Visualization Lab (KVL) Core Lab Division King Abdullah University of Science and Technology Thuwal, Saudi Arabia 23955
Transformer-Based Classification of Bacterial Raman Spectra with LOOCV
Jun 25, 2026Jamile Mohammad Jafari, Thomas BocklitzTransformer Raman Spectroscopy
Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Helmholtzweg 4, 07743, Jena, Germany
Real vs. Complex Spectral Bases for Neural Operators: The Role of Green's Function Alignment
Jun 23, 2026Jason Sulskis, Sathya RaviFourier Neural Operator PDE Operator Learning
Electronic Systems Laboratory, Applied Embedded Systems Division Georgia Tech Research Institute · Department of Computer Science University of Illinois at Chicago
Can Aggregate Invariants Accelerate Continuous Subgraph Matching? Limits, Laws, and a Dynamic Spectral Index
Jun 23, 2026Minghao Chen, Jiale ZhengGraph Matching Spectral Methods
Tencent Technology Shenzhen, China · HUAWEI Noah’s Ark Lab Shenzhen, China
Convex--Concave Quadratic Spectral Filtering for Graph Neural Networks
Jun 23, 2026Ranhui Yan, Jia Cai, Mengzhu Chen +1Graph Neural Networks Spectral Methods
School of Computing and Artificial Intelligence, Guangzhou Xinhua University, Guangzhou, Guangdong, China · School of Statistics and Data Science, Guangdong University of Finance and Economics,2026 Guangzhou, Guangdong, China
A Spectral Theory of Normalized Corrected GNN Propagation
Jun 22, 2026Qihan Chen, Wei Li, Meng Qin +1Graph Neural Networks Spectral Methods
Center for Discrete Mathematics, Fuzhou University · School of Computer and Information Science, Fujian Agriculture and Forestry University · Department of Strategic & Advanced Interdisciplinary Research, Pengcheng Laboratory (PCL), Shenzhen, Guangdong, China
Collapsed Effective Operators for Higher-order Structures
Jun 22, 2026Maximilian Krahn, Lennart Bastian, Vikas Garg +2Graph Clustering Spectral Methods
Department of Computing, Imperial College London, London, UK · Chair of Health Informatics, Technical University of Munich, Germany · Munich Center for Machine Learning, Germany +2
A Standard Processing Pipeline for High-accuracy Measurement of Few-shot Regression on Laser Induced Breakdown Spectroscopy
Jun 20, 2026Hao LiFew-Shot Learning Spectral Methods
Electrical and Computer Engineering University of Arizona Tucson, USA
Reconstructing Randomly Masked Spectra Helps DNNs Identify Discriminant Wavenumbers
Jun 19, 2026Yingying Wu, Jinchao Liu, Yan Wang +3Synthetic Data Augmentation Masked Autoencoders
Institute of Robotics and Automatic Information System (IRAIS), College of Artificial Intelligence, Nankai University, China · VisionMetric Ltd, Canterbury, Kent, UK · School of Physics and Astronomy, University of Kent, UK +1
TF-SNO: Time-Frequency Gated Spectral Neural Operators for Learning Non-Stationary Partial Differential Equations
Jun 19, 2026Yitian Zhou, Chaoning Zhang, Zhenzhen Huang +8PDE Solving PDE Operator Learning
University of Electronic Science and Technology of China Chengdu, China · Institute of Software, Chinese Academy of Sciences Beijing, China · Harbin Institute of Technology Shenzhen, China
Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity
Jun 19, 2026Tejas Pradeep Shirodkar, P. J. NarayananSingular Learning Theory Spectral Methods
IIIT, Hyderabad
Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection
Jun 17, 2026Richard Yi Da XuDeterminantal Point Process Training Data Selection
Hong Kong Baptist University · TadReamk Limited
Geometrical fairness in graph neural networks
Jun 16, 2026Arturo Pérez-Peralta, Sandra Benítez-Peña, Blas Kolic +1Graph Neural Networks Graph Representation Learning
Department of Statistics, University Carlos III of Madrid, Spain · uc3m-Santander Big Data Institute
Multi-Adapter PPO: A Cross-Attention Enhanced Wavelength Selection Framework for LIBS Quantitative Analysis
Jun 16, 2026Hao Li, Man Fung ZhuoReinforcement Learning Proximal Policy Optimization
Electrical and Computer Engineering · University of Arizona · Tucson, USA
The Discrete-Log Clock: How a Transformer Learns Modular Multiplication
Jun 16, 2026Huu Danh NguyenTransformer Interpretability Frequency-Domain Feature Learning
Stanford University
Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performance
Jun 15, 2026Jernej Grlj, Aaron D. LaudaDimensionality Reduction Spectral Methods
Department of Mathematics University of Southern California Los Angeles, CA 90089 · Department of Mathematics Department of Physics University of Southern California Los Angeles, CA 90089
Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data
Jun 14, 2026Jeffery Opoku, David BanaheneWeighted Conformal Prediction Conformal Prediction
aUniversity of Texas Rio Grande Valley, United States · bFlorida International University, United States
Machine Learning for Biomedical Raman Spectroscopy: From Spectral Acquisition to Clinical Translation
Jun 12, 2026Bogdan Oancea, Ana Maria Seciu-Grama, Nicoleta Siminea +10Interpretable ML Raman Spectroscopy
National Institute of Research and Development for Biological Sciences, Romania · University of Bucharest, Romania · University of Turku, Finland +1
Muon $^p$ : Muon with Fractional Spectral Powers
Jun 11, 2026Yihe Dong, Will SawinNeural Network Optimization Muon Optimizer
Princeton University
Adaptive-Frequency Resonate-and-Fire Neurons for Spectral Estimation of Streaming Radar Signals
Jun 11, 2026Stefano Chiavazza, Sen Yuan, Marc Geilen +2Spiking Neural Networks Neuromorphic Computing
Electronic Systems, Eindhoven University of Technology, The Netherlands · MS3 Group, Delft University of Technology, The Netherlands
Limits of spectral learning under noise
Jun 11, 2026Sabin Roman, Ljupco Todorovski, Saso Dzeroski +2Noisy-Label Learning Spectral Methods
Jožef Stefan Institute, Ljubljana, Slovenia · Faculty of Mathematics and Physics, University of Ljubljana, Slovenia · Department of Chemical Engineering, Universitat Rovira i Virgili, Tarragona, Catalonia +2
Enhancing Spectral Embedding through Robust and Flexible Knowledge Transfer in Electronic Health Records
Jun 10, 2026Feiqing Huang, Zongqi Xia, Rong Ma +1Unsupervised Representation Learning Transfer Learning
Department of Biostatistics, Harvard T.H. Chan School of Public Health · Department of Neurology, University of Pittsburgh · Department of Data Science, Dana-Farber Cancer Institute +1
Geometric bias in eigenspace perturbation under random heterogeneous noise
Jun 9, 2026Fengkai Liu, Ke Wang, Wanjie WangRandom Matrix Theory Spectral Methods
Department of Mathematics, Hong Kong University of Science and Technology · Department of Statistics and Data Science, National University of Singapore
The Spectral Dynamics and Noise Geometry of Muon
Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon Optimizer Matrix Optimization
Massachusetts Institute of Technology
Chiaroscuro Attention: Spending Compute in the Dark
Jun 6, 2026Prateek Kumar SikdarTransformer Transformer Attention
Accenture, India
Frequency-Scale Saliency for Spectral Descriptor Analysis in 3D Shape Retrieval
Jun 5, 2026Jianru ShenSpectral Methods
University of Montana, Missoula, MT 59812, USA
Closed-Form Spectral Regularization for Multi-Task Model Merging
Jun 5, 2026Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4Spectral Regularization Multi-Task Learning
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Sun Yat-sen University, Shenzhen 510275, China +1
Monte Carlo Steklov Operators for Large-Scale Geometry Processing in the Wild
Jun 4, 2026Arman Maesumi, Tanish Makadia, Aruna Anderson +33D Representation Learning Spectral Methods
Brown University, USA · Loyola Marymount University, USA · Massachusetts Institute of Technology, USA
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Deep Learning Optimization Momentum Methods
1The Institute of Statistical Mathematics · 2The Graduate Institute for Advanced Studies, SOKENDAI · 3National Institute of Informatics +3

Jun 26, 2026, cs.LGJ/K move · Enter open · S save

Aurora: A Leverage-Aware Spectral Optimizer

Alec Dewulf, Dhruv Pai, Li Yang, Ashley Zhang+1

Tilde Research

Neural Network Optimization Muon Optimizer Matrix Optimization Spectral Methods Neural Network Training Dynamics

Open paper AskPDF arXiv

We show that for tall matrix parameters, like projection matrices in the MLP layers, the Muon update can have row norms that are arbitrarily non-uniform. This can lead to a self-reinforcing feedback loop whereby neurons receive persistently small updates and eventually do not contribute meaningfully to network outputs. This problem is effectively mitigated by an additional row normalization step, but current methods do this in a way that moves the Muon update geometry away from the polar factor of the momentum matrix, which we find is undesirable. We propose Aurora, an optimizer that enforces row-uniformity of matrix parameter updates while respecting Muon's polar factor geometry. Aurora outperforms Muon in our pre-training experiments and, when combined with existing methods, achieves state-of-the-art performance among spectral optimizers on the optimizer track of the modded-nanoGPT speedrun. Additionally, we find that Aurora's empirical gains over Muon scale with the MLP expansion factor, suggesting that Aurora may allow for effective training of very wide MLP layers.

Momentum

Related topics

Latest papers 169