Spectral Norm

Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 86

All topics
CardsList

  1. A Simple Doxastic Deontic Logic for Norm-Guided Decision Making

    Sep 30, 2026Thorsten Engesser, Agata CiabattoniEthicsDecisions

    TU Wien

  2. Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum

    Sep 30, 2026Hanyng Peng, Hui Wang, Yue YuMomentum Stochastic Gradient DescentNewton

    Pengcheng Laboratory, Shenzhen, China

  3. Singularities of Non-negative Matrix Factorization and their application to Bayesian inference

    Sep 28, 2026Naoki Hayashi, Yota Maeda, Yasushi EsakiNonnegative Matrix FactorizationSpectral Norm

    Toyota Central R&D Labs., Inc.

  4. Benign Overfitting for General Norms and Distributions

    Sep 27, 2026Daniel Barzilai, Ohad ShamirOverfittingSpectral Norm

    Weizmann Institute of Science · University of Toronto & Vector Institute

  5. Nuclear Norm-Regularized Bayesian Matrix Completion

    Sep 24, 2026Calvin TolbertTensor CompletionMatrix

    School of Operations Research and Information Engineering Cornell University

  6. Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets

    Sep 21, 2026Edward Raff, James HoltSpectral NormSolvers

    CrowdStrike, Inc., USA · Univ. of Maryland, Baltimore County, USA

  7. Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification

    Sep 20, 2026Kunyu Wang, Dehan Wang, Wenjun ChenAdversarial TrainingAdversarial Examples

    The Chinese University of Hong Kong · Columbia University

  8. Stable Movement for Nondual Lipschitz Convex Optimization: Efficiency and Nearly Optimal Oracle Rates

    Sep 17, 2026David Martínez-Rubio, Cristóbal GuzmánConvex OptimizationLipschitz Continuity

    IMDEA Software Institute, Madrid, Spain · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

  9. The First-Order Oracle Complexity of Lipschitz Convex Optimization in Nondual Settings

    Sep 17, 2026David Martínez-Rubio, Brian Bullins, Cristóbal Guzmán +1Convex OptimizationLipschitz Continuity

    IMDEA Software Institute, Madrid, Spain · Purdue University, West Lafayette, IN, USA · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile +1

  10. On the Regularization Landscape for the Linear Recommendation Models

    Sep 12, 2026Dong Li, Zhenming Liu, Ruoming Jin +4Real-World Content Recommendation ProblemLow-Rank Structure

    Kent State University, USA · College of William and Mary, USA · iLambda, USA

  11. Structured Transforms for Low-Overhead Quantization of Language Models

    Sep 11, 2026Daria Cherniuk, Alexander Rudikov, Boris Kashin +1Large Language Model QuantizationModel Weights

    Institute of Numerical Mathematics · Steklov Mathematical Institute

  12. Optimal Low-Rank Quantum State Tomography with Bounded-Sample Joint Measurements

    Sep 9, 2026Ashwin Nayak, Xingyu ZhouQubitOptimal Sample Complexity

    Department of Combinatorics and Optimization, and Institute for Quantum Computing, University of Waterloo. · Department of Computer Science, University of British Columbia.

  13. Oracle Complexity of Stochastic Fixed-Point Equations with Nonexpansive Maps

    Sep 8, 2026Jelena Diakonikolas, Cristóbal Guzmán, David Martínez-RubioFirst Order Oracle ComplexityFixed-Point Iteration

    Department of Computer Sciences, University of Wisconsin-Madison · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile · IMDEA Software Institute, Madrid, Spain

  14. Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tails

    Sep 3, 2026Shi Fu, Huibo Xu, Qixin Zhang +1Non-GaussianityLong-Tailed Distribution

    Generative AI Lab, College of Computing and Data Science, Nanyang Technological University, Singapore

  15. Exact Recovery Thresholds for Weighted Data Selection in Vector-Valued Linear Regression

    Aug 31, 2026Guangjian ZhangAlignment-Aware SelectionSpectral Norm

  16. Intrinsic Interaction Geometry Controls the Low-Rank Complexity of Softmax Attention

    Aug 28, 2026Yuhe Sui, Jianing Zhang, Yingzhi TangLow-Rank StructureSpectral Norm

    Nanyang Technological University, Singapore · Carnegie Mellon University

  17. Toward a First-Principles Update Geometry for the Language-Model Head

    Aug 23, 2026Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno +2Spectral NormMulti-Head Attention

    MBZUAI

  18. Exploring Oversmoothing with Householder Matrices

    Aug 12, 2026Bhaskar KarolGraph Neural NetworksSpectral Norm

  19. Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

    Aug 12, 2026Kai Qi, Xinji Huang, Hongchun WangSupport Vector MachinesLoss Function

    National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing, 401331, China. · School of Mathematical Sciences, Chongqing Normal University, Chongqing, 401331, China.

  20. Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection

    Jul 30, 2026Arunan JOptimal Sample ComplexityEmpirical Risk Minimization

    Independent Researcher, Chennai, India.

  21. Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm

    Jul 28, 2026Wenzhi Zhong, Edward Milsom, Michael MurraySharpness-Aware MinimizationRegularization

    Department of Mathematical Sciences University of Bath · Department of Computer Science University of Bath

  22. Beyond a Global Norm: Personalizing Toxicity Sensitivity in Language Models Without Retraining

    Jul 25, 2026Rares A. C. Diaconescu, Iulia Slanina, Alina Florea +5ToxicityHarmful Language

    Delft University of Technology

  23. Efficient Learning of Truncated Boolean Product Distributions: Influence to the Rescue

    Jul 24, 2026Rohan Chauhan, Ioannis PanageasOptimal Sample ComplexitySpectral Norm

    University of California, Irvine

  24. Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach

    Jul 20, 2026Siddharth ChandakStochastic ApproximationConcentration Bounds

    Department of Electrical Engineering, Stanford University, Stanford, 94305, CA, USA

  25. Muse: Representation Geometry of Muon Beyond Normalized Momentum

    Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4MuonRepresentation Space

    1Pengcheng Laboratory · 2Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +3

  26. Spectral Concentration and Recovery in Sparse High-Dimensional Random Geometric Graphs

    Jul 15, 2026Manuel Fernandez, Yizhe ZhuInhomogeneous Random GraphsDiscriminative Congruence Transform

    Department of Mathematics, University of Southern California

  27. Cross-Trajectory Chimera Interventions Reveal Dissociable Roles of Weight Magnitude and Direction in Grokking

    Jul 7, 2026Truong Xuan KhanhControlled ComparisonCircuits

    H&K Research Studio, Clevix LLC, Hanoi, Vietnam

  28. NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

    Jun 26, 2026Tianlin Pan, Lianyu Pang, Cheng Da +4Reinforcement Fine-TuningNorm

    The Hong Kong University of Science and Technology · Kuaishou Technology · University of Chinese Academy of Sciences

  29. The Decision Geometry of Covariance Estimation for the Global Minimum-Variance Portfolio under Heavy Tails

    Jun 25, 2026Xavier FonsecaPortfolio OptimizationCovariance

    Academy for AI, Games and Media, Breda University of Applied Sciences, Breda, The Netherlands.

  30. Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

    Jun 24, 2026Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +2AdamMuon

    HSE University · Basic Research of Artificial Intelligence Laboratory (BRAIn Lab)

  31. Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

    Jun 24, 2026Alexander Hägele, Alejandro Hernández-Cano, Atli Kosson +1Adaptive OptimizersDecoupling

    Machine Learning and Optimization Lab, EPFL

  32. Uniform Sampling from High-dimensional Spectral Norm Balls

    Jun 23, 2026Michael R. MetelSpectral NormSingular Value Decomposition

  33. Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm

    Jun 17, 2026Shan Fan, Feng Zhang, Jianjun Wang +2Tensor CompletionLow-Rank Structure

    School of Mathematics and Statistics, Southwest University, Chongqing 400715, China · School of Mathematical Sciences/Research Center for Image and Vision Computing, University of Electronic Science and Technology of China, Chengdu 611731, China · Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology, Shenzhen 518055, China

  34. Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces

    Jun 16, 2026Yahong Yang, Zecheng Zhang, Wei Zhu +2Probabilistic Operator LearningLogarithmic Sobolev Inequalities

    School of Mathematics, Georgia Institute of Technology, 686 Cherry Street, Atlanta, GA 30332-0160, USA. · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Mathematics, Hong Kong Baptist University, FSC1202, Fong Shu Chuen Building, Hong Kong Baptist University, Kowloon Tong, Hong Kong.

  35. Sobolev Approximation by Fixed-Size Neural Networks with Arbitrary Accuracy

    Jun 15, 2026Baicheng Li, Haizhao Yang, Shijun ZhangNeural ApproximationsLogarithmic Sobolev Inequalities

    Department of Mathematics, University of Maryland, College Park, MD, USA · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China

  36. Schattor: Schatten-family methods for deep learning optimization

    Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeNeural Network OptimizationStochastic Optimization

    School of Data Science, The Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore · Department of Mathematics, Linköping University, Sweden

  37. When to use what Schatten-pp norm in deep learning?

    Jun 13, 2026Thomas PethickSpectral NormMuon

  38. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraSpectral NormMuon

    Department of Computer Science, ETH Zurich, Switzerland · Department of Mathematics, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML)

  39. Geometric Domain Adaptation via Optimal Transport for Linear Regression in R^2

    Jun 12, 2026Brian Britos, Mathias BourelDomain AdaptationSpectral Norm

    IESTA, Facultad de Ciencias Económicas y de Administración, Universidad de la República, Montevideo, Uruguay · IRL-2030, Instituto Franco-Uruguayo de Matemática e Interacciones (IFUMI)

  40. Bernstein-Schur Kernels: Random Features by Sketched Modulation and Radial Randomization

    Jun 8, 2026Taha BouhsineAtom-Averaged FeaturesKernel Hilbert Spaces

    Azetta AI

  41. The Tell-Tale Norm: ℓ2\ell_2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models

    Jun 4, 2026Jinyang Zhang, Hongxin Ding, Yue Fang +4LLM Reasoning StrategiesL2

    School of Computer Science, Peking University · National Engineering Research Center of Software Engineering, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education +2

  42. Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithms

    Jun 3, 2026Chong Zhang, Xiang Li, Jia Wang +2Robustness VerificationFisher Information Matrix

    Xi’an Jiaotong-Liverpool University · The Chinese University of Hong Kong

  43. Beyond ℓ2\ell_2-norm and ℓ∞\ell_\infty-norm: A Curvature-Inspired ℓp\ell_p-Norm Scheme for Deep Neural Networks

    Jun 1, 2026Jianhao Xu, Zhuang YangNeural Network OptimizationAdaptive Optimizers

    School of Computer Science and Technology, Soochow University, Suzhou, 215006, China

  44. Improved Distribution Estimation in ℓ∞\ell_\infty

    May 28, 2026Doron Cohen, Aryeh Kontorovich, Yonatan LivshitzConcentration BoundsDistributionally-Robust Optimization

    Department of Computer Science, Ben-Gurion University of the Negev, Beer-Sheva, Israel

  45. Algorithms with Polynomially-Improved Approximation Factors for the 2→q2 \rightarrow q Norm, and Applications

    May 24, 2026Samuel B. Hopkins, Stefan TiegelSpectral NormApproximation Algorithms

    MIT

  46. From SGD to Muon: Adaptive Optimization via Schatten-p Norms

    May 19, 2026Thomas Massena, Corentin Friedrich, Mathieu SerrurierOptimizer DesignMuon

    IRIT, DTIPG - SNCF, UT3 · IRIT · IRT Saint Exupery

  47. Optimal Reconstruction from Linear Queries

    May 19, 2026Yuval Filmus, Shay Moran, Elizaveta NesterovaReconstruction ErrorO(\Bar{K}\Log N)$

    Technion – Israel Institute of Technology · Google Research

  48. Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South

    May 18, 2026Charvi Rastogi, Mukul Bhutani, Minsuk Kahng +13Multilingual SafetyRed-Teaming

    Google DeepMind, USA · Yonsei University, Republic of Korea · Imperial College, UK +4

  49. Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise

    May 18, 2026Jiayu Zhang, Tianyi LinNeural Network OptimizationSpectral Norm

    Department of Industrial Engineering and Operations Research Columbia University

  50. GQA-μP: The maximal parameterization update for grouped query attention

    May 14, 2026Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7Model WeightsParameter-Efficient Fine-Tuning Methods

    UC Davis & MBZUAI IFM · USC & MBZUAI IFM · USC +4

  51. Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters

    May 12, 2026Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1Gradient ClippingStochastic Gradient Descent

    MBZUAI · University of Southern California

  52. Muon is Not That Special: Random or Inverted Spectra Work Just as Well

    May 11, 2026Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6MuonSpectral Norm

    University of Cambridge · University of Tübingen · University of Oxford

  53. Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds

    May 10, 2026Yibang Li, Bihari Lal Pandey, Ravi Sah +4MuonRiemannian Optimization

    Nanyang Technological University, Singapore. · Indian Institute of Technology Bombay, India. · University of Sydney, Australia. +1

  54. OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling

    May 8, 2026Yuxuan Lou, Yang YouSpectral NormScaling

    National University of Singapore

  55. Fortifying Time Series: DTW-Certified Robust Anomaly Detection

    May 8, 2026Shijie Liu, Tansu Alpcan, Christopher Leckie +1Time-Series Anomaly DetectionTime Series

    Department of Electrical and Electronic Engineering University of Melbourne, Melbourne, Australia · School of Computing and Information Systems University of Melbourne, Melbourne, Australia

  56. Disagreement-Regularized Importance Sampling for Adversarial Label Corruption

    May 8, 2026Csongor Horváth, Ida-Maria Sintorn, Prashant SinghNoisy LabelsImportance

    Department of Information Technology, Uppsala University, Uppsala, Sweden · Science for Life Laboratory, Uppsala University, Uppsala, Sweden

  57. QuadNorm: Resolution-Robust Normalization for Neural Operators

    May 8, 2026Bum Jun Kim, Makoto Kawano, Yusuke Iwasawa +1Batch NormalizationNeural Operators

    The University of Tokyo, Japan

  58. Sample Complexity of Stochastic Optimization with Integer Variables

    May 8, 2026Hongyu Cheng, Yinghao Zheng, Marco Molinaro +1Stochastic OptimizationOptimal Sample Complexity

    Dept. of Applied Mathematics & Statistics Johns Hopkins University Baltimore, MD 21218 · Microsoft Research (Redmond) · Dept. of Computer Science, PUC-Rio

  59. When and Why SignSGD Outperforms SGD: A Theoretical Study Based on ℓ1\ell_1-norm Lower Bounds

    May 7, 2026Hongyi Tao, Dingzhi Yu, Lijun ZhangStochastic Gradient DescentSignsgd

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China

  60. Optimal Confidence Band for Kernel Gradient Flow Estimator

    May 7, 2026Yuqian Cheng, Zhuo Chen, Qian LinConfidence IntervalsWasserstein Gradient Flows

    Department of Mathematical Sciences Tsinghua University Beijing, 100084, China · Department of Statistics and Data Science2026 Tsinghua University Beijing, 100084, ChinaMay