Curvature-Aware Optimization

Latest papers

21 papers1 paper in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM
Oct 14, 2026Alessia Rossi, Felipe Campelo, Gabriel L. OliveiraCurvature-Aware Optimization Sharpness-Aware Minimization
School of Engineering Mathematics and Technology, University of Bristol, Bristol, Lovelace Building, Ada, Bristol, BS8 1TW, UK
Curvature-Conditioned Multiscale Momentum with Sphere Constraints for LLM Pretraining
Aug 28, 2026Shuchen Zhu, Yuxin Fang, Mingze Wang +1Curvature-Aware Optimization Momentum Methods
ByteDance Seed · Peking University
Curvature-Aware Zeroth-Order Optimization for Memory-Efficient Test-Time Adaptation
Aug 12, 2026Junming Zhang, Shuyu Yin, Peilin Liu +2On-Device Training Test-Time Adaptation
Shanghai Jiao Tong University, China
MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning
Aug 5, 2026Tongle Wu, Huanyu Dong, Ying Sun +1Language Model Pretraining Muon Optimizer
School of Electrical Engineering and Computer Science, The Pennsylvania State University, University Park, PA, USA · Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China
Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam
Jul 4, 2026Ashmitha R, Jörg FrochteGradient Descent Adaptive Gradient Methods
Department of Artificial Intelligence and Data Science, Sri Ramakrishna Engineering College, Anna University, Coimbatore 641022, Tamil Nadu, India · Interdisciplinary Institute for Applied AI and Data Science Ruhr (AKIS), Department of Electrical Engineering and Computer Science, Bochum University of Applied Sciences, Am Hochschulcampus 1, 44801 Bochum, Germany
Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedules
Jun 29, 2026Muhammad Hamza, Ayush GoelLearning Rate Scheduling Stochastic Optimization Convergence
Indian Institute of Technology Kharagpur · Indian Institute of Technology Kharagpur (IIT KGP)
Why Muon Outperforms Adam: A Curvature Perspective
Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Loss Landscape Sharpness Neural Network Optimization
National University of Singapore · Yale University · University of Minnesota
From Non-Convex to Strongly Convex: Curvature-Adaptive FTPL for Online Optimization
Jun 1, 2026Moses Charikar, Chirag Pabbaraju, Ambuj TewariOnline Convex Optimization Follow-the-Perturbed-Leader
Stanford University. · University of Michigan, Ann Arbor.
Exploiting weight-space symmetries for approximating curvature
May 30, 2026Artem Artemev, Rui Xia, Benjamin M. Boyd +4Neural Network Optimization Second-Order Optimization
MediaTek, Cambridge, UK · University of Cambridge, Cambridge, UK · Vector Institute, Toronto, Canada
Towards Efficient LLMs Annealing with Principled Sample Selection
May 29, 2026Yuanjian Xu, Jianing Hao, Wanbo Zhang +2Language Model Pretraining Efficient Language Model Training
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Microsoft Research Asia, Beijing, China · Fudan University, Shanghai, China
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
Mila, Université de Montréal · Samsung – SAIL Montreal
Accelerated Gradient Descent for Faster Convergence with Minimal Overhead
May 15, 2026Manuel Graca, L. Miguel Silveira, Arlindo Oliveira +1Deep Learning Optimization Gradient Descent
IST, ULisboa Lisbon, Portugal · Old Dominion University Norfolk, VA USA
Curvature Beyond Positivity: Greedy Guarantees for Arbitrary Submodular Functions
May 8, 2026Yixin Chen, Alan KuhnleSubmodular Optimization Approximation Algorithms
Department of Computer Science & Engineering, Texas A&M University
Layerwise LQR for Geometry-Aware Optimization of Deep Networks
May 5, 2026Simon Dufort-Labbé, Pierre-Luc Bacon, Razvan Pascanu +2Deep Learning Optimization Second-Order Optimization
Mila, Université de Montréal · Samsung – SAIL Montreal
Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks
Apr 16, 2026Kang An, Chenhao Si, Shiqian Ma +1Neural Network Optimization Second-Order Optimization
School of Data Science The Chinese University of Hong Kong, Shenzhen Shenzhen, China
Efficient Bilevel Optimization with KFAC-Based Hypergradients
Mar 31, 2026Disen Liao, Felix Dangel, Yaoliang YuKronecker-Factored Approximation Bilevel Optimization
University of Waterloo · Vector Institute
Second-Order, First-Class: A Composable Stack for Curvature-Aware Training
Mar 26, 2026Mikalai Korbit, Mario ZanonCurvature-Aware Optimization Deep Learning Optimization
IMT School for Advanced Studies Lucca Lucca, Italy
Vision Transformers that Never Stop Learning
Mar 8, 2026Caihao Sun, Mingqi Yuan, Shiyuan Wang +1Curvature-Aware Optimization Loss of Plasticity
University of Hong Kong · The Hong Kong Polytechnic University · Technische Universität Dresden +1
Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
Mar 1, 2026Theophilus Amaefuna, Hitesh Vaidya, Anshuman Chhabra +1LLM Pruning LLM Compression
Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, Florida, USA
MSINO: Curvature-Aware Sobolev Optimization for Manifold Neural Networks
Feb 26, 2026Suresan ParethRiemannian Neural Networks Neural Network Optimization
Second-Order Optimization with Asynchronous Curvature
Oct 17, 2025El Mahdi Chayti, Martin JaggiSecond-Order Optimization Curvature-Aware Optimization
Machine Learning and Optimization Laboratory EPFL

Oct 14, 2026J/K move · Enter open · S save

One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM

Alessia Rossi, Felipe Campelo, Gabriel L. Oliveira

School of Engineering Mathematics and Technology, University of Bristol, Bristol, Lovelace Building, Ada, Bristol, BS8 1TW, UK

Curvature-Aware Optimization Sharpness-Aware Minimization Neural Network Optimization Neural Network Generalization

Open paper AskPDF arXiv

Poor generalisation in deep neural networks is frequently associated with convergence to sharp minima. Sharpness-Aware Minimisation (SAM) improves generalisation of deep neural networks by minimising a combination of the training loss and the local sharpness of the loss landscape, biasing convergence towards flat minima. However, SAM relies on a fixed perturbation radius

ρ

that ignores local curvature, leading to inefficient behaviour: the perturbation may be too small to escape poor basins defined by sharp local minima, or too large to allow convergence to solutions with good loss-sharpness tradeoff. We reinterpret

ρ

as the intensity of the perturbation signal and propose Curvature-Aware Adaptive Perturbation for SAM (CAP-SAM), a geometry-aware method that modulates the perturbation radius based on a sharpness ratio, enabling a more effective search for robust minima on the loss landscape. We prove that CAP-SAM preserves the same convergence rate as standard SAM in non-convex stochastic settings. Extensive experiments indicate that our approach consistently outperforms SAM and its most prominent variants across diverse network architectures on CIFAR-10, CIFAR-100, and ImageNet.