Nonconvex Optimization

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles
Jul 10, 2026Jiseok Chae, Donghwan KimLearning Rate Scheduling Optimization Convergence Analysis
Natural Science Research Institute KAIST Daejeon, Republic of Korea · Department of Mathematical Sciences KAIST Daejeon, Republic of Korea
Nonconvex Composite Functional Constraints via First-Order Augmented Lagrangian Methods under Local Regularity
Jul 9, 2026Linglingzhi Zhu, Jiajin LiPrimal-Dual Optimization Nonconvex Optimization
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, USA. · Sauder School of Business, University of British Columbia, Vancouver, BC, Canada.
Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates
Jul 8, 2026Jianing Liu, Dong H. ZhangGauss-Newton Optimization Riemannian Optimization
State Key Laboratory of Chemical Reaction Dynamics and Department of Chemical Physics, University of Science and Technology of China, Hefei, 230026, China · State Key Laboratory of Chemical Reaction Dynamics, Dalian Institute of Chemical Physics, Chinese Academy of Sciences, Dalian, 116023, China. · University of Chinese Academy of Sciences, Beijing, 100049, China +1
On the Condition Number Upper Bound of the L-BFGS Inverse Hessian Approximation Matrix with a Two-Sided Geometric Envelope Safeguarding Mechanism
Jul 7, 2026Don LiSecond-Order Optimization Nonconvex Optimization
Department of Mathematics & Statistics Portland State University Portland, OR 97201, USA
Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusions
Jul 6, 2026Kyohei Suzuki, Konstantinos SlavakisTemporal-Difference Learning Nonconvex Optimization
Department of Information and Communications Engineering, Institute of Science Tokyo, Yokohama, 226-8501, Japan
Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimization
Jul 2, 2026Siyuan Zhang, Nachuan Xiao, Xin LiuStochastic Optimization Nonconvex Optimization
Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics
Jun 30, 2026Yudong Wei, Liang Zhang, Bingcong Li +1Low-Rank Matrix Decomposition Nonconvex Optimization
Department of Mathematics, ETH Zurich, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland
A Fast Convergent Algorithm for Solving Non-convex Partially-Decoupled Generalized Nash Equilibrium Problems
Jun 26, 2026Bennet Outland, Vishala AryaGame Theory Nonconvex Optimization
Smead Aerospace Engineering Sciences Department, 3775 Discovery Drive, Boulder, CO, USA
Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization
Jun 26, 2026Shuang Li, Zhihui Zhu, Qiuwei LiOptimization Convergence Analysis Distributed Optimization
Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa 50014, USA · Department of Computer Science and Engineering, Ohio State University, Columbus, Ohio 43210, USA · DAMO Academy, Alibaba Group US, Bellevue, Washington 98004, USA
Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization
Jun 26, 2026Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef MrouehMaximum Mean Discrepancy Nonconvex Optimization
CMAP, CNRS, Ecole Polytechnique, IP Paris · CERMICS, CNRS, ENPC, IP Paris · IBM Research
Finding Stationary Points by Comparisons
Jun 25, 2026Helin Wang, Chenyi Zhang, Xiwen Tao +2Black-Box Optimization Zeroth-Order Optimization
School of Electronics Engineering and Computer Science, Peking University · Center on Frontiers of Computing Studies, Peking University · Computer Science Department, Stanford University +1
Adam Converges in Nonsmooth Nonconvex Optimization
Jun 21, 2026Zijian LiuHeavy-Tailed Noise Nonconvex Optimization
Department of Technology, Operations, and Statistics Leonard N. Stern School of Business New York University New York, NY 10012, USA
On the Oracle Complexity of Interpolation-Based Gradient Descent
Jun 18, 2026Dongmin Lee, William Lu, Anuran MakurOracle Complexity Gradient Descent
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and the Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
Semiglobal Input-Delay Tolerance Algorithm for Distributed Nonconvex Optimization of Networked Nonlinear Systems
Jun 18, 2026Jing-Zhe Xu, Zhi-Wei Liu, Ming-Feng Ge +2Dynamical Systems Optimization Convergence Analysis
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · School of Mechanical Engineering and Electronic Information, China University of Geosciences, Wuhan 430074, China
Low-Rank Tensor Completion Based on Fractional Regularization with Ky Fan p-k Norm
Jun 17, 2026Shan Fan, Feng Zhang, Jianjun Wang +2Tensor Completion Nonconvex Optimization
School of Mathematics and Statistics, Southwest University, Chongqing 400715, China · School of Mathematical Sciences/Research Center for Image and Vision Computing, University of Electronic Science and Technology of China, Chengdu 611731, China · Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology, Shenzhen 518055, China
The Complexity of Min-Max Optimization for Quadratic Polynomials
Jun 15, 2026Martino Bernasconi, Matteo Castiglioni, Andrea Celli +1Min-Max Optimization Nonconvex Optimization
Bocconi University · Politecnico di Milano · University of Oxford
When to use what Schatten- $p$ norm in deep learning?
Jun 13, 2026Thomas PethickDeep Learning Optimization Muon Optimizer
Operator Calculus for Population-Based Optimization: Modular Convergence and Finite-Population Guarantees
Jun 12, 2026Pekka Malo, Lauri Viitasaari, Patrik Nummi +3Evolutionary Optimization Optimization Convergence Analysis
Aalto University, Department of Information and Service Management, Finland. · Aalto University, Department of Information and Communications Engineering, Finland. · Indian Institute of Management Ahmedabad, Operations and Decision Sciences, India. +1
Sharp First-Order Lower Bounds for Higher-Order Smooth Nonconvex Optimization
Jun 3, 2026Dongruo ZhouOptimization Convergence Analysis Nonconvex Optimization
Department of Computer Science, Indiana University Bloomington, IN 47408, USA
From Non-Convex to Strongly Convex: Curvature-Adaptive FTPL for Online Optimization
Jun 1, 2026Moses Charikar, Chirag Pabbaraju, Ambuj TewariOnline Convex Optimization Nonconvex Optimization
Stanford University. · University of Michigan, Ann Arbor.
Learning-based Directed Graph Abstraction of Combinatorial Spaces for Order-Preserving Search in Mixed-Combinatorial Nonlinear Optimization
May 31, 2026Gishnu Madhu, Feng Liu, Souma ChowdhuryGraph Structure Learning Graph Neural Networks
University at Buffalo Buffalo, New York, 14260
GLENS: Global Search via Learning from Solver Iterates with Diffusion Models
May 29, 2026Anjian Li, Bartolomeo Stellato, Ryne BeesonLearning to Optimize Diffusion Sampling
Department of Electrical and Computer Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Operations Research and Financial Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, 08544, NJ, USA.
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
May 28, 2026Sharan Vaswani, Yifan Sun, Reza BabanezhadOptimization Convergence Analysis Gradient Descent
1Simon Fraser University · 2Stony Brook University · 3Samsung AI, Montreal.
Caspar: CUDA Accelerator for Symbolic Programming with Adaptive Reordering
May 28, 2026Emil Martens, Aaron Miller, Matias Varnum +1Bundle Adjustment GPU Kernel Optimization
Norwegian University of Science and Technology · Skydio
MoSSP: A Momentum-Based Single-Loop Stochastic Penalty Method for Nonconvex Constrained DC-Regularized Optimization
May 28, 2026Luxuan Li, Chunfeng Cui, Xiao WangStochastic Optimization Nonconvex Stochastic Optimization
School of Mathematical Sciences, Beihang University, Beijing 100191, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China.
Designing Active Tether-Net Systems for Space Debris Capture with Graph-Learning-Aided Mixed-Combinatorial Optimization
May 27, 2026Feng Liu, Achira Boonrath, Gishnu Madhu +2Metaheuristic Optimization Nonconvex Optimization
University at Buffalo Buffalo, New York, 14260
Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
May 24, 2026Huangyu Xu, Jingqin Yang, Qianqian Xu +1Nonconvex Optimization
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · IIIS, Tsinghua University, Beijing, China +3
Zeroth-Order Nonconvex Nonsmooth Optimization with Heavy-Tailed Noise
May 23, 2026Zhuanghua Liu, Luo LuoStochastic Optimization Gradient Clipping
RA-DCA: A Randomized Active-Set DCA for Directional Stationarity in Max-Structured DC Programs
May 22, 2026Yi-Shuai NiuNonconvex Optimization
Beijing Institute of Mathematical Sciences and Applications (BIMSA), Beijing, China
Convex Compositional Reasoning Models
May 22, 2026Meir Roketlishvili, Semyon Semenov, Maksim Bobrin +7Compositional Reasoning Energy-Based Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Applied AI Institute, Computational Imaging Lab · AXXX

Jul 10, 2026, cs.LGJ/K move · Enter open · S save

Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles

Jiseok Chae, Donghwan Kim

Natural Science Research Institute KAIST Daejeon, Republic of Korea · Department of Mathematical Sciences KAIST Daejeon, Republic of Korea

Learning Rate Scheduling Optimization Convergence Analysis Nonconvex Optimization Gradient Descent Dynamics Stochastic Gradient Descent

Open paper AskPDF arXiv

Schedule-Free methods have attracted growing interest for alleviating the burden of designing and tuning a learning rate scheduler, while matching and sometimes even outperforming optimizers with tuned schedulers. Despite their strong empirical results, their convergence theory in nonconvex optimization, where modern machine learning objectives typically arise, has remained largely unexplored. In this paper, we provide worst-case analyses of Schedule-Free gradient descent and Schedule-Free stochastic gradient descent, in their standard form and without auxiliary modifications or restrictive conditions, for smooth but possibly nonconvex objectives. Based on a Lyapunov analysis derived from the continuous-time limiting ordinary differential equation associated with these methods, we show that Schedule-Free gradient descent and Schedule-Free stochastic gradient descent achieve the optimal worst-case convergence rates attainable among first-order methods. We further formulate Schedule-Free gradient descent as a nonautonomous dynamical system and prove strict-saddle avoidance under an arbitrarily small one-time perturbation. These theoretical results provide a better understanding of the strong performance that Schedule-Free methods demonstrate.

Momentum

Related topics

Latest papers 96