Optimization Convergence Analysis

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 58

All topics

Quantized Stochastic Primal-Dual Methods for Distributed Optimization under Relaxed Global Geometry
Jun 9, 2026Susmit Sarkar, Abhinav Raghuvanshi, Kushal Chakrabarti +1Stochastic Optimization Primal-Dual Optimization
Indian Institute of Technology Bombay, Mumbai, Maharashtra, India · Tata Consultancy Services Research, Thane, Maharashtra, India
Accelerating SAV-based optimization via randomized low-rank Hessian approximation
Jun 9, 2026Ryo Sagawa, Daisuke Furihata, Yuto MiyatakeOptimization Convergence Analysis Second-Order Optimization
Department of Pure and Applied Mathematics, Graduate School of Information Science and Technology, The University of Osaka · D3 Center, The University of Osaka
Improved Convergence Analysis of Topology Dependence in Decentralized SGD
Jun 8, 2026Yuki Takezawa, Anastasia Koloskova, Sebastian U. StichOptimization Convergence Analysis Decentralized Learning
Toyota Motor Corporation · University of Zurich · CISPA Helmholtz Center for Information Security
Sharp First-Order Lower Bounds for Higher-Order Smooth Nonconvex Optimization
Jun 3, 2026Dongruo ZhouOptimization Convergence Analysis Nonconvex Optimization
Department of Computer Science, Indiana University Bloomington, IN 47408, USA
Finite-Iteration Local Dynamics and Warm Starts for Alternating Power Iteration in Spiked Tensor PCA
Jun 2, 2026Yanjin Xiang, Zhihua ZhangOptimization Convergence Analysis Tensor Decomposition
Peking University
Analytical Evaluation of DCA Convergence Properties for Minimizing Prediction Functions of Gaussian RBF Support Vector Regression
Jun 2, 2026Yohei Kakimoto, Yuto Omae, Hirotaka TakahashiOptimization Convergence Analysis Support Vector Machines
Nihon University · Tokyo City University
FOAM: Frequency and Operator Error-Based Adaptive Damping Method for Reducing Staleness-Oriented Error for Shampoo
Jun 1, 2026Kyunghun Nam, Sumyeong AhnDeep Learning Optimization Optimization Convergence Analysis
KENTECH, Naju, Republic of Korea.
Accelerating Min-Max Optimization via Power-Law Stepsizes
Jun 1, 2026Yue Wu, Weiqiang Zheng, Yang Cai +1Learning Rate Scheduling Last-Iterate Convergence
University of Southern California · Yale University
Global Convergence of a Line-Search Filter Differential Dynamic Programming Method
May 31, 2026Ming Xu, Iman ShamesOptimization Convergence Analysis Constrained Optimization
School of Computer and Communication Sciences EPFL 1015 Lasanne, Switzerland · Department of Electrical and Electronic Engineering University of Melbourne Parkville, VIC, 3010, Australia
A Unifying View of Anchoring via Operator-Side Tikhonov Regularization
May 29, 2026Zihao ChenLast-Iterate Convergence Optimization Convergence Analysis
UC Berkeley
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
May 28, 2026Sharan Vaswani, Yifan Sun, Reza BabanezhadOptimization Convergence Analysis Gradient Descent
1Simon Fraser University · 2Stony Brook University · 3Samsung AI, Montreal.
A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm
May 28, 2026Sakshi Kumari, Shyam Kumar M, Sushmitha POptimization Convergence Analysis Stochastic Optimization Convergence
#Department of Mathematics Indian Institute of Technology Patna Bihta 801 106 India · *Department of Mechanical Engineering Indian Institute of Technology Kharagpur2026 Kharagpur IndiaMay 28
Deep-layer limit and stability analysis of the basic forward-backward-splitting induced network (II): learning problems
May 26, 2026Xuan Lin, Chunlin WuOptimization Convergence Analysis Neural Network Optimization
China Academy of Aerospace System and Innovation, 100088, Beijing, China · School of Mathematical Sciences, Nankai University, 94 Weijin Road, 300071, Tianjin, China
Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning
May 25, 2026Zhaoyu Zhu, Rui Gao, Shuang LiOptimization Convergence Analysis Wasserstein Gradient Flows
Shanghai Jiao Tong University · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing
May 15, 2026Zeji Yi, Chaoyi Pan, Guanya Shi +1Optimization Convergence Analysis Black-Box Optimization
Carnegie Mellon University
When and Why is Optimistic Multiplicative Weights Slow? The Geometry of Energy Dissipation
May 13, 2026John Lazarsfeld, Anas Barakat, Georgios Piliouras +2Nash Equilibrium Last-Iterate Convergence
SUTD · Google DeepMind · NUS CQT +2
Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow
May 10, 2026Lénaïc Chizat, Maria Colombo, Roberto Colombo +1Dynamical Systems Optimization Convergence Analysis
When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$ -norm Lower Bounds
May 7, 2026Hongyi Tao, Dingzhi Yu, Lijun ZhangSign-Based Optimization Optimization Convergence Analysis
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China
A Parameter-Free First-Order Algorithm for Non-Convex Optimization with $\tilde{\mkern1mu O}(ε^{-5/3})$ Global Rate
May 4, 2026Sichao Xiong, Sadok Jerad, Coralia CartisOptimization Convergence Analysis Gradient Descent
Department of Computer Science University of Oxford Oxford, OX1 3QG · Mathematical Institue University of Oxford Oxford, OX2 6GG
Gradient Regularized Newton Boosting Trees with Global Convergence
May 1, 2026Nikita Zozoulenko, Daniel Falkowski, Thomas Cass +1Gradient-Boosted Decision Trees Optimization Convergence Analysis
Department of Mathematics, Imperial College London, UK · Department of Mathematics, Stockholm University, Sweden · School of Computer Science, University of St. Gallen, Switzerland
Man, Machine, and Mathematics
Apr 29, 2026Akshunna S. DograOptimization Convergence Analysis Statistical Learning Theory
The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), USA · EPSRC Centre for Doctoral Training: Mathematics of Random Systems, UK · Center of Mathematical Sciences and Applications, Harvard University, USA +3
Trajectory-Restricted Optimization Conditions and Geometry-Aware Linear Convergence
Apr 18, 2026Faris Chaudhry, Anthea Monod, Keisuke YanoOptimization Convergence Analysis Constrained Optimization
Imperial College London, UK · Institute of Statistical Mathematics, Japan
On the Effectiveness of the z-Transform Method in Quadratic Optimization
Jul 4, 2025Francis BachOptimization Convergence Analysis Gradient Descent Dynamics
SIERRA · Inria - Ecole Normale Sup´erieure · PSL Research University
On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
May 17, 2025Huan Li, Yiming Dong, Zhouchen LinOptimization Convergence Analysis Stochastic Optimization Convergence
Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University, Tianjin, China. · National Key Lab of General AI, School of Intelligence Science and Technology, Peking University, Beijing, China.
Negative Stepsizes Make Gradient-Descent-Ascent Converge
May 2, 2025Henry Shugart, Jason M. AltschulerLearning Rate Scheduling Optimization Convergence Analysis
UPenn
Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes
Jun 7, 2024Si Yi Meng, Antonio Orvieto, Daniel Yiming Cao +1Dynamical Systems Logistic Regression
Department of Computer Science Cornell University Ithaca, NY, USA · ELLIS Institute Tübingen MPI for Intelligent Systems Tübingen AI Center, Germany
Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients
Mar 28, 2024Johannes Müller, Semih Çaycı, Guido MontúfarPolicy Gradient Optimization Convergence Analysis
Department of Mathematics, RWTH Aachen University, Aachen, 52062, Germany · Departments of Mathematics and Statistics & Data Science, University of California, Los Angeles, 90095, USA · Max Planck Institute for Mathematics in the Sciences, Leipzig, 04103, Germany
Silver Rate Is (Almost) Optimal for Gradient Descent
Date pendingYuhan Ye, Kaizhao LiuLearning Rate Scheduling Optimization Convergence Analysis
MIT

Jun 9, 2026, math.OCJ/K move · Enter open · S save

Quantized Stochastic Primal-Dual Methods for Distributed Optimization under Relaxed Global Geometry

Susmit Sarkar, Abhinav Raghuvanshi, Kushal Chakrabarti, Mayank Baranwal

Indian Institute of Technology Bombay, Mumbai, Maharashtra, India · Tata Consultancy Services Research, Thane, Maharashtra, India

Stochastic Optimization Primal-Dual Optimization Optimization Convergence Analysis Distributed Optimization

Open paper AskPDF arXiv

We study distributed optimization with stochastic gradients and finite-bit communication modeled by random (unbiased) quantization. We propose q-PDGD, a quantized stochastic primal-dual method, and analyze it under relaxed global geometry. Under restricted secant inequality (RSI), a constant step-size yields linear contraction to an explicit neighborhood determined by gradient noise, quantization distortion, and network connectivity, while a diminishing step-size achieves O(1/k) convergence without shared-minimizer assumptions. Under Polyak-Lojasiewicz (PL) inequality, we obtain linear-to-neighborhood convergence in the same stochastic quantized setting. Our results match the best-known centralized stochastic rates in oracle complexity, and are supported by experiments demonstrating the predicted tradeoffs between quantization level, step-size choice, and graph structure.

Momentum

Related topics

Latest papers 58