Stochastic Optimization

Ask about this topic

Overview

Stochastic optimization seeks good decisions when exact objectives or gradients are unavailable, observations are noisy, or the environment changes, making it central to machine learning, planning, and resource allocation. Common approaches include stochastic gradient descent and adaptive optimizers, variance reduction, mirror and proximal methods, and online or bandit algorithms; robust and risk-sensitive methods address heavy-tailed noise and costly failures. Researchers evaluate these methods through convergence, sample-complexity, and regret guarantees, as well as experiments in deep-model training, constrained planning, and simulation. A continuing challenge is to make guarantees hold under realistic noise, dependence, and changing data while keeping computation, communication, and tuning costs manageable.

Latest papers

CardsList

210 papers34 papers in the last four weeks, up 70% on the four weeks before. 0.2% of all new papers.

  1. The Privacy Price of Tail-Risk Learning: Effective Tail Sample Size in Differentially Private CVaR Optimization

    May 15, 2026El Mustapha MansouriStochastic OptimizationDifferential Privacy

    School of Engineering, Institute of Science Tokyo, Yokohama, Kanagawa 226-8501, Japan

  2. Stochastic Non-Smooth Convex Optimization with Unbounded Gradients

    May 15, 2026Dmitry KovalevStochastic OptimizationGradient Clipping

    Yandex Research

  3. Unified High-Probability Analysis of Stochastic Variance-Reduced Estimation

    May 14, 2026Zhankun Luo, Antesh Upadhyay, M. Berk Sahin +3Stochastic OptimizationVariance Reduction

    School of Electrical and Computer Engineering and Purdue University West Lafayette, IN 47907 · Department of Computer Science Purdue University West Lafayette, IN 47907

  4. Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe

    May 14, 2026El Mahdi Chayti, Mohamad FakhouriStochastic OptimizationMomentum Methods

    Machine Learning and Optimization Laboratory (MLO) EPFL, Switzerland

  5. Scalable Solution of the Stochastic Multi-path Traveling Salesman Problem via Neural Networks

    May 14, 2026Xiaochen Chou, Ludovica Di Marco, Enza MessinaNeural Surrogate ModelingStochastic Optimization

    Department of Informatics, Systems and Communication, University of Milano-Bicocca, Viale Sarca 336, Milan, 20125, Italy.

  6. Stochastic global optimization of continuous functions via random walks on Grassmannians

    May 13, 2026Kartik Gupta, Stephen D. Miller, Pradeep Ravikumar +1Stochastic OptimizationSubspace Optimization

  7. Regret Equals Covariance: A Closed-Form Characterization for Stochastic Optimization

    May 13, 2026Irene AldridgeStochastic OptimizationRegret Minimization

  8. Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization

    May 13, 2026Yaxin Yu, Long Chen, Minfu FengStochastic OptimizationMomentum Methods

    School of Mathematics Sichuan University Chengdu, China · Department of Mathematics University of California, Irvine Irvine, CA, USA

  9. NeuroRisk: Physics-Informed Neural Optimization for Risk-Aware Traffic Engineering

    May 13, 2026Yingming Mao, Ximeng Liu, Jingyi Cheng +8Stochastic OptimizationNeural Network Optimization

    Xi’an Jiaotong University · Shanghai Innovation Institute · Shanghai Jiao Tong University +2

  10. Adaptive Multi-Round Allocation with Stochastic Arrivals

    May 12, 2026Yuqi Pan, Davin Choo, Haichuan Wang +3Stochastic OptimizationSequential Decision Making

    Harvard University · University of Witwatersrand · Wits Health Consortium +1

  11. Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters

    May 12, 2026Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1Stochastic OptimizationGradient Clipping

    MBZUAI · University of Southern California

  12. Muon is Not That Special: Random or Inverted Spectra Work Just as Well

    May 11, 2026Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6Stochastic OptimizationDeep Learning Optimization

    University of Cambridge · University of Tübingen · University of Oxford

  13. Optimistic Dual Averaging Unifies Modern Optimizers

    May 11, 2026Thomas Pethick, Wanyun Xie, Roman Machacek +1Stochastic OptimizationDeep Learning Optimization

    Independent Researcher · EPFL (LIONS) · University of Bern

  14. Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates

    May 10, 2026Anik Kumar Paul, Nibedita Roy, Nagesh Talagani +3Stochastic OptimizationSign-Based Optimization

    Computer Science and Automation, Indian Institute of Science, Bengaluru · Edwardson School of Industrial Engineering, Purdue University · Department of Computer Science, IMT Atlantique

  15. Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction

    May 9, 2026Zhirayr Tovmasyan, Artavazd Maranjyan, Peter RichtárikStochastic OptimizationDistributed Optimization

    KAUST

  16. Convergent Stochastic Training of Attention and Understanding LoRA

    May 8, 2026Zhengkai Sun, Dibyakanti Kumar, Alejandro F Frangi +2Stochastic OptimizationLow-Rank Adaptation

  17. Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling

    May 8, 2026Manojlo Vukovic, Dusan JakoveticStochastic OptimizationNonconvex Stochastic Optimization

    Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia · Faculty of Sciences, University of Novi Sad, Novi Sad, Serbia

  18. Sample Complexity of Stochastic Optimization with Integer Variables

    May 8, 2026Hongyu Cheng, Yinghao Zheng, Marco Molinaro +1Stochastic OptimizationSample Complexity

    Dept. of Applied Mathematics & Statistics Johns Hopkins University Baltimore, MD 21218 · Microsoft Research (Redmond) · Dept. of Computer Science, PUC-Rio

  19. Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition

    May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Stochastic OptimizationMomentum Methods

    MBZUAI · Penn State University · Georgia Institute of Technology +1

  20. Learning to Cut: Reinforcement Learning for Benders Decomposition

    May 7, 2026Haochen Cai, Xian YuStochastic Optimization

    Department of Integrated Systems Engineering The Ohio State University Columbus, OH, USA, 43210 · Department of Integrated Systems Engineering The Ohio State UniversityMay Columbus, OH, USA, 43210

  21. On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows

    May 7, 2026Xinglin Wang, Zishen Liu, Shaoxiong Feng +9Stochastic OptimizationCost-Aware Inference

    School of Computer Science, Beijing Institute of Technology · Xiaohongshu Inc

  22. Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization

    May 5, 2026Sharan Sahu, Abir Sarkar, Cameron J. Hogan +1Stochastic OptimizationStochastic Optimization Convergence

    Department of Statistics and Data Science Cornell University

  23. Cost-Aware Learning

    Apr 30, 2026Clara Mohri, Amir Globerson, Haim Kaplan +2Stochastic OptimizationRL for Language Models

    Kempner Institute · Harvard University · Google Research +1

  24. Variational Neural Belief Parameterizations for Robust Dexterous Grasping under Multimodal Uncertainty

    Apr 28, 2026Clinton Enwerem, Shreya Kalyanaraman, John S. Baras +1Belief-Space PlanningStochastic Optimization

    Department of Electrical & Computer Engineering and Institute for Systems Research, University of Maryland, College Park, MD, USA

  25. Categorical Optimization with Bayesian Anchored Latent Trust Regions for Structural Design under High-Dimensional Uncertainty

    Apr 28, 2026Zhangyong Liang, Jie Hou, Huanhuan Gao +1Stochastic OptimizationSurrogate-Assisted Optimization

    National Center for Applied Mathematics, Tianjin University, Weijin Road 92, 300072, Tianjin, China · School of Mechanical Engineering, Northwestern Polytechnical University, Youyixi Road 127, 710072, Xi’an, China · School of Mechanical and Aerospace Engineering, Jilin University, Renmin Street 5988, 130025, Changchun, China +1

  26. Stochastic simultaneous optimistic optimization

    Apr 27, 2026Michal Valko, Alexandra Carpentier, Rémi MunosStochastic OptimizationMulti-Armed Bandits

    INRIA Lille - Nord Europe, SequeL team, 40 avenue Halley 59650, Villeneuve d’Ascq, France · Statistical Laboratory, CMS, Wilberforce Road, CB3 0WB, University of Cambridge, United Kingdom

  27. Robust Out-of-Distribution Stochastic Optimization

    Apr 22, 2026Xianyu Li, Huan Xu, Xiaolin Huang +1Stochastic OptimizationOOD Generalization

    Department of Automation, Tsinghua University · Antai College of Economics & Management, Shanghai Jiao Tong University · Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University

  28. TESO: Online Tracking of Essential Matrix by Stochastic Optimization

    Apr 21, 2026Jaroslav Moravec, Radim Šára, Akihiro SugimotoStochastic OptimizationStereo Vision

    Czech Technical University · in Prague, Czech Republic · National Institute of Informatics +1

  29. Towards Fully Parameter-Free Stochastic Optimization: Grid Search with Self-Bounding Analysis

    Apr 18, 2026Yuheng Zhao, Yu-Hu Yan, Amit Attia +3Stochastic OptimizationNonconvex Stochastic Optimization

    National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Tel Aviv University

  30. Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance

    Apr 17, 2026Arda Fazla, Ege C. Kaya, Antesh Upadhyay +1Stochastic OptimizationOracle Complexity

    School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA.