Stochastic Optimization

Ask about this topic

Overview

Stochastic optimization seeks good decisions when exact objectives or gradients are unavailable, observations are noisy, or the environment changes, making it central to machine learning, planning, and resource allocation. Common approaches include stochastic gradient descent and adaptive optimizers, variance reduction, mirror and proximal methods, and online or bandit algorithms; robust and risk-sensitive methods address heavy-tailed noise and costly failures. Researchers evaluate these methods through convergence, sample-complexity, and regret guarantees, as well as experiments in deep-model training, constrained planning, and simulation. A continuing challenge is to make guarantees hold under realistic noise, dependence, and changing data while keeping computation, communication, and tuning costs manageable.

Latest papers

CardsList

209 papers34 papers in the last four weeks, up 70% on the four weeks before. 0.2% of all new papers.

  1. Agent-Based Transport Model Calibration via Online Stochastic Batch Optimization

    Oct 23, 2026Abdelkader Dib, Milos BalacAgent-Based ModelingStochastic Optimization

    Center for Sustainable Future Mobility (CSFM), ETH Zürich, Switzerland

  2. When Momentum Helps Polyak--Ruppert Averaging

    Oct 20, 2026Alexandra Suvroikova, Darina Dvinskikh, Yuriy Dorn +1Stochastic OptimizationStochastic Optimization Convergence

    Independent researcher · HSE University · MSU AI Institute +1

  3. A Statistical Viewpoint on Adam, Shampoo and Muon

    Oct 17, 2026Tetiana Parshakova, Robert M. Gower, Paulo OrensteinStochastic OptimizationAdaptive Gradient Methods

  4. Near-Optimal Regret for Semi-Bandit Learning of Monotone Stochastic Problems

    Oct 17, 2026Zhengjia Zhuo, Viswanath NagarajanStochastic OptimizationCombinatorial Semi-Bandits

    Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.

  5. A general recipe for data-adaptive anytime-valid envelopes for iterative algorithms

    Oct 16, 2026Soham Bonnerjee, Aaditya RamdasStochastic Optimization

    University of Pennsylvania · Stanford University

  6. Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems

    Oct 16, 2026Mohsen Amidzadeh, Lauri Viitasaari, Mario Di FrancescoStochastic Optimization

    Department of Computer Science, Aalto University, Finland · Department of Information and Service Management, Aalto University, Finland

  7. When is a better LLM worth paying for? Stochastic Optimization with Switching Mixed-Fidelity Oracles

    Oct 15, 2026Swati Gupta, Tianjiao LiStochastic OptimizationMulti-Fidelity Optimization

    MIT Sloan School of Management · IBM Research

  8. Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent

    Oct 14, 2026Zijun Chen, Zihan ZhangStochastic OptimizationLast-Iterate Convergence

    Department of Computer Science and Engineering, Hong Kong University of Science and Technology

  9. Optimization for One, Feasibility for Most: Decoupling Optimization from Feasibility in Chance-Constrained Programs

    Oct 13, 2026Justin Dumouchelle, Shunyu Yao, Beste Basciftci +1Chance-Constrained OptimizationStochastic Optimization

    University of Calgary · University of Iowa

  10. Steering the Fidelity in State Preparation on Noisy Bosonic Systems with Risk-Seeking Optimization

    Oct 13, 2026Riccardo Piantoni, Riccardo Pellini, Maurizio Ferrari Dacrema +1Quantum State PreparationStochastic Optimization

    Politecnico di Milano, Italy

  11. Principled Design of Expected Improvement for Continuous Simulation Optimization

    Oct 13, 2026Songhao Wang, Jianglin Xia, Haowei Wang +1Stochastic OptimizationStochastic Optimization Convergence

    College of Business, Southern University of Science and Technology, Shenzhen, CHINA · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore, SINGAPORE

  12. Bayesian Optimization of CVaR with Unknown Hyperparameters

    Oct 13, 2026Valentin Margraf, Jonas Hanselle, Viktor Bengs +3Bayesian OptimizationConditional Value-at-Risk

    LMU Munich · Munich Center for Machine Learning (MCML) · German Research Center for Artificial Intelligence (DFKI) +3

  13. Risk-Averse Inference-Time Diffusion Model Alignment with Finite-Sample Guarantees

    Oct 13, 2026Yuchen Liang, Xian YuDiffusion Model AlignmentStochastic Optimization

    Indiana University Indianapolis · The Ohio State University

  14. Near-Optimal Regret Bounds for Monotone Stochastic Optimization under Semi-Bandit Feedback

    Oct 12, 2026Tao Hu, Rohan GhugeStochastic OptimizationMonotonicity Constraints

    Department of Information, Risk, and Operations Management, University of Texas at Austin, Austin, USA.

  15. Minimax Optimality of Shampoo for Nonstationary Stochastic Optimization

    Oct 12, 2026Abir Sarkar, Nirmalya Mandal, Martin T. WellsStochastic OptimizationNonconvex Stochastic Optimization

    Department of Statistics and Data Science, Cornell University · Department of Statistics, NC State University

  16. Efficient Preconditioned Optimization Methods with Frequent Directions

    Oct 12, 2026Sifan Yang, Yuanyu Wan, Yibo Wang +1Adaptive Gradient MethodsOnline Convex Optimization

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Software Technology, Zhejiang University, Ningbo, China +1

  17. Partially Informed Stochastic Optimization via Zeroth-Order Residual Tracking

    Oct 9, 2026El Mahdi Chayti, Alireza Mirrokni, Martin JaggiStochastic OptimizationNonconvex Stochastic Optimization

    Machine Learning and Optimization Laboratory (MLO), EPFL

  18. Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates

    Oct 9, 2026El Mahdi Chayti, Alireza Mirrokni, Martin JaggiLMO MethodsMulti-Objective Optimization

    Machine Learning and Optimization Laboratory (MLO), EPFL

  19. Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets

    Oct 9, 2026Sebastian PokuttaLMO MethodsOracle Complexity

    Zuse Institute Berlin and TU Berlin

  20. Optimal Oracle Complexity for Finite-Sum Monotone Inclusions

    Oct 4, 2026Qihao ZhouStochastic OptimizationOracle Complexity

    Independent Researcher.

  21. Chance-Constrained Bi-Objective Evolutionary Optimization for the Open-Pit Mining Operational Planning Problem

    Oct 3, 2026Ishara Hewa Pathiranage, Aneta NeumannStochastic OptimizationChance-Constrained Optimization

    Machine Learning and Optimisation, School of Computer Science and Information Technology, Adelaide University, Adelaide, Australia

  22. Probabilistic Algorithms for Ising Machines from Optimization to Generative AI

    Oct 2, 2026Corentin Delacour, Xiuqi Zhang, Abdelrahman S. Abdelrahman +4Stochastic OptimizationMetaheuristic Optimization

    Department of Electrical and Computer Engineering, University of California, Santa Barbara, Santa Barbara, CA 93106, USA · Electrical Engineering Department, King Fahd University of Petroleum & Minerals (KFUPM), Dhahran 31261, Saudi Arabia

  23. Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance

    Oct 1, 2026Arda Fazla, Antesh Upadhyay, Ege C. Kaya +2Stochastic OptimizationLanguage Model Pretraining

    Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907, USA

  24. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic OptimizationLast-Iterate Convergence

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  25. Optimal Momentum Methods for Stochastic Multilevel Compositional Optimization

    Oct 1, 2026Wei Jiang, Rui Yan, Sifan Yang +3Stochastic OptimizationMomentum Methods

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Software Technology, Zhejiang University, Ningbo, China

  26. The Row Normalization Puzzle in Muon

    Sep 30, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationOptimization Convergence Analysis

    Department of Industrial Engineering and Operations Research Columbia University

  27. Adaptive Rotation for iSOMA: Geometry, Benchmarking, and Noise Robustness in Variational Quantum Objectives

    Sep 29, 2026Vojtěch Novák, Ivan ZelinkaStochastic OptimizationBlack-Box Optimization

    Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Ostrava, Czech Republic. · IT4Innovations National Supercomputing Center, VSB-Technical University of Ostrava, Ostrava, 708 00, Czech Republic. · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Klaipeda, Lithuania.

  28. Markovian Nonconvex ADMM for Reinforcement Learning: Bellman-Resolvent Stability Beyond Smooth Blocks

    Sep 29, 2026Zhaojun PengAlternating Direction Method of MultipliersRL Control

  29. Stochastic Heavy Ball with Polyak Step Size and Armijo Line Search: A General Convergence Analysis

    Sep 29, 2026Jiawei Zhang, Qitan Shi, Yuantao GuStochastic OptimizationStochastic Optimization Convergence

    Tsinghua University

  30. Convex Optimization Is Free When Accuracy Is Expensive

    Sep 28, 2026Arthur Paing, Arthur JacotStochastic OptimizationGradient Descent

    Ecole polytechnique Palaiseau, France · Courant Institute, NYU New York, USA