Random Reshuffling

Follow this topic Ask about this topic

Latest papers

12 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Sampling and Curvature in First-Exit Comparisons for Momentum SGD
Oct 18, 2026Haiyi LiSGD with Momentum Stochastic Gradient Descent
Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent
Oct 14, 2026Zijun Chen, Zihan ZhangStochastic Optimization Last-Iterate Convergence
Department of Computer Science and Engineering, Hong Kong University of Science and Technology
Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
Oct 7, 2026Murat Bilgehan Ertan, Marten van DijkPrivacy Accounting Random Reshuffling
Affiliated with Vrije Universiteit Amsterdam.
Stochastic Optimization Under Power-Law Spectra: Tight Bounds and Shuffling Analysis
Sep 28, 2026Christopher De Sa, Thomas Dybdahl Ahle, Yaroslav Bulatov +1Random Reshuffling Stochastic Optimization Convergence
Together.AI, Cornell University · Normal Computing · Stanford University
Centered Permutation Prefixes for SGD with Random Reshuffling: Sharp Rates, Hölder Geometry, and Composite Proximal Extensions
Sep 4, 2026Jiaxiang LiRandom Reshuffling Stochastic Optimization Convergence
Grado Department of Industrial and Systems Engineering, Virginia Tech.
Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization
Aug 6, 2026Grigory MalinovskyByzantine-Robust Federated Learning Communication-Efficient Distributed Training
King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia
Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Aug 4, 2026Junwen Qiu, Bohao Ma, Andre Milzarek +1Stochastic Optimization Nonconvex Stochastic Optimization
Industrial Systems Engineering and Management, National University of Singapore, Singapore. · School of Data Science, The Chinese University of Hong Kong, Shenzhen, China.
Random Reshuffling Dominates Stochastic Gradient Descent
Jun 30, 2026Zijian LiuStochastic Optimization Convergence Random Reshuffling
Stern School of Business, New York University
Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts
May 8, 2026Yi-Chang Chen, Feng-Ting Liao, Da-shan Shiu +1CoT Reasoning Causal Interventions in Language Models
MediaTek Research · Artificial Intelligence Center of Research Excellence, National Taiwan University
Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
Apr 11, 2026Konstantinos Emmanouilidis, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Rene VidalRandom Reshuffling Richardson Extrapolation
Department of Computer Science, University of Pennsylvania · University of Wisconsin-Madison, Archimedes Research Unit
Learning to Shuffle: Block Reshuffling and Reversal Schemes for Stochastic Optimization
Mar 31, 2026Lam M. Nguyen, Dzung T. Phan, Jayant KalagnanamStochastic Optimization Convergence Stochastic Gradient Descent
IBM Research, Thomas J. Watson Research Center, Yorktown Heights, NY, USA
Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning
Mar 17, 2026Junwen Qiu, Leilei Mei, Junyu ZhangRandom Reshuffling Stochastic Optimization
Industrial Systems Engineering and Management, National University of Singapore

Oct 18, 2026J/K move · Enter open · S save

Sampling and Curvature in First-Exit Comparisons for Momentum SGD

Haiyi Li

SGD with Momentum Stochastic Gradient Descent Neural Network Training Dynamics Random Reshuffling

Open paper AskPDF arXiv

First-exit prediction for momentum SGD depends on sampling information that pointwise gradient laws omit. Strongly convex finite pools with identical pointwise laws and matched iid momentum dynamics can have different reshuffled exits; even analytic pools with uniformly close pointwise laws can have separated exit probabilities. For persistent sampling in one dimension, we establish an explicit

O(h)

mean-exit comparison with a finite-rate switching process under nonvanishing component velocities and step-size conditions; iid sampling instead averages to an ODE. Retaining realized minibatches and sampled curvature yields finite-window exit brackets beyond this regime. In 32 trained ridge-logistic runs, 29 meet prespecified targets with rounding-certified brackets; an enclosure control explains most of the width advantage. Exact-real certificates establish finite means at the original tested radii for three ResNet-18 checkpoints and give local finite-mean conditions for tanh and ViT networks; their upper bounds remain loose, and original-radius ViT mean guarantees remain unresolved despite 128 fresh continuation paths. A two-network replication gives 32/32 successful seeds in each of four finite-path persistence comparisons. Independent equal-length controls retain slower exits after refresh resets than without resets, but the refresh-versus-random ordering reverses with update length at both networks. These interventions do not causally decompose raw momentum SGD. Neural step-size scaling does not validate the switching limit.