Second-Order Optimization

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 63

All topics

L-SR1: Learned Symmetric-Rank-One Preconditioning
Aug 17, 2025Gal Lifshitz, Shahar Zuler, Ori Fouks +1Learning to Optimize Second-Order Optimization
Tel Aviv University, Tel Aviv, Israel.
Multi-Objective Hyperparameter Search via Damped Gauss--Newton Optimization
Jan 7, 2024Qinwu Xu, Yifan JiangGauss-Newton Optimization Second-Order Optimization
Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration
Oct 11, 2023Zeyang Li, Chuxiong Hu, Yunan Wang +4Markov Decision Processes Reinforcement Learning
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA 02139, United States · Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China · College of Artificial Intelligence and the School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China

Aug 17, 2025, cs.LGJ/K move · Enter open · S save

L-SR1: Learned Symmetric-Rank-One Preconditioning

Gal Lifshitz, Shahar Zuler, Ori Fouks, Dan Raviv

Tel Aviv University, Tel Aviv, Israel.

Learning to Optimize Second-Order Optimization Quasi-Newton Methods

Open paper AskPDF arXiv

End-to-end deep learning has achieved impressive results but often relies on large labeled datasets, exhibits limited generalization to unseen scenarios, and incurs substantial computational cost. Classical optimization methods, in contrast, are more data-efficient and lightweight but frequently suffer from slow convergence. Learned optimizers aim to bridge this gap, yet existing approaches have focused primarily on first-order methods, while learned second-order optimization has received much less attention. We introduce L-SR1, a learned second-order optimizer inspired by the classical Symmetric Rank-One (SR1) method. At its core, L-SR1 employs a Projection-Guided Secant Mechanism (PGSM) that generates positive semi-definite preconditioners and biases meta-training toward the quasi-Newton secant relation. Through controlled analytic benchmarks, we study stability, generalization across problem dimensions, and search direction quality, and further evaluate L-SR1 on Monocular Human Mesh Recovery (HMR), where it outperforms both classical and learned optimization-based baselines. With a compact model and no reliance on task-specific fine-tuning or annotated data, L-SR1 demonstrates strong generalization and can be integrated into a broad range of iterative optimization problems to accelerate convergence and reduce the required number of iterations.

Momentum

Related topics

Latest papers 63

L-SR1: Learned Symmetric-Rank-One Preconditioning

Multi-Objective Hyperparameter Search via Damped Gauss--Newton Optimization

Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration