L BFGS

L-BFGS (Limited-memory Broyden–Fletcher–Goldfarb–Shanno) is a quasi-Newton optimization algorithm used extensively in machine learning and other fields requiring efficient optimization of complex functions. Current research focuses on improving L-BFGS's performance in large-scale settings, particularly for training deep neural networks, through techniques like momentum-based modifications and preconditioning strategies, often incorporating distributed computing for scalability. These advancements aim to enhance the speed and stability of L-BFGS, leading to more efficient training of machine learning models and improved solutions for problems in system identification and other applications where high-dimensional optimization is crucial.

Papers

May 28, 2024

From Learning to Optimize to Learning Optimization Algorithms
Camille Castera, Peter Ochs
LeArning Abstract Optimization Algorithm Learned Optimizers Classical Algorithm Classical Optimization L BFGS

March 6, 2024

Linear and nonlinear system identification under $\ell_1$- and group-Lasso regularization via L-BFGS-B
Alberto Bemporad
State Space Model Discrete Time System Identification Linear Ordered Data Nonlinear System Identification Group Lasso L BFGS

February 12, 2024

Topological safeguard for evasion attack interpreting the neural networks' behavior
Xabier Echeberria-Barrio, Amaia Gil-Lerchundi, Iñigo Mendialdua, Raul Orduna-Urrutia
Neural Network BEHAVIOR Explanation Evasion Attack L BFGS

July 25, 2023

mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization
Yue Niu, Zalan Fabian, Sunwoo Lee, Mahdi Soltanolkotabi, Salman Avestimehr
Large Scale Hessian Matrix Scalable Neural Quasi Newton Method Quasi Newton L BFGS

June 30, 2023

Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
Alena Kopaničáková, Hardik Kothari, George Em Karniadakis, Rolf Krause
Training Data Physic Informed Neural Network General Strategy Adaptive Preconditioner Domain Decomposition L BFGS

October 26, 2022

LP-BFGS attack: An adversarial attack based on the Hessian with limited pixels
Jiebao Zhang, Wenhua Qian, Rencan Nie, Jinde Cao, Dan Xu
Adversarial Attack Hessian Matrix White Box Missing Pixel L BFGS

November 29, 2021

Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning
Sanae Lotfi, Tiphaine Bonniot de Ruisselet, Dominique Orban, Andrea Lodi
Adaptive Importance Second Order Stochastic Optimization First Order Algorithm Large Scale Machine Learning Second Order Algorithm L BFGS

L BFGS

Papers

From Learning to Optimize to Learning Optimization Algorithms

Linear and nonlinear system identification under $\ell_1$- and group-Lasso regularization via L-BFGS-B

Topological safeguard for evasion attack interpreting the neural networks' behavior

mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization

Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies

LP-BFGS attack: An adversarial attack based on the Hessian with limited pixels

Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning