AG-BFGS: Foundations for Accelerated Geometry Learning
Organizations: Dept of Mathematics, UFOP, Brazil · PPGMMC, CEFET-MG, Brazil · CSDT School, Aston University - PPGMMC, CEFET-MG · SLB Cambridge Research, High Cross, Madingley Road, Cambridgeshire, UK
Abstract
We introduce an accelerated geometry-learning framework for BFGS that incorporates momentum into the learning of Hessian inverse approximations. We prove that the resulting accelerated updates satisfy explicit stability conditions and preserve positive definiteness within admissible parameter regions. Numerical experiments demonstrate consistent improvements over the corresponding non-accelerated schemes while maintaining numerical stability. These results provide the first rigorous theoretical foundation for accelerated quasi-Newton learning and bridge the gap between empirical observations and mathematical guarantees for accelerated second-order methods.
Figures & tables
| Dataset | aBFGS | |
|---|---|---|
| Australian | 30.00 | 0.50 |
| Phishing | 1228.33 | 0.10 |
| Mushroom | 8124.00 | 0.01 |
| Splice | 1000.00 | 0.90 |
Quasi-Newton Methods
Quasi-Newton methods approximate curvature from gradient and step information to accelerate optimization without forming full Hessians.
See all 17 papers