math.OCNov 19, 2026

Safeguarded Learning-Assisted Nonlinear Conjugate Gradient Methods

Authors: Morteza Kimiaei, Saman Babaie--Kafaki

Organizations: Faculty of Mathematics, University of Vienna, Oskar–Morgenstern–Platz 1, A–1090 Vienna, Austria · Faculty of Engineering, Free University of Bozen–Bolzano, NOI Techpark, via Bruno Buozzi 1, 39100 Bolzano, Italy

Abstract

A learning-assisted framework for Nonlinear Conjugate Gradient (NCG) methods is developed in which lightweight online prediction is used to support, rather than replace, the classical optimization process. The proposed approach retains the standard two-term NCG structure and introduces two independent online extreme learning machine models. The first model predicts an NCG coefficient during restart recovery or when persistent stagnation is detected, whereas the second model is used only after a line-search failure in finite precision arithmetic to propose a bounded correction to the deterministic fallback step size. Neural predictions are accepted only when they satisfy deterministic numerical safeguards, including descent, scale, and out-of-distribution requirements; otherwise, the corresponding deterministic recovery or fallback mechanism is retained. Both learning models are updated online using quantities already generated along the optimization trajectory and therefore require no additional function or gradient evaluations for training. These learning frameworks are applied to six well-known classical NCG methods, producing corresponding low-storage learning-assisted variants with deterministic safeguards. Numerical experiments on 480 CUTEst test problems show improved robustness for all six variants at broadly comparable evaluation costs. A PDE-constrained conductivity-identification example further illustrates the potential of the proposed approach to reduce evaluation costs and computational time for problems with expensive function and gradient evaluations. These findings suggest that restricted online learning can improve the robustness and computational performance of classical NCG methods while preserving their low-storage structure and deterministic optimization safeguards, and is particularly promising when function or gradient evaluations are expensive.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebra

    May 16, 2025Ivan Bioli, Carlo Marcati, Giancarlo SangalliNeural PDE SolversGradient Descent

  2. Learned Preconditioning for a Primal-Dual Interior-Point Method

    Sep 28, 2026Abhinav Madabhushi, Jialin Liu, Minxin ZhangPrimal-Dual OptimizationLearning to Optimize