Conjugate Gradient

Conjugate gradient (CG) methods are iterative optimization algorithms used to efficiently solve large-scale linear systems and minimize functions, particularly relevant in machine learning and scientific computing. Current research focuses on improving CG's performance through techniques like preconditioning (including data-driven approaches using neural networks), adapting CG for non-convex problems and specific manifold structures (e.g., Stiefel manifold), and enhancing its stability in low-precision arithmetic and ill-conditioned settings. These advancements significantly impact diverse fields by accelerating computations in Gaussian process regression, improving the efficiency of model predictive control, and enabling more effective adversarial attacks and deep learning optimization.