Gradient Based Solver

Gradient-based solvers are optimization algorithms used to find the parameters that minimize a given objective function, crucial in training machine learning models and solving partial differential equations (PDEs). Current research focuses on improving convergence speed and robustness, particularly addressing challenges like local optima and high dimensionality, through techniques such as combining gradient descent with sampling methods, employing difference-of-convex function representations, and developing novel architectures like LocalMixer for forward gradient methods. These advancements are impacting various fields, from robotics (contact-rich manipulation planning) and autonomous navigation to improving the efficiency and biological plausibility of neural network training.