Block Coordinate

Block coordinate methods are optimization techniques that iteratively update subsets of a model's parameters, rather than all parameters simultaneously. Current research focuses on applying these methods to large-scale problems, such as training large language models (LLMs) and solving extensive-form games, leveraging their memory efficiency and potential for faster convergence compared to full-gradient methods. This approach is proving valuable in reducing computational costs and improving the scalability of various machine learning and optimization tasks, leading to advancements in areas like natural language processing and game theory.