Proximal Gradient Descent

Proximal gradient descent (PGD) is an optimization algorithm used to solve problems involving the sum of a smooth and a non-smooth function, often incorporating regularization terms to promote desirable properties like sparsity or low rank. Current research focuses on extending PGD's application to complex settings, including federated learning, matrix completion, and training neural networks with embedded optimization layers, often employing techniques like variance reduction and adaptive noise for improved efficiency and privacy. These advancements are impacting diverse fields, enabling improved performance in tasks such as image reconstruction, recommendation systems, and training large language models through enhanced optimization and model compression.