Smooth Optimization

Smooth optimization focuses on developing algorithms that efficiently find optimal solutions for problems where the objective function is smooth, or can be made so through techniques like barrier methods or novel activation functions. Current research emphasizes improving convergence rates and sample complexity for various settings, including neural networks (using weight conditioning and novel activation functions like SMU), multi-objective optimization (via smooth scalarization), and online convex optimization (with adaptive step sizes and mirror descent). These advancements have significant implications for machine learning, control systems, and other fields requiring efficient optimization of complex models, leading to improved performance and theoretical guarantees.