Global Minimum
Finding global minima in complex, high-dimensional optimization problems, particularly those arising in deep learning and scientific computing, is a central challenge. Current research focuses on understanding the properties of global minima in various model architectures (e.g., deep neural networks with ReLU activation, graph neural networks) and optimization algorithms (e.g., stochastic gradient descent, Adam), including the impact of regularization and initialization strategies. This research aims to improve the efficiency and reliability of finding optimal solutions, impacting fields like machine learning, where generalization performance is closely tied to the choice of minimum, and scientific computing, where accurate solutions to differential equations are crucial. A key area of investigation involves characterizing the landscape around global minima, identifying conditions that ensure convergence, and developing methods to avoid undesirable local minima.
Papers
Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix
Charles C. Margossian, Lawrence K. Saul
Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation
Sonja Hannibal, Arnulf Jentzen, Do Minh Thang