Linear Mode Connectivity

Linear mode connectivity (LMC) investigates the existence of low-loss linear paths between different solutions found by training machine learning models, particularly deep neural networks and tree ensembles. Current research focuses on understanding the underlying geometric properties of loss landscapes that enable or hinder LMC, exploring the roles of model architecture (including sparsity and weight-sharing), training strategies, and dataset characteristics. This research is significant because LMC offers insights into the effectiveness of optimization algorithms, facilitates model merging and averaging techniques (e.g., in federated learning), and improves our understanding of deep learning dynamics.