Flat Minima

Flat-minima research studies how the shape of a model’s loss landscape relates to what it learns, including its ability to generalize, resist perturbations, and adapt across tasks. Methods such as sharpness-aware minimization (SAM), Hessian-based regularization, and geometry-aware perturbations try to steer training toward flatter solutions; theory also examines how SGD noise, learning rates, and batch size shape the minima it reaches. Work spans neural-network theory and practical settings such as federated learning, fine-tuning, and vision tasks. A central open question is whether flatness reliably predicts generalization: parameter symmetries and task differences can make common measures misleading, and flatter solutions do not always perform better.