Learning Coefficient

The local learning coefficient (LLC) is a novel complexity measure for neural networks, derived from singular learning theory, that aims to quantify a model's effective dimensionality beyond simply counting parameters. Current research focuses on applying LLC to analyze the training dynamics and emergent structures in various architectures, including transformers and deep linear networks, to understand how training algorithms and data influence model complexity. This work offers a more nuanced understanding of model generalization and efficiency, potentially leading to improved training strategies and more interpretable deep learning models.