Functional Dimension

Functional dimension investigates the effective dimensionality of the function space spanned by a parameterized model, such as a neural network, considering the redundancies introduced by parameter symmetries and optimization processes. Current research focuses on understanding how this dimension evolves during training, particularly in deep ReLU networks and score-based generative models, analyzing the interplay between parameter space geometry and the resulting function space. These studies aim to improve our understanding of implicit regularization in deep learning and provide insights into the efficiency and generalization capabilities of various machine learning models.