Box Dimension
Box dimension, or dimensionality, is a crucial concept across diverse fields, encompassing the intrinsic dimensionality of data, the effective dimensionality of model representations, and the impact of dimensionality on algorithm performance. Current research focuses on estimating and manipulating dimensionality in various contexts, employing techniques ranging from differential geometry and algebraic geometry to neural network architectures like autoencoders and graph convolutional networks, and exploring the interplay between dimensionality and model properties such as generalization ability and computational efficiency. Understanding and controlling dimensionality is vital for improving model interpretability, efficiency, and robustness across machine learning, data analysis, and scientific modeling.
Papers
Losing dimensions: Geometric memorization in generative diffusion
Beatrice Achilli, Enrico Ventura, Gianluigi Silvestri, Bao Pham, Gabriel Raya, Dmitry Krotov, Carlo Lucibello, Luca Ambrogioni
On the token distance modeling ability of higher RoPE attention dimension
Xiangyu Hong, Che Jiang, Biqing Qi, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou