Topological Generalization

Topological generalization explores how the geometric and topological properties of training data and model parameters relate to a machine learning model's ability to generalize to unseen data. Current research focuses on using topological data analysis techniques, such as fractal dimension and persistent homology, to quantify model complexity and predict generalization performance in various architectures, including deep neural networks, graph neural networks, and decentralized stochastic gradient descent. These investigations aim to improve our understanding of generalization beyond traditional statistical learning theory, potentially leading to more robust and reliable machine learning models. The findings could significantly impact the design and optimization of algorithms for diverse applications.

Papers

July 11, 2024

Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms
Rayna Andreeva, Benjamin Dupuis, Rik Sarkar, Tolga Birdal, Umut Şimşekli
Training Dynamic Generalization Gap Generalization Capability Topological Generalization

June 4, 2024

On the Limitations of Fractal Dimension as a Measure of Generalization
Charlie B. Tan, Inés García-Redondo, Qiquan Wang, Michael M. Bronstein, Anthea Monod
Strong Generalization Fundamental Limitation Persistent Homology Generalization Gap Fractal Dimension Topological Generalization

October 10, 2023

Advective Diffusion Transformers for Topological Generalization in Graph Learning
Qitian Wu, Chenxiao Yang, Kaipeng Zeng, Fan Nie, Michael Bronstein, Junchi Yan
Graph Learning Diffusion Transformer Graph Encoder Graph Diffusion Latent Graph Topological Generalization

October 7, 2022

Toward an Over-parameterized Direct-Fit Model of Visual Perception
Xin Li
Vision Model Visual Perception Computational Model Parameterized Model High Dimensional Space Topological Generalization

August 19, 2022

Graph Convolutional Networks from the Perspective of Sheaves and the Neural Tangent Kernel
Thomas Gebhart
Convolutional Neural Network Visual Perspective Neural Tangent Kernel Deep Learning Algorithm Sheaf Diffusion Topological Generalization

June 25, 2022

Topology-aware Generalization of Decentralized SGD
Tongtian Zhu, Fengxiang He, Lan Zhang, Zhengyang Niu, Mingli Song, Dacheng Tao
Stochastic Gradient Descent Vanilla Gradient Descent Decentralized SGD Decentralized Stochastic Gradient Descent Topological Generalization