Manifold Hypothesis

The manifold hypothesis posits that high-dimensional data often resides near a lower-dimensional manifold, a significant simplification impacting machine learning's success. Current research focuses on understanding how this hypothesis affects the performance of various models, including diffusion models, neural networks, and generative adversarial networks, with investigations into the impact of manifold geometry (e.g., curvature) on learning difficulty and the development of algorithms that explicitly leverage manifold structure for improved efficiency and robustness. This research is crucial for advancing theoretical understanding of deep learning, improving model performance in high-dimensional settings, and enabling more efficient and reliable solutions to inverse problems and other applications.

Papers

October 31, 2024

A Geometric Framework for Understanding Memorization in Generative Models
Brendan Leigh Ross, Hamidreza Kamkari, Tongzi Wu, Rasa Hosseinzadeh, Zhaoyan Liu, George Stein, Jesse C. Cresswell, Gabriel Loaiza-Ganem
Generative Model Full Model Generative Question Limited Memorization Memorization Effect Manifold Hypothesis Geometric Framework

October 11, 2024

Linear Convergence of Diffusion Models Under the Manifold Hypothesis
Peter Potaptchik, Iskander Azangulov, George Deligiannidis
Diffusion Model Generative Model Convergence Guarantee Linear Convergence Manifold Hypothesis Backward Stochastic Differential Equation Step Diffusion Model

October 8, 2024

Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion
Enrico Ventura, Beatrice Achilli, Gianluigi Silvestri, Carlo Lucibello, Luca Ambrogioni
Generative Diffusion Model Class Manifold Generative Process Random Matrix Civil Engineering Phase Generative Diffusion Manifold Hypothesis Latent Geometry Spectral Gap

September 27, 2024

Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
Iskander Azangulov, George Deligiannidis, Judith Rousseau
Diffusion Model Gaussian Process Early Stage Convergence Denoising Diffusion Probabilistic Model High Dimension Low Dimensional Manifold Manifold Hypothesis

August 13, 2024

Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds
Hong Ye Tan, Subhadip Mukherjee, Junqi Tang, Carola-Bibiane Schönlieb
Class Manifold Data Dimensionality Low Dimensional Manifold Generalization Property Manifold Hypothesis Blessing Generation Sobolev Training Statistical Complexity

July 3, 2024

A Geometric Framework for Adversarial Vulnerability in Machine Learning
Brian Bell
Machine Learning Adversarial Attack Adversarial Perturbation Adversarial Vulnerability Domain Adversarial Manifold Hypothesis Geometric Framework

June 3, 2024

Hardness of Learning Neural Networks under the Manifold Hypothesis
Bobak T. Kiani, Jason Wang, Melanie Weber
Neural Network Manifold Learning Low Dimensional Manifold Hardness Result Non Linear Manifold Manifold Hypothesis

May 27, 2024

DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models
Hengkang Wang, Xu Zhang, Taihui Li, Yuxiang Wan, Tiancong Chen, Ju Sun
Diffusion Model Inverse Problem Pre Trained Diffusion Model Reverse Diffusion Manifold Hypothesis Nonlinear Inverse Problem

April 3, 2024

Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections
Gabriel Loaiza-Ganem, Brendan Leigh Ross, Rasa Hosseinzadeh, Anthony L. Caterini, Jesse C. Cresswell
Timely Survey Generative Adversarial Network Latent Diffusion Model Variational Autoencoders Camera Lens New Connection Manifold Hypothesis Unknown Manifold

February 20, 2024

Learning on manifolds without manifold learning
H. N. Mhaskar, Ryan O'Dowd
LeArning Abstract Average Approximation Manifold Learning Class Manifold Function Approximation Manifold Hypothesis ONE Shot Unknown Manifold

December 20, 2023

Independent Mechanism Analysis and the Manifold Hypothesis
Shubhangi Ghosh, Luigi Gresele, Julius von Kügelgen, Michel Besserve, Bernhard Schölkopf
High Dimensional Latent Feature Independent Component Analysis Manifold Hypothesis Nonlinear Independent Component Analysis

March 29, 2023

Storage and Learning phase transitions in the Random-Features Hopfield Model
Matteo Negri, Clarissa Lauditi, Gabriele Perugini, Carlo Lucibello, Enrico Malatesta
Phase Transition Manifold Hypothesis Data Storage Hopfield Model Random Feature Attention

December 1, 2022

The Effect of Data Dimensionality on Neural Network Prunability
Zachary Ankner, Alex Renda, Gintare Karolina Dziugaite, Jonathan Frankle, Tian Jin
Neural Network Mixed Effect Data Dimensionality Dimensional Input Low Dimensional Structure Manifold Hypothesis

November 23, 2022

CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds
Jesse C. Cresswell, Brendan Leigh Ross, Gabriel Loaiza-Ganem, Humberto Reyes-Gonzalez, Marco Letizia, Anthony L. Caterini
Density Estimation Class Manifold Low Dimensional Manifold Manifold Hypothesis High Dimensional Representation Calorimeter Shower Fast Generation

October 26, 2022

Adversarial Purification with the Manifold Hypothesis
Zhaoyuan Yang, Zhiwei Xu, Jing Zhang, Richard Hartley, Peter Tu
Adversarial Example Adversarial Robustness Variational Autoencoders Manifold Learning Adversarial Purification Manifold Hypothesis

September 30, 2022

Topological Singularity Detection at Multiple Scales
Julius von Rohrscheidt, Bastian Rieck
Dimensional Manifold Manifold Hypothesis Local Intrinsic

August 24, 2022

Statistical exploration of the Manifold Hypothesis
Nick Whiteley, Annie Gray, Patrick Rubin-Delanchy
Low Dimensional Manifold Dimensional Manifold High Dimensional Space Manifold Hypothesis

August 19, 2022

IAN: Iterated Adaptive Neighborhoods for manifold learning and dimensionality estimation
Luciano Dyballa, Steven W. Zucker
Dimensionality Reduction Manifold Learning Manifold Hypothesis Minimal Problem Adaptive Neighborhood

August 10, 2022

Convergence of denoising diffusion models under the manifold hypothesis
Valentin De Bortoli
Diffusion Model Generative Model Early Stage Convergence Audio Synthesis Manifold Hypothesis

July 14, 2022

Distance Learner: Incorporating Manifold Prior to Model Training
Aditya Chetan, Nipun Kwatra
Adversarial Training Adversarial Robustness Model Training Class Manifold Low Dimensional Manifold Dimensional Manifold Manifold Hypothesis Online Learner