Product Manifold

Product manifolds, representing data as points in spaces beyond the familiar Euclidean geometry (e.g., incorporating spherical or hyperbolic components), are increasingly used in machine learning to capture complex data structures. Current research focuses on developing algorithms to learn and utilize these manifolds effectively, including adapting decision trees, graph neural networks, and generative models to non-Euclidean spaces, and analyzing the resulting manifold geometry using tools from topology and Riemannian geometry. This work aims to improve model performance in various applications, from image generation and natural language processing to biological pathway analysis, by better representing the intrinsic geometry of the data. The resulting insights into data structure and model behavior are expected to advance both theoretical understanding and practical applications of machine learning.

Papers

October 3, 2024

Mixed-curvature decision trees and random forests
Philippe Chlenski, Quentin Chu, Raiyan R. Khan, Antonio Khalil Moretti, Itsik Pe'er
Decision Tree Random Forest Class Manifold Product Manifold

August 15, 2024

Understanding the Local Geometry of Generative Model Manifolds
Ahmed Imtiaz Humayun, Ibtihel Amara, Candice Schumann, Golnoosh Farnadi, Negar Rostamzadeh, Mohammad Havaei
Manifold Learning Pre Trained Generative Model Product Manifold Local Geometry

June 9, 2024

Hidden Holes: topological aspects of language models
Stephen Fitz, Peter Romero, Jiyan Jonas Schneider
Large Language Model Language Model Topological Feature Recurrent Network Autoregressive Language Model Product Manifold

June 7, 2024

Mixed-Curvature Decision Trees and Random Forests
Philippe Chlenski, Quentin Chu, Itsik Pe'er
Random Forest Class Manifold Dimensional Manifold Product Manifold Mixed Curvature

March 31, 2024

Deep Extrinsic Manifold Representation for Vision Tasks
Tongtong Zhang, Xian Wei, Yuanxiang Li
Vision Task Non Euclidean Product Manifold Geodesic Loss

February 1, 2024

A Manifold Representation of the Key in Vision Transformers
Li Meng, Morten Goodwin, Anis Yazidi, Paal Engelstad
Vision Transformer Multi Head Self Attention Attention Block Flamelet Generated Manifold Product Manifold

January 27, 2024

Product Manifold Representations for Learning on Biological Pathways
Daniel McNeela, Frederic Sala, Anthony Gitter
LeArning Abstract Biological Pathway Product Manifold

October 24, 2023

Constructing and Machine Learning Calabi-Yau Five-folds
R. Alawadhi, D. Angella, A. Leonardo, T. Schettini Gherardini
Dimensional Space Product Manifold Calabi Yau Projective Geometry

October 3, 2023

A simple connection from loss flatness to compressed representations in neural networks
Shirui Chen, Stefano Recanatesi, Eric Shea-Brown
Neural Network Deep Neural Network Meaningful Representation Neural Representation New Connection Generalization Capacity Compressed Representation Product Manifold Loss Surface

September 9, 2023

Gromov-Hausdorff Distances for Comparing Product Manifolds of Model Spaces
Haitz Saez de Ocariz Borde, Alvaro Arroyo, Ismael Morales, Ingmar Posner, Xiaowen Dong
Latent Geometry Model Space Flamelet Generated Manifold Product Manifold Hausdorff Distance

July 10, 2023

Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold
Tuc Nguyen-Van, Dung D. Le, The-Anh Ta
Graph Representation Learning Graph Representation Real World Graph Heterogeneous Graph Learning Product Manifold Mixed Curvature

February 28, 2023

Parametrizing Product Shape Manifolds by Composite Networks
Josua Sassen, Klaus Hildebrandt, Martin Rumpf, Benedikt Wirth
Low Dimensional Manifold Data Manifold Shape Space Product Manifold Hybrid Network Geodesic Regression

February 16, 2023

Exploring the Representation Manifolds of Stable Diffusion Through the Lens of Intrinsic Dimension
Henry Kvinge, Davis Brown, Charles Godfrey
Meaningful Representation Camera Lens Stable Diffusion Internal Representation Intrinsic Dimension Latent Encoding Product Manifold

November 26, 2022

Latent Graph Inference using Product Manifolds
Haitz Sáez de Ocáriz Borde, Anees Kazi, Federico Barbero, Pietro Liò
Latent Graph Product Manifold Latent Graph Inference

September 18, 2022

The Geometry of Self-supervised Learning Models and its Impact on Transfer Learning
Romain Cosentino, Sarath Shekkizhar, Mahdi Soltanolkotabi, Salman Avestimehr, Antonio Ortega
Transfer Learning Self Supervised Global Impact Geometric Analysis Geometric Control Product Manifold Manifold Augmentation

May 16, 2022

Manifold Characteristics That Predict Downstream Task Performance
Ruan van der Merwe, Gregory Newman, Etienne Barnard
Adversarial Attack Self Supervised Method Flamelet Generated Manifold Projected Gradient Descent Product Manifold Representation Quality Downstream Task Performance