Helgason-Fourier Random Features for Scalable Hyperbolic Latent Variables
Organizations: Hokkaido University · The Institute of Statistical Mathematics · Waseda University
Abstract
Hyperbolic geometry is well suited to embedding hierarchical data, capturing tree-like structure faithfully even in the low dimensions where Euclidean dimensionality reduction is severely constrained. We introduce Random Helgason--Fourier Features (RHFF), an explicit finite-dimensional feature map for kernels on hyperbolic space. We characterize the spectral density from which RHFF frequencies are drawn, allowing kernels such as the Matérn family to be specified directly in the frequency domain. We also prove a uniform approximation error bound for RHFF. Building on this representation, we develop a hyperbolic latent variable model called RHFF-LVM, which requires no numerical quadrature, variational bounds or auxiliary variables unlike previous hyperbolic GP-LVMs. Our RHFF-LVM recovers the metric of a synthetic tree more faithfully than all baselines and attains the best mean rank, driven by large gains on tree-structured atlases at . Furthermore, it fits points in seconds, making it the first non-parametric hyperbolic LVM to reach this scale.
Figures & tables
| Toy-tree | |||
| Method | loc | glo | tree corr |
| PCA | 14.9 0.0 | 79.7 0.0 | 77.2 0.0 |
| UMAP McInnes et al. (2018) | 69.0 0.4 | 58.7 3.9 | 58.4 3.4 |
| BGP-LVM Titsias and Lawrence (2010) | 71.5 0.3 | 75.8 1.5 | 75.5 1.3 |
| RFF-LVM Gundersen et al. (2021) | 72.2 0.7 | 74.9 1.6 | 74.9 1.4 |
| Poincaré Map Klimovskaia et al. (2020) | 69.4 0.4 | 83.6 0.8 | 82.4 0.7 |
| Paul | Plass | Tasic | C. elegans | MNIST | Zebrafish | Zeisel | MouseDev | Mean rank | |||||||||||
| Method | loc | glo | loc | glo | loc | glo | loc | glo | loc | glo | loc | glo | loc | glo | loc | glo | loc | glo | both |
| PCA | 1.5 | 27.9 | 1.1 | 65.9 | 1.1 | 88.1 | 1.9 | 71.0 | 0.8 | 52.7 | 0.9 | 92.3 | 0.4 | 77.2 | 0.5 | 94.5 | 9.00 | 3.35 | 6.18 |
| UMAP McInnes et al. (2018) | 5.3 | 12.4 | 13.3 | 67.2 | 14.3 | 49.1 | 22.4 | 60.6 | 11.7 | 35.7 | 13.2 | 79.3 | 11.6 | 53.1 | 8.3 | 63.4 | 1.12 | 7.83 | 4.47 |
| BGP-LVM Titsias and Lawrence (2010) | 2.8 | 71.8 | 3.3 | 72.4 | 4.8 | 73.3 | 2.5 | 47.8 | 0.9 | 67.0 | 0.2 | 10.5 | - | - | - | - | - | - | - |
| RFF-LVM Gundersen et al. (2021) | 2.8 | 69.3 | 9.1 | 85.1 | 6.7 | 73.8 | 6.2 | 74.4 | 3.3 | 41.6 | 3.0 | 84.6 | 1.7 | 72.5 | 0.9 | 86.2 | 4.30 | 4.93 | 4.61 |
| Poincaré Map Klimovskaia et al. (2020) | 4.1 | 13.5 | 10.9 | 78.8 | 13.9 | 47.6 | - | - | - | - | - | - | - | - | - | - | - | - | - |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Types | Class size (min/med/max) | Category | ||
|---|---|---|---|---|---|
| toytree | 1,008 | 6 | 63 | 16 / 96 / 512 | synthetic control |
| Paul [ 28 ] | 2,730 | 20 | 1,000 | 1 / 138 / 373 | scRNA-seq |
| Plass [ 29 ] | 21,612 | 51 | 50 | 13 / 202 / 6,343 | scRNA-seq |
| Tasic [ 37 ] | 23,822 | 133 | 3,000 | 4 / 115 / 1,404 | scRNA-seq |
| C. elegans [ 27 ] | 54,649 | 36 | 50 | 25 / 349 / 17,520 | scRNA-seq |
| MNIST [ 18 ] | 60,000 | 10 | 784 | 5,421 / 5,936 / 6,742 | image control |
| Global fidelity ( ) | Spectrum heat | ||||||
|---|---|---|---|---|---|---|---|
| Dataset | Type | heat | glo | loc | |||
| Paul | tree | 64.2 | +36.3 | -5.1 | +0.1 | ||
| Plass | tree | 90.1 | +24.2 | +5.0 | -0.5 | ||
| C. elegans | tree | 78.8 | +7.8 | +4.5 | +1.0 | ||
| Zebrafish | tree | 90.9 | -1.4 | +6.3 | +1.1 | ||
| MouseDev | tree | 91.1 | -3.3 | +5.0 | +1.3 | ||
Hyperbolic Representation Learning
Hyperbolic Representation Learning uses negatively curved spaces to encode hierarchies and scale-free relations in compact, geometry-aware embeddings.
See all 99 papers