Unsupervised Latent Space Alignment with Hyperspherical Geodesic Matching

Cameron Ryan; Vivek Sivaraman Narayanaswamy; Kowshik Thopalli; Shusen Liu

cs.LGAug 28, 2026

Unsupervised Latent Space Alignment with Hyperspherical Geodesic Matching

Authors: Cameron Ryan, Vivek Sivaraman Narayanaswamy, Kowshik Thopalli, Shusen Liu

Organizations: Northeastern University, Boston, MA 02115. Work performed during a summer internship at Lawrence Livermore National Laboratory · Lawrence Livermore National Laboratory, Livermore, CA 94551-0808.

Abstract

Independently trained neural networks tend to encode the same data with similar latent geometries. These latent geometries are not directly compatible, yet they can be nearly the same up to some class of transformations. While there exists many methods for alignment between different latent spaces, it is typically done using a set of shared sample correspondences, known as anchors. This leaves a fundamental question: are the geometric signatures of different latent spaces representing similar data sufficient to recover an alignment between them? To that end, we introduce HGA (Hyperspherical Gaussian Alignment), a method that directly optimizes a transformation between two latent spaces by maximizing a geometric measure of "fit" between them. Since it is driven by the geometry of the latent spaces rather than paired data, HGA can operate in both an unsupervised and weakly supervised regime. On tasks such as model stitching or multilingual word embedding correspondence recovery, HGA manages to match supervised results with minimal or no supervision.

Figures & tables

Dataset	Supervised Ortho	Unsupervised Latent Functional Maps	Unsupervised HGA
MNIST	$96.79\%$	$27.73\pm 14.39\%$	$96.55\pm 0.09\%$
CIFAR10	$73.48\%$	$19.62\pm 9.54\%$	$72.80\pm 0.30\%$
CIFAR100	$34.51\%$	$1.42\pm 0.33\%$	$31.20\pm 0.96\%$

Table 1: Stitching accuracy (%) of fully supervised and unsupervised methods on test sets for each dataset. HGA optimization is done with 50 restarts for each dataset, and the rotation matrix with the highest objective function score is used for model stitching. Results show mean and standard deviation for 3 runs of Latent Functional Maps and HGA with different random seeds.

Figure 1: Word correspondence accuracy vs number of anchors for various alignment methods on word embeddings from FastText

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Objective function score at $\sigma=0.2$ vs optimization step for 10 random orthogonal matrix initializations. Color denotes the final objective function score reached. Noticeable jumps are due to a decrease in $\sigma$ for the objective function $J_{\sigma}(R)$ being used to compute optimization gradient.

Layer	$\sigma$	$\sigma_{\text{start}}$	LR	LR final	Stitching Acc. (HGA unsupervised)	Stitching Acc. (Ortho Supervised)
4	0.1	4	0.1	0.0005	70%	70%
5	0.1	4	0.1	0.0005	62%	71%
6	0.1	4	0.1	0.001	75%	73%
7	0.2	4	0.01	0.0005	75%	76%
8	0.1	4	0.01	0.0005	76%	77%

Appendix

Table 2: CIFAR10 best unsupervised stitching accuracies and corresponding hyperparameters for each model layer. Results from an experiment running the optimization algorithm with a single restart and different hyperparameters for each run. Far right column shows stitching accuracy for orthogonal least squares solution with 1024 anchor points.

Figure 3: Percent of Anchors Corrupted vs. Model Stitching Accuracy for the latent spaces of two neural networks trained on the MNIST dataset.

Panel of Figure 4 — Figure 4: Scatter plots show objective function scores vs stitching accuracies for various optimization trials with slightly different optimization hyperparameters on CIFAR10.

Panel of Figure 5 — Figure 5: Implied kernel subsets and model stitching confusion matrices for unsupervised stitching in MNIST, CIFAR10, and CIFAR100

LFM			HGA
R	Word	Score	R	Word	Score
dog (in anchor list)
1	chien	0.050	1	chien	0.929
2	chat	0.040	2	chiens	0.692
3	animaux	0.031	3	chat	0.683
4	peau	0.028	4	animal	0.609

Appendix

Figure 6: Qualitative comparison of nearest French neighbours retrieved by LFM and HGA for English query words, given 100 anchor words. Correct translations are shown in bold . For anchor words ( dog , house ) both methods rank the correct translation first, but for words absent from the anchor list ( bird , horse , flower , forest ) LFM largely returns unrelated or noisy tokens, whereas HGA still recovers the correct translation.

Figure 7: Subset of kernel between a model trained on MNIST excluding the “0” class, and a regular model trained on MNIST

Panel of Figure 8 — Figure 8: Kernel sum distributions for each class between a model trained on MNIST excluding the “0” class, and a regular model trained on MNIST

Rank	Orthogonal Procrustes	Latent Functional Maps	Spherical Alignment
1	banner	pennant	banner
2	pennant	banner	pennant
3	minivan	tag	scoreboard
4	scoreboard	jersey	tag
5	tag	game	jersey
6	woman	visor	board game

Appendix

Table 3: Concepts from the THINGS dataset with the lowest average kernel sums. We show results using three different alignment methods (ranked ascending).

Latent Space Alignment

Latent space alignment maps learned representations into compatible coordinates to enable comparison, transfer, and interoperability across models or data sources.

See all 76 papers

Multi-Way Representation Alignment
Feb 5, 2026Akshit Achara, Tatiana Gaintseva, Mateo Mahaut +5Neural Representation Geometry Latent Space Alignment
King’s College London, London, United Kingdom · Queen Mary University of London, London, United Kingdom · Universitat Pompeu Fabra, Barcelona, Spain +5
Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs
Sep 18, 2026Léo Nicollier, Enric Meinhardt-Llopis, Marc Pic +2Joint-Embedding Predictive Architecture Latent Space Geometry
CB, ATT · Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, France · Advanced Track and Trace +5
Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Models
Jun 3, 2026Shuwen Yu, Zhanxuan Hu, Yi Zhao +2Cross-Modal Representation Learning Vision Foundation Model Adaptation
Yunnan Normal University, Kunming, China · Kunming University of Science and Technology, Kunming, China

Feb 5, 2026J/K move · Enter open · S save

Multi-Way Representation Alignment

Akshit Achara, Tatiana Gaintseva, Mateo Mahaut, Pritish Chakraborty+4

King’s College London, London, United Kingdom · Queen Mary University of London, London, United Kingdom · Universitat Pompeu Fabra, Barcelona, Spain+5

Neural Representation Geometry Latent Space Alignment

Open paper AskPDF arXiv

The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of

M \ge 3

models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.

Abstract

Latent Space Alignment

Explore similar work

Multi-Way Representation Alignment

Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs

Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Models