From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures

Florian Rottach; William Rudman; Bastian Rieck; Harrisen Scells; Carsten Eickhoff

cs.LGNov 27, 2025

From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures

Authors: Florian Rottach, William Rudman, Bastian Rieck, Harrisen Scells, Carsten Eickhoff

Organizations: University of Tübingen · Boehringer Ingelheim GmbH, Biberach (Riss), Germany · The University of Texas at Austin · Fribourg University

Abstract

Studying the shape of text embedding spaces enhances our understanding of model behavior and how geometry affects downstream task performance. While a range of topological and geometric metrics have been explored, they are typically analyzed in isolation, and the connections between them remain largely unknown. In this paper, we present Unified Topological Signatures (UTS) for characterizing the structure of embedding spaces. We evaluate our method across a wide range of text embedding models and retrieval datasets, finding that it outperforms individual metrics in downstream performance prediction. Through several experiments, we demonstrate that the signatures characterize embedding spaces more accurately, improve the prediction of model-specific properties and biases, and reveal novel insights into representational similarity that challenge the global claims of the Platonic Representation Hypothesis.

Figures & tables

Figure 1: Unified Topological Signatures of embedding spaces. Left: We construct signature vectors for embedding spaces by computing various topological descriptors. We use the vectors for downstream prediction tasks as well as for measuring representational similarity. Right: We compute local signatures based on the neighborhood of individual embeddings and use them to detect retrievability bias in document corpora.

Table 2

	Recall			MAP			NDCG
Method	@5	@20	@100	@5	@20	@100	@5	@20	@100
TwoNN	$0.07_{\pm.09}$	$0.08_{\pm.11}$	$0.04_{\pm.09}$	$0.07_{\pm.11}$	$0.06_{\pm.11}$	$0.04_{\pm.09}$	$0.06_{\pm.11}$	$0.07_{\pm.10}$	$0.05_{\pm.09}$
Eff. rank	$0.44_{\pm.11}$	$0.47_{\pm.08}$	$0.42_{\pm.09}$	$0.44_{\pm.09}$	$0.43_{\pm.08}$	$0.43_{\pm.07}$	$0.45_{\pm.09}$	$0.45_{\pm.08}$	$0.45_{\pm.07}$
Mean sim.	$0.17_{\pm.13}$	$0.18_{\pm.15}$	$0.20_{\pm.11}$	$0.17_{\pm.11}$	$0.18_{\pm.12}$	$0.19_{\pm.13}$	$0.20_{\pm.13}$	$0.19_{\pm.14}$	$0.19_{\pm.14}$
IsoScore	$0.23_{\pm.15}$	$0.34_{\pm.06}$	$0.30_{\pm.07}$	$0.24_{\pm.10}$	$0.24_{\pm.11}$	$0.25_{\pm.10}$	$0.24_{\pm.10}$	$0.27_{\pm.09}$	$0.26_{\pm.10}$
UTS-PCA	$0.49_{\pm.03}$	$0.56_{\pm.05}$	$0.55_{\pm.10}$	$0.49_{\pm.01}$	$0.49_{\pm.01}$	$0.50_{\pm.02}$	$0.48_{\pm.03}$	$0.52_{\pm.01}$	$0.52_{\pm.02}$

Table 1: Spearman’s

\rho

for retrieval performance prediction.

Figure 5: Feature importance across CV-folds.

Method	Model	Family	Architecture	Activation
Random	0.04	0.08	0.50	0.33
Most frequent	0.04	0.17	0.72	0.59
Embeddings	$0.26_{\pm.01}$	$0.37_{\pm.02}$	$0.78_{\pm.01}$	$0.38_{\pm.01}$
TwoNN dimension	$0.02_{\pm.00}$	$0.09_{\pm.01}$	$0.02_{\pm.00}$	$0.32_{\pm.01}$
Effective rank	$0.39_{\pm.02}$	$0.47_{\pm.02}$	$\mathbf{0.98}_{\pm.00}$	$0.78_{\pm.02}$
Mean similarity	$0.23_{\pm.05}$	$0.46_{\pm.04}$	$0.63_{\pm.04}$	$0.56_{\pm.02}$

Table 2: Cross-validated balanced accuracy of UTS versus baselines.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 12 — Figure 12: IsoScore of embedding spaces by different models and datasets sorted by MTEB score.

Figure 14: Average correlations of topological descriptors and model properties.

Figure 17: Average distance of UTS components across all models.

Figure 18: Relationship between normalized Recall@20 and effective rank for different datasets.

	Model			Model family			Architecture			Activation
Dim.	Acc.	#Cls	BL	Acc.	#Cls	BL	Acc.	#Cls	BL	Acc.	#Cls	BL
All	$0.66\pm 0.02$	27	0.04	$0.68\pm 0.07$	12	0.17	$0.98\pm 0.01$	2	0.72	$0.84\pm 0.12$	3	0.59
4096	$0.89\pm 0.10$	3	0.37	$1.00\pm 0.00$	2	0.73	$1.00\pm 0.00$	1	1.00	$1.00\pm 0.00$	1	1.00
1024	$0.68\pm 0.23$	5	0.20	$0.68\pm 0.23$	5	0.20	$0.78\pm 0.26$	2	0.80	$0.78\pm 0.26$	2	0.80
768	$0.73\pm 0.12$	8	0.13	$0.83\pm 0.15$	6	0.38	$1.00\pm 0.00$	1	1.00	$0.86\pm 0.06$	2	0.63
384	$0.76\pm 0.01$	4	0.25	$0.93\pm 0.13$	2	0.50	$1.00\pm 0.00$	1	1.00	$1.00\pm 0.00$	1	1.00

Appendix

Table 3: Balanced accuracy for predicting model-specific properties from topological signatures across all folds, controlling for embedding dimensionality. #Cls: number of classes; BL: majority class baseline.

Figure 19: Clustering of Unified Topological Signatures and their normalized Recall@100.

Figure 20: Retrievability bias of encoder and decoder models on different datasets.

Figure 21: Comparison of accuracies by different models for predicting retrievability.

	Model	Model Family	Architecture	Activation
All	$\mathbf{0.66\pm 0.02}$	$\mathbf{0.68\pm 0.07}$	$\mathbf{0.98\pm 0.01}$	$\mathbf{0.84\pm 0.10}$
Remove PH descriptors	$0.64\pm 0.02$	$0.67\pm 0.07$	$0.98\pm 0.01$	$0.82\pm 0.07$
Remove ID descriptors	$0.62\pm 0.02$	$0.67\pm 0.07$	$0.98\pm 0.01$	$0.84\pm 0.09$
Remove Density descriptors	$0.41\pm 0.01$	$0.40\pm 0.07$	$0.98\pm 0.01$	$0.70\pm 0.20$
Remove Isotropy descriptors	$0.39\pm 0.01$	$0.47\pm 0.07$	$0.98\pm 0.01$	$0.78\pm 0.20$

Appendix

Table 4: Balanced accuracy (mean

\pm

standard deviation across cross-validation folds) for predicting model-specific properties after successively removing descriptor groups.

	Model	Model Family	Architecture	Activation
Max normalization	$\mathbf{0.66\pm 0.02}$	$\mathbf{0.68\pm 0.07}$	$\mathbf{0.98\pm 0.01}$	$\mathbf{0.84\pm 0.10}$
Mean normalization	$0.66\pm 0.04$	$0.68\pm 0.09$	$0.98\pm 0.01$	$0.83\pm 0.05$
Robust scaler	$0.65\pm 0.04$	$0.68\pm 0.09$	$0.98\pm 0.01$	$0.84\pm 0.05$
Standard scaler	$0.65\pm 0.04$	$0.68\pm 0.09$	$0.98\pm 0.01$	$0.84\pm 0.06$

Appendix

Table 5: Balanced accuracy (mean

\pm

standard deviation across cross-validation folds) obtained using different normalization procedures.

	Model	Model Family	Architecture	Activation
Random Forest	$\mathbf{0.66\pm 0.02}$	$\mathbf{0.68\pm 0.05}$	$\mathbf{0.98\pm 0.01}$	$\mathbf{0.84\pm 0.10}$
MLP	$0.57\pm 0.04$	$0.66\pm 0.05$	$0.96\pm 0.01$	$0.77\pm 0.09$
Nearest Neighbor	$0.49\pm 0.07$	$0.63\pm 0.07$	$0.93\pm 0.04$	$0.78\pm 0.10$

Appendix

Table 6: Impact of classifier choice on downstream prediction tasks. Balanced accuracy (mean

\pm

standard deviation across cross-validation folds) obtained using different prediction models.

Descriptor	Min	Mean	Max
Magnitude Area	43.8468	1314.3146	1735.8951
Magnitude Dimension	2.2974	4.9628	5.6931
PCA Dimension	34.0000	118.6526	310.0000
TwoNN Dimension	4.7281	20.0096	37.0267
Silhouette Score	0.0059	0.0376	0.2335
Isolation Score	0.0000	0.1255	0.4551

Appendix

Table 7: Distribution of descriptor values across all embedding spaces.

Figure 28: Differences between clusters of highly retrievable and non-retrievable documents for QuoraRetrieval dataset.

Figure 29: Feature importance for predicting retrievability.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance
May 21, 2026Amanda Myntti, Jenna Kanerva, Veronika Laippala +1Text Embedding Evaluation Text Embedding Models
TurkuNLP, University of Turku, Finland · ELLIS Institute Finland
Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
Jun 28, 2026V. S. Raghu ParupudiSemantic Textual Similarity Neural Representation Geometry
University of California, San Diego
Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework
Jul 26, 2026Madhav S BaidyaModel Selection Text Embedding Evaluation
Indian Institute of Technology (BHU) Varanasi

May 21, 2026J/K move · Enter open · S save

Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance

Amanda Myntti, Jenna Kanerva, Veronika Laippala, Filip Ginter

TurkuNLP, University of Turku, Finland · ELLIS Institute Finland

Text Embedding Evaluation Text Embedding Models Latent Space Geometry

Open paper AskPDF arXiv

In this paper, we show that high-performing embedding models organize their embedding spaces in a consistent way. We evaluate 25 contemporary embedding models on five MTEB tasks spanning four diverse task categories (retrieval, bitext mining, pair classification, and summarization) in both English and multilingual settings, and reveal that nearest-neighbor overlap and magnitude differences in independent component analysis (ICA) between paired text instances strongly correlate (even up to 0.97) with performance on the given task. Ultimately, we show that embedding tasks display varying degrees of linearity and reliance on retention of local information. Our results further the understanding of embeddings, their relation to model performance, and shed light on possible future training objectives and optimizing conditional embeddings.

Abstract

Representation Geometry in Language Models

Explore similar work

Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance

Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework