ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers

Effrosyni Sokli; Isaac Roberts; Alexander Schulz; Barbara Hammer; Gabriella Pasi

cs.IRSep 5, 2026

ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers

Authors: Effrosyni Sokli, Isaac Roberts, Alexander Schulz, Barbara Hammer, Gabriella Pasi

Organizations: Department of Informatics, Systems and Communication, DISCo University of Milano-Bicocca Milan, Italy · CITEC - Center for Cognitive Interaction Technology Bielefeld University Bielefeld, Germany

Abstract

Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.

Figures & tables

Figure 1. The ExpertLens framework. Phase I: Visualization of the MoE induced embedding space through discriminative DeepView projections for structure analysis. Phase II: Examination of the partitioning of the embedding space into subspaces and association with expert specialization. Phase III: CAVs are automatically extracted and mapped to individual experts.

Panel of Figure 2 — Figure 2. UMAP (a) and DeepView (b & c) visualizations of the embedding space for the same query (yellow circle) and its relevant document (red circle) across three model variants: (a) Fine-tuned (without MoE), (b) Unsupervised MoE (SB-MoE), and (c) Supervised MoE (DenseC3). Each plot shows the top-1000 ranked documents retrieved for this query by each model, with colors indicating expert assignment in MoE variants.

Panel of Figure 3 — Figure 3. DeepView plot of Query ID: “181476" from MSMARCO by the Unsupervised MoE variant (top), and Query ID: “5a7199725542994082a3e88f" from HotpotQA by the Supervised MoE variant (bottom).

Dataset	Corpus Size	Train Queries	Test Queries
MSMARCO v1 passage corpus (MSMARCO ( Nguyen et al., 2016 ) )	8.8M passages	532k (1.1 avg rel.)	7k
TREC Deep Learning Track 2019 (TREC DL 19 ( Craswell et al., 2020 ) )	8.8M passages	503k (215 avg rel.)	43
TREC Deep Learning Track 2020 (TREC DL 20 ( Craswell et al., 2021 ) )	8.8M passages	503k (211 avg rel.)	54
Natural Questions (NQ ( Kwiatkowski et al., 2019 ) )	2.6M passages	132k (1.2 avg rel.)	3.5k
HotpotQA ( Yang et al., 2018 )	5.2M documents	85k (2 avg rel.)	7.4k

Table 1. Dataset statistics. The average number of relevance judgments per query is indicated in parentheses.

Model	Parameters	Embedding Size	Pre-Training Strategy
TinyBERT ( Jiao et al., 2020 )	14.5M	312	Knowledge Distillation
BERT-Small ( Turc et al., 2019 )	29M	512	Knowledge Distillation
BERT-Base ( Devlin et al., 2019 )	110M	768	Self-Supervised Learning
ColBERT ( Khattab and Zaharia, 2020 )	110M	768	IR Fine-Tuned

Table 2. Backbone bi-encoder models statistics. Parameter count, embedding size, and pre-training strategy.

Panel of Figure 4 — Figure 4. Discriminative UMAP (center) reveals clear clusters of documents processed by the same experts for a given query compared to regular UMAP (left). Evaluation metrics across $\lambda$ values for query 5a78c56455429974737f7876 (right). Unsupervised MoE on HotpotQA.

		TREC19			TREC20			MSMARCO			NQ			HotpotQA
Retriever	Variant	P@20	P@100	$\text{cos}_{dist}\downarrow$	P@20	P@100	$\text{cos}_{dist}\downarrow$	P@1	P@2	$\text{cos}_{dist}\downarrow$	P@1	P@2	$\text{cos}_{dist}\downarrow$	P@1	P@2	$\text{cos}_{dist}\downarrow$
TinyBERT	fine-tuned	.333	.171	.341	.276	.107	.334	.120	.095	.243	.129	.105	.239	.256	.175	.349
	unsupervised	.341	.175	.368	.276	.107	.360	.118	.094	.262	.137	.113	.268	.263	.178	.382
	supervised	.330	.171	.417	.278	.109	.406	.123	.095	.298	.148	.118	.309	.260	.177	.406
BERT-Small	fine-tuned	.344	.184	.193	.268	.114	.191	.115	.097	.157	.085	.075	.202	.308	.217	.263
	unsupervised	.349	.185	.220	.268	.113	.218	.115	.097	.180	.088	.078	.225	.316	.222	.259

Table 3. Assessment of the neighborhood structure within three different embedding spaces, as these are formulated with and without the employment of two contrastive MoE variants across all datasets. Scores measure the precision@k per model and dataset at different cutoffs (%), as well as the absolute proximity of each relevant document

d^{+}

to its corresponding query

q

via cosine distance. Best results for each setting are in bold text.

	MSMARCO					NQ						HotpotQA
Metric	Corpus	top-1000	E0	E3	E4	Corpus	top-1000	E0	E2	E4	E5	Corpus	top-1000	E0	E1	E2	E3	E5
Docs	8.8M	1000	477.23	38.35	480.64	2.6M	1000	732.10	27.38	15.94	217.39	5.2M	1000	418.99	85.59	13.24	367.33	114.24
$ari\downarrow$	$8.762$	$8.500$	$\underline{7.742}$	$8.746$	${9.234}$	$14.329$	$13.521$	${13.653}$	$12.618$	$\underline{11.485}$	$12.975$	$10.270$	$10.115$	$\underline{10.004}$	$10.270$	${10.947}$	$10.143$	$10.367$
$H$	$0.542$	$0.765$	$\underline{0.792}$	$\mathbf{0.875}$	$0.796$	$0.562$	$0.751$	$\underline{0.761}$	$0.889$	$\mathbf{0.902}$	$0.813$	$0.525$	$0.736$	$\underline{0.771}$	$0.852$	$\mathbf{0.907}$	$0.776$	$0.838$
$R_{\text{Voc}}$	$0.003$	$0.182$	$\underline{0.262}$	$\mathbf{0.549}$	$0.264$	$0.006$	$0.180$	$\underline{0.208}$	$0.663$	$\mathbf{0.731}$	$0.373$	$0.013$	$0.205$	$\underline{0.295}$	$0.525$	$\mathbf{0.732}$	$0.299$	$0.476$
$CP$	$0.600$	$0.674$	$\underline{0.703}$	$\mathbf{0.798}$	$0.707$	$0.583$	$0.649$	$\underline{0.659}$	$0.810$	$\mathbf{0.843}$	$0.714$	$0.656$	$0.695$	$0.723$	$0.783$	$\mathbf{0.829}$	$\underline{0.721}$	$0.767$

Table 4. Textual characteristics analysis for Unsupervised (SB-MoE), reporting the mean of each metric across the top-100 DBSCAN-selected queries per dataset. Considering active experts only, i.e., those handling at least 10 documents for a given query on average per dataset. Bold : highest value per expert per dataset; underline : lowest value per expert per dataset.

	MSMARCO							NQ							HotpotQA
Metric	Corpus	Top-1k	E1	E2	E3	E4	E5	Corpus	Top-1k	E1	E2	E3	E4	E5	Corpus	Top-1k	E2	E3	E5
Docs	8.8M	1000	96.07	146.22	484.94	51.96	212.27	2.6M	1000	70.25	93.58	528.87	34.09	268.46	5.2M	1000	79.10	451.40	451.86
$ari\downarrow$	$8.762$	$8.839$	$\underline{}{8.337}$	${9.471}$	$8.505$	$9.347$	$9.398$	$14.329$	$13.051$	$12.904$	$\underline{}{12.358}$	${13.573}$	$14.819$	$12.583$	$10.270$	$9.993$	${11.023}$	$\underline{9.252}$	$10.508$
$H$	$0.542$	$0.756$	$0.839$	$0.819$	$\underline{0.778}$	$\mathbf{0.860}$	$0.814$	$0.562$	$0.749$	$0.840$	$0.832$	$\underline{0.772}$	$\mathbf{0.860}$	$0.789$	$0.525$	$0.734$	$\mathbf{0.833}$	$\underline{0.762}$	$0.767$
$R_{\text{Voc}}$	$0.003$	$0.163$	$0.383$	$0.338$	$\underline{0.223}$	$\mathbf{0.470}$	$0.325$	$0.006$	$0.179$	$0.483$	$0.456$	$\underline{0.248}$	$\mathbf{0.553}$	$0.294$	$0.013$	$0.197$	$\mathbf{0.468}$	$0.282$	$\underline{0.269}$
$CP$	$0.600$	$0.678$	$0.754$	$0.731$	$\underline{0.710}$	$\mathbf{0.776}$	$0.721$	$0.583$	$0.648$	$0.744$	$0.738$	$\underline{0.678}$	$\mathbf{0.766}$	$0.683$	$0.656$	$0.697$	$\mathbf{0.760}$	$0.739$	$\underline{0.708}$

Table 5. Textual characteristics analysis for Supervised (DenseC3), reporting the mean of each metric across the top-100 DBSCAN-selected queries per dataset. Considering active experts only, i.e., those handling at least 10 documents for a given query on average per dataset. Bold : highest value per expert per dataset; underline : lowest value per expert per dataset.

Panel of Figure 5 — Figure 5. Sankey diagrams displaying how concepts propagate through the experts.

Panel of Figure 6 — Figure 6. Expert assignment distributions on all datasets as reported in the original paper of the supervised MoE variant (DenseC3) ( Sokli et al., 2025b ) .

Panel of Figure 7 — Figure 7. We plot the $Q_{\text{concept}}$ error, including 1%-10% of the top concepts for each dataset (denoted by color) and embedding type (denoted by line type).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8. The zero-shot prompt provided to Gemma-3-27b to automatically generate semantic labels and descriptions for discovered SAE concepts based on top-activating queries. The shown concepts are from the HotpotQA dataset.

Interpretable ML

Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.

See all 304 papers

Xetrieval: Mechanistically Explaining Dense Retrieval
May 28, 2026Zhixin Cai, Jun Bai, Yang Liu +7Text Embedding Models Dense Retrieval
School of Computer Science and Engineering, Beihang University · 2State Key Laboratory of General Artificial Intelligence, BIGAI
Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers
Sep 11, 2026Anubhav Shrestha, Safal Shrestha, Minwu Kim +2Information Retrieval Mixture-of-Experts Language Models
New York University Abu Dhabi · New York University
The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level
Apr 2, 2026Jeremy Herbst, Stefan Wermter, Jae Hee LeeMixture-of-Experts Interpretability Mixture-of-Experts Models
Department of Informatics, University of Hamburg, Hamburg, Germany.

May 28, 2026J/K move · Enter open · S save

Xetrieval: Mechanistically Explaining Dense Retrieval

Zhixin Cai, Jun Bai, Yang Liu, Jiaqi Li+6

School of Computer Science and Engineering, Beihang University · 2State Key Laboratory of General Artificial Intelligence, BIGAI

Text Embedding Models Dense Retrieval

Open paper AskPDF arXiv

Explaining why dense retrievers assign high relevance scores remains challenging because retrieval decisions are made through opaque high-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose Xetrieval, an embedding-level mechanistic framework for explaining dense retrieval. Xetrieval first introduces a lightweight reasoning internalizer that approximates Chain-of-Thought reasoning directly in the embedding space with a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse, human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, Xetrieval provides feature-level explanations of individual retrieval decisions. Experiments on diverse retrievers and benchmarks show that Xetrieval uncovers coherent interpretable features, yields stronger pair-level intervention effects, and supports task-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval .

Abstract

Interpretable ML

Explore similar work

Xetrieval: Mechanistically Explaining Dense Retrieval

Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers

The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level