Retrieval Grounding Latent Reasoning for Dense Retrieval

Gang Zhou; Xiongxi Yu; Hu Tian; Yang Wei; Lu Pan; Ke Zeng; Shibiao Xu; Xiaolong Zheng

cs.AIAug 14, 2026

Retrieval Grounding Latent Reasoning for Dense Retrieval

Authors: Gang Zhou, Xiongxi Yu, Hu Tian, Yang Wei, Lu Pan, Ke Zeng, Shibiao Xu, Xiaolong Zheng

Organizations: School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Meituan LongCat Interaction Team · School of Management Science and Engineering, Central University of Finance and Economics · Institute of Automation, Chinese Academy of Sciences

Abstract

Reasoning-intensive retrieval requires text representations to capture not only semantic similarity, but also the reasoning needed to determine relevance under a given retrieval instruction. Existing reasoning-enhanced embedding models improve retrieval by incorporating reasoning information into dense representations, yet their supervision is typically dominated by the final retrieval objective. As a result, latent reasoning trajectories may learn shortcut reasoning patterns that preserve retrieval performance without producing meaningful incremental retrieval gains. We propose Retrieval Grounding Latent Reasoning (RGLT), a latent reasoning framework for dense retrieval that explicitly connects intermediate latent transitions with retrieval improvements. RGLT performs non-autoregressive reasoning in hidden space through an instruction-conditioned latent reasoning trajectory constructed from silent tokens. It combines process-supervised explicit-to-implicit distillation with retrieval-grounded supervision, using stage-wise CoT reconstruction to shape intermediate latent states and retrieval-effect credit to optimize incremental retrieval gains across the latent reasoning trajectories. Experiments on reasoning-intensive retrieval benchmarks show that RGLT consistently outperforms strong baselines while preserving efficient embedding inference.

Figures & tables

Figure 1: Comparison of latent reasoning supervision. Existing methods supervise latent states through state alignment, which does not guarantee retrieval gains during reasoning evolution. RGLT instead transfers stage-wise ranking effects

\Delta^{*}_{k}

from explicit CoT to latent transitions, ensuring that each reasoning step contributes to retrieval improvement.

Figure 2: Architecture of instruction-conditioned latent reasoning. The query and instruction are first compressed into a unified anchor state. During encoding, the anchor explicitly modulates the key and value representations of appended silent tokens via a gated residual to guide reasoning. Finally, the hidden state of the terminal silent token is directly extracted as the final query embedding.

Figure 3: The training pipeline of RGLT. It illustrates the generation of the latent reasoning trajectory and the subsequent joint optimization using three complementary losses: text reconstruction loss ( $\mathcal{L}_{\mathrm{rec}}$ ), retrieval-effect transfer loss ( $\mathcal{L}_{\mathrm{eff}}$ ), and direct retrieval supervision loss ( $\mathcal{L}_{\mathrm{ret}}$ ).

Standard dense retrievers
Model	Size	BRIGHT		FollowIR					BrowseComp-Plus
		nDCG@10	R@10	Robust04	News21	Core17	Avg.	Avg.	R@5	R@100	R@1000
				MAP@5	nDCG@5	MAP@5	Score	p-MRR
BGE-M3	0.6B	11.40	14.39	1.50	21.40	7.50	10.10	-3.90	4.10	21.70	47.20
E5-Mistral-7B-Instruct	7B	16.50	20.12	2.70	28.80	13.70	15.10	-1.50	9.30	37.10	70.20
Qwen3-Embedding-8B	8B	14.00	17.31	3.10	25.50	10.10	12.90	7.20	7.70	31.60	61.30

Table 1: Overall performance on reasoning-intensive retrieval benchmarks. All values are percentages.

Panel of Figure 4 — Figure 4: Domain-level BRIGHT nDCG@10. RGLT outperforms LaSER on 9 of 12 domains.

Variant	nDCG@10	R@10
RGLT	34.20	39.97
w/o stage retrieval-effect matching	31.26	36.15
w/o intent retrieval-effect matching	31.69	36.85
w/o ST16-vs-base advantage	26.39	32.56
replace effect matching with state alignment	30.64	35.69
Gated retrieval-credit fusion variant	32.14	37.94

Table 2: Component ablations and architectural variants on BRIGHT. The lower block contains independently trained variants and is not a one-factor ablation.

Query representation	nDCG@10	R@10
Base query state	26.39	30.24
Stage 1 terminal (ST4)	32.25	34.96
Stage 2 terminal (ST8)	32.48	35.25
Stage 3 terminal (ST12)	33.56	37.14
Stage 4 terminal (ST16)	34.20	39.97

Table 3: BRIGHT performance of cumulative latent stages. ST4–ST16 are the terminal states of four-token stages.

Method	Latency	Relative	Text gen.	Index cost
	(ms/query)
Base retriever (Qwen3-8B)	$21.5$	$1.00\times$	No	None
Rewrite-then-Retrieve	$4000$	$186\times$	Yes	None
LaSER (Qwen3-8B)	$32.5$	$1.51\times$	No	None
RGLT (Qwen3-8B)	24.0	$\textbf{1.12}\times$	No	None

Table 4: Query-side inference efficiency on BRIGHT. Latency is measured over 80 queries with batch size 8 on one NVIDIA A100 80GB GPU.

Figure 5: Impact of the latent budget (

K

) on retrieval performance.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Backbone	Qwen3-8B
GPU	$4\times 8$ NVIDIA A100 80GB
Precision / attention	bfloat16 / FlashAttention 2
Per-GPU / global query batch	4 / 128
Per-query candidates	1 positive + 3 hard negatives
Cross-device document pool	512

Appendix

Table 5: Training configuration.

Dense Retrieval

Dense retrieval uses learned vector representations to find relevant documents, passages, products, or tools by semantic similarity.

See all 152 papers

LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval
Mar 2, 2026Jiajie Jin, Yanzhao Zhang, Mingxin Li +4Dense Retrieval Information Retrieval
Gaoling School of Artificial Intelligence, Renmin University of China · Tongyi Lab, Alibaba Group
Learning When to Think: Utility-Guided Adaptive Reasoning for Efficient Text Retrieval
Oct 15, 2026Yingtao Luo, Kangyi Zhao, Shiyi Du +1Text Embedding Models Efficient Language Model Reasoning
Carnegie Mellon University · University of Pittsburgh
When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models
Apr 29, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuRetrieval-Augmented Generation Retrieval-Augmented Reasoning
The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China

Mar 2, 2026J/K move · Enter open · S save

LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval

Jiajie Jin, Yanzhao Zhang, Mingxin Li, Dingkun Long+3

Gaoling School of Artificial Intelligence, Renmin University of China · Tongyi Lab, Alibaba Group

Dense Retrieval Information Retrieval Implicit Reasoning in Language Models Self-Distillation

Open paper AskPDF arXiv

LLMs have fundamentally transformed dense retrieval, upgrading backbones from discriminative encoders to generative architectures. However, a critical disconnect remains: while LLMs possess strong reasoning capabilities, current retrievers predominantly utilize them as static encoders, leaving their potential for complex reasoning unexplored. To address this, existing approaches typically adopt rewrite-then-retrieve pipelines to generate explicit CoT rationales before retrieval. However, this incurs prohibitive latency. In this paper, we propose LaSER, a novel self-distillation framework that internalizes explicit reasoning into the latent space of dense retrievers. Operating on a shared LLM backbone, LaSER introduces a dual-view training mechanism: an Explicit view that explicitly encodes ground-truth reasoning paths, and a Latent view that performs implicit latent thinking. To bridge the gap between these views, we design a multi-grained alignment strategy. Beyond standard output alignment, we introduce a trajectory alignment mechanism that synchronizes the intermediate latent states of the latent path with the semantic progression of the explicit reasoning segments. This allows the retriever to think silently and effectively without autoregressive text generation. Extensive experiments on both in-domain and out-of-domain reasoning-intensive benchmarks demonstrate that LaSER significantly outperforms state-of-the-art baselines. Furthermore, analyses across diverse backbones and model scales validate the robustness of our approach, confirming that our unified learning framework is essential for eliciting effective latent thinking. Our method successfully combines the reasoning depth of explicit CoT pipelines with the inference efficiency of standard dense retrievers.

Abstract

Dense Retrieval

Explore similar work

LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval

Learning When to Think: Utility-Guided Adaptive Reasoning for Efficient Text Retrieval

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models