LLMs have fundamentally transformed dense retrieval, upgrading backbones from discriminative encoders to generative architectures. However, a critical disconnect remains: while LLMs possess strong reasoning capabilities, current retrievers predominantly utilize them as static encoders, leaving their potential for complex reasoning unexplored. To address this, existing approaches typically adopt rewrite-then-retrieve pipelines to generate explicit CoT rationales before retrieval. However, this incurs prohibitive latency. In this paper, we propose LaSER, a novel self-distillation framework that internalizes explicit reasoning into the latent space of dense retrievers. Operating on a shared LLM backbone, LaSER introduces a dual-view training mechanism: an Explicit view that explicitly encodes ground-truth reasoning paths, and a Latent view that performs implicit latent thinking. To bridge the gap between these views, we design a multi-grained alignment strategy. Beyond standard output alignment, we introduce a trajectory alignment mechanism that synchronizes the intermediate latent states of the latent path with the semantic progression of the explicit reasoning segments. This allows the retriever to think silently and effectively without autoregressive text generation. Extensive experiments on both in-domain and out-of-domain reasoning-intensive benchmarks demonstrate that LaSER significantly outperforms state-of-the-art baselines. Furthermore, analyses across diverse backbones and model scales validate the robustness of our approach, confirming that our unified learning framework is essential for eliciting effective latent thinking. Our method successfully combines the reasoning depth of explicit CoT pipelines with the inference efficiency of standard dense retrievers.
Figures & tables
Figure 1. Left: Comparison between our proposed pipeline and conventional retriever and rewrite-then-retrieve method. Right: Comparison of performance and latency of different methods on the Bright benchmark. Letency represents the time consumption of processing a single query. The “Rewrite-then-Retrieve” method uses a 0.6B retriever and a 3B rewriter.
Figure 2. Illustration of the LaSER. During training, each step involves two inference pathways: the explicit-view and the latent-view (shown on the left). The inputs for these two paths differ, with the explicit-view incorporating additional information. However, during the inference phase, only the latent-view mode is utilized. The loss function is computed based on the results from both paths (shown on the right).
Instruction:
Given a question, your mission is to follow the instructions below:
1. Identify the essential problem.
2. Think step by step to reason and describe what information could be relevant and helpful to address the questions in detail.
3. Draft an answer with as many thoughts as you have.
The given question:
[Begin of Question]
Table 1. The prompt used for annotating the explicit reasoning path of the given query.
Split
Dataset
# Q
# C
Q Len.
Reas. Len.
Train
ReasonEmb
81,659
–
222.1
984.8
Test
Bright (ID)
1,384
1,145,164
240.8
2,412.8
FollowIR (OOD)
104
98,312
76.6
–
BC-Plus (OOD)
830
100,195
123.2
–
Table 2. Statistics of the datasets used in our experiments. “Q” and “C” denote Query and Corpus respectively. “Len.” denotes average token count. “Reas. Len.” denotes average reasoning length annotated by external LLM.
Models
Size
StackExchange
Coding
Theorem-based
Avg.
Bio.
Earth.
Econ.
Psy.
Rob.
Stack.
Sus.
Leet.
Pony
AoPS
TheoQ.
TheoT.
Standard Dense Retrievers
BGE-M3
0.6B
7.2
13.3
12.5
12.8
12.6
10.0
10.1
15.6
30.7
1.5
5.6
5.0
11.4
Multilingual-E5-Large-Instruct
0.6B
15.0
24.6
14.0
14.6
16.1
10.3
13.8
15.1
2.1
3.0
11.6
6.9
12.3
E5-Mistral-7B-Instruct
7B
17.9
28.6
18.5
16.5
18.6
13.4
20.0
19.7
6.3
2.0
14.2
22.1
16.5
Qwen3-Embedding-0.6B
0.6B
12.7
26.3
17.9
16.5
12.5
12.4
12.2
14.3
0.7
3.1
17.2
26.5
14.4
Table 3. Main retrieval performance on the Bright benchmark. We report nDCG@10 for all subsets. The best results in the same backbone setting are formatted in bold and the second best are underlined . † indicates the pipeline method that uses an external LLM to rewrite queries during inference.
Models
Size
Robust04
News21
Core17
FollowIR-Avg
BrowseComp-Plus
MAP@5
p-MRR
nDCG@5
p-MRR
MAP@5
p-MRR
Score
p-MRR
R@5
R@100
R@1000
Standard Dense Retrievers
BGE-M3
0.6B
1.5
-1.6
21.4
-1.3
7.5
-8.8
10.1
-3.9
4.1
21.7
47.2
Multilingual-E5-Large-Instruct
0.6B
2.4
2.7
25.0
1.2
8.5
-6.0
12.0
-0.7
10.1
36.8
68.6
E5-Mistral-7B-Instruct
7B
2.7
2.5
28.8
0.8
13.7
-7.8
15.1
-1.5
9.3
37.1
70.2
Qwen3-Embedding-0.6B
0.6B
2.8
8.9
27.3
3.6
11.4
2.7
13.8
5.1
8.0
29.1
64.8
Table 4. Retrieval performance. Columns under Robust04, News21, and Core17 belong to the FollowIR benchmark.
Method
Bright
FollowIR
BrowseComp-Plus
nDCG@10
Score
p-MRR
R@5
R@1000
LaSER (Qwen3-0.6B)
23.10
11.49
-0.02
6.76
54.88
w/o Explicit View
20.59
9.78
-1.71
6.48
50.14
w/o Latent View
19.93
8.62
-0.11
5.93
52.25
w/o Process Align.
22.33
11.63
-1.58
6.18
52.43
w/o Output Distill.
19.97
7.55
0.43
7.03
53.02
Table 5. Ablation study of our proposed method. For FollowIR, we report the average score on all subsets.
Figure 3. Performance comparison of LaSER and baselines across different backbones and model sizes on Bright.
Figure 4. Comparison of the performance of using rewrite path on Bright between our method and basic training.
Figure 5. Effect of latent thinking steps during training and inference.
School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Meituan LongCat Interaction Team · School of Management Science and Engineering, Central University of Finance and Economics +1