Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in multi-step reasoning and calling search tools at appropriate steps. However, existing retrieval-augmented reasoning approaches still rely on separate retrieval models, limiting LRMs to deciding when to retrieve and how to formulate queries rather than leveraging their pretrained knowledge and reasoning capabilities for retrieval itself. This structural separation introduces a representation bottleneck, as retrieval relies on separately learned representations that may obscure fine-grained distinctions needed for reasoning. Moreover, these approaches commonly employ fixed top-k retrieval, which can introduce superfluous documents and increase retrieval noise and context overhead. To address this, we reformulate retrieval from query-document relevance matching into the exploration of answer-containing paths within the corpus, and propose FREESON (Retriever-FREE Retrieval-Augmented ReaSONing), a framework that enables LRMs to directly navigate the original corpus text to locate information that satisfies their specific needs, serving as both the reasoner and the retriever. Specifically, we introduce CT-MCTS (Corpus-Traversing Monte Carlo Tree Search), a retrieval-specialized variant of Monte Carlo Tree Search (MCTS). CT-MCTS iteratively expands and evaluates exploration paths, starting from multiple promising entry points across the corpus. This allows LRMs to selectively reference specific segments identified during traversal, instead of fetching a fixed top-k set. Across nine benchmarks, FREESON improves average EM and F1 by 15 points over four retriever-based baselines on five general-domain QA tasks, matching the strongest baseline. It also gains 12 points on fact-checking and 13 points on medical QA under zero-shot transfer, performing on par with the much larger MedGemma-27B.
Figures & tables
Figure 1: Overview of retrieval-augmented reasoning on (a) domain-specific evidence navigation and (b) fine-grained entity disambiguation. FREESON (left) runs CT-MCTS over a subject-adapted CorpusTree, expanding nodes of G tokens ( G=6 in (a) and G=3 in (b); colored by simulation order), with each simulation scored by a rollout (not shown). Search-R1 (right) retrieves related but non-answering documents using a dense retriever. Traces and retrieved passages are verbatim model outputs.
Fact-checking
Multi-hop QA
Single-hop QA
Method
FEVER
2WikiMultihopQA
MuSiQue
HotpotQA
PopQA
TriviaQA
ACC
EM
F1
EM
F1
EM
F1
EM
F1
EM
F1
Reasoning without retrieval
GLM-4.7-Flash-30B-A3B
0.66
0.29
0.35
0.09
0.18
0.27
0.37
0.30
0.35
0.64
0.73
Qwen3.8-27B
0.69
0.30
0.35
0.10
0.18
0.26
0.37
0.23
0.27
0.55
0.64
Gemma-4-12B
0.61
0.24
0.28
0.06
0.14
0.23
0.33
0.19
0.22
0.53
0.62
Table 1: Overall performance on single- and multi-hop QA, and fact-checking tasks. Bold and underlining indicate the best and second-best results, respectively. FREESON operates without an external retriever, yet matches or outperforms the baselines.
Method
MedQA-US
PubMedQA
BioASQ
ACC
ACC
ACC
Medical LMs
MedGemma-27B
0.88
0.43
0.81
BioMistral-7B
0.39
0.19
0.73
MedGemma-1.5-4B
0.67
0.51
0.67
Retrieve-then-reason
Table 2: Zero-shot transfer to the medical domain. FREESON is evaluated using the same experimental settings as in Table 1 , replacing only the corpus with 125.8K medical textbook snippets. FREESON-7B achieves competitive performance with MedGemma-27B.
Method
Top-k
# Docs
Coverage
Prec.
Search-R1 + E5
3
2.99
0.87
0.34
FREESON
3
1.24
0.83
0.81
Table 3: Retrieval quality on 2WikiMultihopQA. FREESON achieves 2.4 × higher precision while maintaining comparable evidence coverage.
Model
Value estimation
MedQA-US
PubMedQA
BioASQ
Avg.
ACC
ACC
ACC
ACC
GPT-OSS-20B
Self-eval
85.3
56.7
84.7
75.57
vh_medical
85.0
59.3
84.7
76.33
vh_general
85.0
58.7
85.3
76.33
Gemma-4-12B
Self-eval
85.3
54.7
79.3
73.10
vh_medical
84.3
55.3
79.3
72.97
Table 4: Comparison of self-evaluation and learned value estimators under in-domain and zero-shot transfer settings. All three approaches achieve comparable downstream QA performance.
PopQA
2WikiMultihopQA
Decoding
EM
F1
EM
F1
Greedy Search
0.17
0.21
0.21
0.23
Beam Search
0.18
0.21
0.23
0.25
CT-MCTS
0.44
0.45
0.54
0.60
Table 5: Comparison of decoding strategies.
Figure 2: Impact of node granularity (1 – 10 tokens per node) on efficiency and performance. Left : Higher granularity reduces latency (s). Middle : Retrieved paths become longer while rollout counts remain similar, indicating more efficient search per computation. Right : Higher granularity generally improves performance.
PopQA
TriviaQA
2WikiMultiHop
Avg.
Value estimation
EM
F1
EM
F1
EM
F1
EM
F1
LLM Synthetic Trajectory
0.41
0.41
0.48
0.61
0.53
0.59
0.47
0.54
CT-MCTS Rollout
0.44
0.45
0.51
0.63
0.54
0.60
0.50
0.56
Table 6: Comparison of on-policy and off-policy value learning. CT-MCTS rollouts yield consistently higher QA performance than LLM-generated synthetic trajectories.
Performance
Efficiency
Multiple Expansion
EM
F1
Latency
M=1
0.41
0.46
21s
M=2
0.54
0.60
31s
M=3
0.54
0.60
36s
M=4
0.54
0.60
44s
Table 7: Effect of the number of expanded nodes ( M ) per simulation. On 2WikiMultihopQA, performance improves significantly at M=2 and quickly saturates due to constrained token space.
Efficiency
Metric
FREESON
Dual-encoder
Indexing time ( ↓ )
64 mins (cpu)
177 mins (gpu)
Index memory ( ↓ )
11GB
61GB
Inference latency ( ↓ )
17.21s
9.36s
Inference FLOPs ( ↓ )
∼5.5×1013
∼3.1×1013
Table 8: Comparison of indexing time, index memory, inference latency, and FLOPs between FREESON and Search-R1. FREESON builds an FM-index, while the dual encoder uses embedding-based indexing.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Settings
FEVER
PopQA
TriviaQA
HotPotQA
2WikiMultihopQA
MuSiQue
TriviaQA
[ 56 ]
[ 54 ]
[ 59 ]
[ 58 ]
[ 55 ]
[ 57 ]
[ 59 ]
Dataset statistics
# Examples
100
500
500
500
500
500
500
Gold answer count
Single
Many
Single
Single
Single
Single
Single
Retrieval settings
Corpus
Wikipedia-dpr
Wikipedia-dpr
Wikipedia-dpr
Wikipedia-dpr
Wikipedia-2wiki
Wikipedia-dpr
Wikipedia-dpr
Appendix
Table 9: Dataset statistics and retrieval settings for general-domain and medical QA benchmarks. We randomly sample evaluation examples to reduce computational cost. “Many” denotes multiple gold answers, while “Single” denotes one.
Method
Subset ( N=500 )
Full Set ( N=12,576 )
EM / F1
EM / F1
Search-R1 + BM25
–
0.5394 / 0.6085
Search-R1 + E5
0.54 / 0.61
0.5558 / 0.6360
FREESON (Ours)
0.55 / 0.63
0.5742 / 0.6440
Appendix
Table 10: Comparison of subset and full-set evaluation on 2WikiMultihopQA. The results show consistent performance trends across both settings.
Efficiency
Performance
Exploration
Node Granularity
Latency ( ↓ )
Avg Rollout Count
Avg Path Length.
EM ( ↑ )
F1 ( ↑ )
Exploration
G=1
67s
39.14
11
0.30
0.33
32
G=2
60s
63.13
42
0.48
0.54
16
G=3
36s
60.19
60
0.54
0.60
12
G=4
34s
60.60
84
0.52
0.58
10
G=5
30s
60.60
99
0.52
0.59
9
Appendix
Table 11: Analysis of efficiency, performance, and exploration behavior under varying node granularity. Avg Rollout Counts for G=4 and G=5 are imputed with the mean (excluding G=1 ), as single-token nodes ( G=1 ) often fail to yield valid rollouts due to limited semantic context.
Large reasoning models such as DeepSeek-R1 and OpenAI o1 generate extended chains of thought spanning thousands of tokens, yet their integration with retrieval-augmented generation (RAG) remains fundamentally misaligned. Current RAG systems optimize for providing context before reasoning begins, while reasoning models require evidence injection during multi-step inference chains. We introduce ReaLM-Retrieve, a reasoning-aware retrieval framework that addresses this mismatch through three key innovations: (1) a step-level uncertainty detector that identifies knowledge gaps at reasoning-step granularity rather than token or sentence level; (2) a retrieval intervention policy that learns when external evidence maximally benefits ongoing reasoning; and (3) an efficiency-optimized integration mechanism that reduces per-retrieval overhead by 3.2x compared to naive integration. Experiments on MuSiQue, HotpotQA, and 2WikiMultiHopQA demonstrate that ReaLM-Retrieve achieves on average 10.1% absolute improvement in answer F1 over standard RAG (range: 9.0-11.8% across the three benchmarks) while reducing retrieval calls by 47% compared to fixed-interval approaches like IRCoT (all improvements significant at p<0.01, paired bootstrap). On the challenging MuSiQue benchmark requiring 2-4 hop reasoning, our method achieves 71.2% F1 with an average of only 1.8 retrieval calls per question. Analysis shows that ReaLM-Retrieve also improves retrieval quality itself, achieving 81.3% Recall@5 with consistently higher precision and MRR than fixed-interval baselines on supporting evidence, establishing new state-of-the-art efficiency-accuracy trade-offs for reasoning-intensive retrieval tasks.
Dongxin Guo, Jikun Wu, Siu Ming Yiu
The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China
Retrieval-Augmented Generation (RAG) expands the knowledge of Large Language Models (LLMs), yet current static retrieval methods struggle with complex, multi-hop problems. While recent dynamic retrieval strategies offer improvements, they face two key challenges: 1) irrelevant retrieved noise can misdirect the reasoning process, and 2) processing full documents incurs prohibitive computational and latency costs. To address these issues, we propose OThink-SRR1, a framework that enhances large models with an iterative Search-Refine-Reason process trained via reinforcement learning. Its core Refine stage distills retrieved documents into concise, relevant facts before reasoning. We introduce GRPO-IR, an end-to-end reinforcement learning algorithm that rewards accurate evidence identification while penalizing excessive retrievals, thus training the model to be both focused and efficient. Experiments on four multi-hop QA benchmarks show our approach achieves superior accuracy over strong baselines while using fewer retrieval steps and tokens. This positions OThink-SRR1 as a potent foundational model for information-seeking agents.
Deep search agents have proven effective in enhancing LLMs by retrieving external knowledge during multi-step reasoning. However, existing methods often generate a single query for retrieval at each reasoning step, limiting information coverage and introducing high noise. This may result in low signal-to-noise ratios (SNR) during search, degrading reasoning accuracy and leading to unnecessary reasoning steps. In this paper, we introduce MultiSearch, an RL-based framework that addresses these limitations through multi-query retrieval and explicit merging of retrieved information. At each reasoning step, MultiSearch generates queries from multiple perspectives and retrieves external information in parallel, expanding the scope of relevant information and mitigating the reliance on any single retrieval result. Then, the agent consolidates and refines retrieved information at the merging process, improving the SNR and ensuring more accurate reasoning. Additionally, we propose a reinforcement learning framework with a multi-process reward design to optimize agents for both multi-query retrieval and information consolidation. Extensive experiments on seven benchmarks demonstrate that MultiSearch outperforms baseline methods, enhancing the SNR of retrieval and improving reasoning performance in question-answering tasks.
Jiabei Liu, Wenyu Mao, Junfei Tan +4
University of Science and Technology of China · WeChat Technical Architecture Department, Tecent Inc.