Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning
Authors: Bo Li, Mingda Wang, Gexiang Fang, Shikun Zhang, Wei Ye
Organizations: National Engineering Research Center for Software Engineering, Peking University · School of Computer Science, Peking University · School of Health Sciences and Biomedical Engineering, Hebei University of Technology
We revisit retrieval-augmented generation (RAG) by embedding retrieval control directly into generation. Instead of treating retrieval as an external intervention, we express retrieval decisions within token-level decoding, enabling end-to-end coordination without additional controllers or classifiers. Under the paradigm of Retrieval as Generation, we propose GRIP (Generation-guided Retrieval with Information Planning), a unified framework in which the model regulates retrieval behavior through control-token emission. Central to GRIP is Self-Triggered Information Planning, which allows the model to decide when to retrieve, how to reformulate queries, and when to terminate, all within a single autoregressive trajectory. This design tightly couples retrieval and reasoning and supports dynamic multi-step inference with on-the-fly evidence integration. To supervise these behaviors, we construct a structured training set covering answerable, partially answerable, and multi-hop queries, each aligned with specific token patterns. Experiments on five QA benchmarks show that GRIP surpasses strong RAG baselines and is competitive with GPT-4o while using substantially fewer parameters.
Figures & tables
Figure 1: Overview of the GRIP framework. The top half illustrates the training process: four behavior types are constructed for supervised fine-tuning, followed by reinforcement learning to improve answer quality and retrieval control. The bottom half depicts self-triggered information planning, where GRIP dynamically emits control tokens to decide whether to retrieve, generate new queries, or terminate reasoning, all within a unified decoding trajectory.
Figure 2: Four types of structured training samples in GRIP, each aligned with specific token patterns for controlling retrieval behavior. This design enables the model to learn when to retrieve, plan new queries, or finalize answers via language-native supervision.
HotpotQA
PopQA
NQ
WebQ
TriviaQA
Avg. Score
EM
ROUGE
F1
EM
ROUGE
F1
EM
ROUGE
F1
EM
ROUGE
F1
EM
ROUGE
F1
Training-free Method
Instruct
17.2
21.6
25.9
17.4
19.6
23.2
14.4
23.8
20.3
14.8
25.1
29.4
46.2
45.3
55.1
26.6
GPT-3.5 Turbo
26.2
32.4
38.2
29.1
31.1
35.3
20.7
34.3
30.0
15.9
27.4
31.9
55.6
57.8
69.7
35.7
GPT-4o
33.2
40.2
47.0
30.6
32.3
39.9
26.5
42.7
28.3
23.5
32.3
37.0
65.7
64.3
78.2
41.4
Single RAG
26.1
31.6
37.2
22.8
26.6
31.1
19.3
28.4
24.8
14.0
22.7
26.6
46.4
47.0
56.8
30.8
Table 1: Main results on five QA benchmarks. We report EM, ROUGE (average of ROUGE-1/2/L), and F1; Avg.Score is the unweighted mean over all datasets and metrics. All open-source baselines are reproduced using official implementations and released checkpoints under the same evaluation setting, and results are averaged over three random seeds. Results on a domain-specific dataset are provided in Appendix B .
NQ (54.3%)
WebQ (24.3%)
Avg. Score
EM
ROUGE
F1
EM
ROUGE
F1
DRAGIN
30.3
33.9
39.3
42.5
48.4
52.9
41.2
GainRAG
31.6
34.4
39.6
33.0
40.7
45.6
37.4
GRIP
31.1
34.0
39.8
44.9
50.7
55.6
42.7
Table 2: Subset where DRAGIN/GainRAG retrieve at least once while GRIP answers without retrieval. Percentages denote the subset proportion per dataset.
NQ (23.1%)
WebQ (43.8%)
Avg. Score
EM
ROUGE
F1
EM
ROUGE
F1
DRAGIN
11.4
16.7
19.0
18.6
23.8
28.3
19.6
GainRAG
8.8
14.8
17.4
11.1
18.4
22.4
15.5
GRIP
15.4
20.3
23.4
19.9
24.6
28.9
22.1
Table 3: Subset where GRIP performs two retrievals while DRAGIN/GainRAG retrieve at most once. Percentages denote the subset proportion per dataset.
Hotpot
PopQA
NQ
WebQ
Trivia
Avg. Count
DRAGIN
1.13
1.26
1.06
1.02
1.08
1.11
GainRAG
1.0
1.0
1.0
1.0
1.0
1.0
R1-Searcher
5.75
5.34
5.09
4.67
4.76
5.12
GRIP
1.44
1.58
0.76
1.15
1.25
1.24
w/o RL
1.99
1.98
0.77
1.26
1.99
1.60
Table 4: Mean retrieval count per dataset. GRIP adapts retrieval frequency across tasks, and RL further reduces retrieval calls while preserving this adaptivity. Hotpot and Trivia denote HotpotQA and TriviaQA, respectively.
Figure 3: The percentage of samples where the gold answer span is found in the top-1 or top-3 retrieved passages. GRIP-generated queries significantly improve answer presence in retrieved contexts across both NQ and WebQ, indicating more effective retrieval behavior.
Max B
Avg.Count (per dataset)
Avg. Score
Hotpot
PopQA
NQ
WebQ
Trivia
Avg
3
1.44
1.58
0.76
1.15
1.25
1.24
41.0
5
1.56
1.73
1.01
1.45
1.31
1.41
41.2
7
1.66
1.79
1.12
1.65
1.35
1.51
41.5
10
1.74
1.86
1.22
1.90
1.39
1.62
41.8
Table 5: Effect of the maximum retrieval budget B at inference time. Hotpot and Trivia denote HotpotQA and TriviaQA, respectively.
Teacher
Training
Avg.Score
GPT-4o-mini
GRIP (SFT)
40.7
GRIP (SFT+RL)
41.0
LLaMA3-8B-Instruct
GRIP (SFT)
38.8
GRIP (SFT+RL)
39.0
Qwen3-32B
GRIP (SFT)
40.2
GRIP (SFT+RL)
40.3
Table 6: Teacher replacement results, where only the GPT-labeled subset is re-annotated.
Model
MMLU (Acc)
MBPP (Pass@1)
Instruct
66.56
54.6
SFT-RAG
62.93
47.8
GRIP
65.73
53.8
Table 7: General capability evaluation on closed-form non-RAG benchmarks.
Pairwise comparison (GRIP vs.)
Win
Equal
Loss
Instruct
51.0
37.5
11.5
SFT-RAG
90.5
8.5
1.0
R1-Searcher
59.5
32.5
8.0
Table 8: Pairwise GPT-4o evaluation on CNN/DailyMail summarization. Results are averaged over both output orders to mitigate position bias.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
#Samples
Type
HotpotQA
7,405
Multi-hop
PopQA
14,267
Open-domain
NQ
3,610
Open-domain
WebQ
2,032
Open-domain
TriviaQA
11,313
Reading Comprehension
BioASQ
885
Domain-specific
Appendix
Table 9: Statistics and types of the test datasets used in our evaluation. The datasets vary in size and reasoning focus, covering multi-hop, open-domain, reading comprehension, and biomedical QA tasks.
ROUGE
F1
Avg.Score
FLARE
30.2
69.4
49.8
DRAGIN
47.1
81.2
64.2
SFT-RAG
39.1
58.6
48.9
Self-RAG
45.9
68.8
57.4
INFO-RAG
39.4
59.2
49.3
RobustRAG
53.9
80.9
67.4
Appendix
Table 10: Performance on the domain-specific BioASQ benchmark (higher is better). Avg.Score is the unweighted mean of ROUGE and F1.
Hotpot
PopQA
NQ
WebQ
Trivia
Avg. CoverEM
GPT-4o
38.6
45.2
48.5
43.6
82.1
51.6
DRAGIN
31.6
23.0
38.4
41.5
68.0
40.5
GainRAG
38.0
44.4
39.1
36.9
64.2
44.5
GRIP
45.0
49.1
41.2
52.4
74.9
52.5
Appendix
Table 11: CoverEM scores across five QA datasets. GRIP achieves the best average CoverEM. Hotpot and Trivia denote HotpotQA and TriviaQA, respectively.
Figure 4: Type- α : Samples that are directly answerable using the model’s internal knowledge alone, where the model emits [ANSWER] followed by the answer and concludes with [SOLVED] without any retrieval.
Figure 5: Type- β : Samples where the model produces partial or noisy answers containing the gold answer but lacking clarity, encouraging it to recognize uncertainty and issue a retrieval request through [INTERMEDIARY] followed by [RETRIEVE] .
Figure 6: Type- γ : Multi-hop or complex knowledge samples where both parametric memory and basic retrieval fail, requiring the model to iteratively construct better sub-questions and reason across multiple planning steps before concluding with [ANSWER] and [SOLVED] .
Figure 7: Type- θ : Cases where the retrieved content covers the answer superficially but demands further reasoning and content compression, training the model to selectively synthesize key information via [INTERMEDIARY] → [RETRIEVE] → [ANSWER] transitions.
Reward metric for rans
Avg.Score
Avg.Count
BLEU (default)
41.0
1.24
ROUGE
40.5
1.24
F1
40.8
1.25
EM
40.6
1.24
Appendix
Table 12: Reward-metric ablation for RL: we only change the overlap metric used in the answer-fidelity reward rans ; all other settings are fixed.
Figure 8: Comparison of retrieval behavior distributions before and after reinforcement learning on NQ and WebQ.
Figure 9: Example of GRIP falling back to internal knowledge and deferring retrieval when no context is available.
Figure 10: Example of GRIP refining its retrieval query over multiple rounds to obtain a better answer.
Figure 11: Example of GRIP refining, reformulating, and compressing retrieved knowledge into a final answer using [ANSWER] .
Figure 12: PCA visualization of the model’s hidden states before emitting [ANSWER] (blue) and [RETRIEVE] (orange) on the NQ dataset. Each point corresponds to the hidden representation of the last token in the input prompt. The distinct clusters indicate that the model encodes separable internal states for different control decisions.
Figure 13: Prompt used for data generation with GPT-4o-mini.
Method
BM25
DPR
Hybrid
Single-RAG (training-free)
30.8
31.9
33.7
RobustRAG (training-based)
33.2
32.1
33.8
GRIP
41.0
39.2
41.0
Appendix
Table 13: Retriever ablation on the main evaluation suite. We replace the default BM25 retriever with a dense DPR retriever and a BM25+DPR hybrid retriever (denoted Hybrid ), while keeping the backbone (LLaMA3-8B), corpus (Wikipedia), and top- k passages ( k=3 ) fixed. Single-RAG and RobustRAG are representative training-free and training-based baselines from the main table, respectively.
Figure 14: Realized retrieval-count distributions of GRIP under different maximum retrieval budgets B . Each subplot corresponds to one benchmark; curves show the fraction of examples that trigger r retrieval calls at inference time. The dashed vertical line marks r=3 , the maximum retrieval depth observed in our training trajectories. As B increases, GRIP places more probability mass beyond r>3 , indicating extrapolation beyond the training depth, while the distribution remains concentrated at small r , suggesting selective and task-aware retrieval rather than mechanically consuming the available budget.
High-capacity encoders in retrieval-augmented generation (RAG) can let the query dominate the latent state, leaving retrieved evidence functionally irrelevant. We call this failure mode query dominance. To address it, we introduce GRIP (Grounded Reasoning via Information-Restricted Premises), which imposes capacity asymmetry: the decoder keeps full-dimensional access to the query, while retrieved evidence passes through a severe stochastic bottleneck. This forces the evidence channel to encode only the residual information unavailable from the query. Across five reasoning benchmarks, GRIP outperforms strong iterative baselines, cuts a query--latent mutual-information diagnostic by roughly 30× (14.8 → 0.47 bits), and reduces hallucination by 73%. Residual-alignment analysis further shows that the bottleneck output occupies subspaces less aligned with the query than baseline representations.
Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own internal representations. We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation. By construction, INTRA unifies retrieval and generation, eliminating the retriever-generator mismatch typical of RAG pipelines. This design also amortizes context encoding by reusing precomputed encoder states across queries. On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality. Our results demonstrate that attention-based models already possess a retrieval mechanism that can be elicited, rather than added as an external module.
Elad Hoffer, Yochai Blau, Edan Kinderman +3
NVIDIA · Department of Electrical Engineering, Technion, Haifa, Israel
Retrieval-Augmented Generation (RAG) expands the knowledge of Large Language Models (LLMs), yet current static retrieval methods struggle with complex, multi-hop problems. While recent dynamic retrieval strategies offer improvements, they face two key challenges: 1) irrelevant retrieved noise can misdirect the reasoning process, and 2) processing full documents incurs prohibitive computational and latency costs. To address these issues, we propose OThink-SRR1, a framework that enhances large models with an iterative Search-Refine-Reason process trained via reinforcement learning. Its core Refine stage distills retrieved documents into concise, relevant facts before reasoning. We introduce GRPO-IR, an end-to-end reinforcement learning algorithm that rewards accurate evidence identification while penalizing excessive retrievals, thus training the model to be both focused and efficient. Experiments on four multi-hop QA benchmarks show our approach achieves superior accuracy over strong baselines while using fewer retrieval steps and tokens. This positions OThink-SRR1 as a potent foundational model for information-seeking agents.