Complex questions in retrieval-augmented generation require evidence spread over several documents, and an answer is reliable only when the retriever finds all of it. Chain embedding retrievers collect such evidence step by step, re-encoding the question together with the passages retrieved so far, and we find that they are already strong: trained only on gold histories, a small chain embedding retriever that calls no language model outperforms graph-based systems and is competitive with a search agent. This training, however, leaves a gap. At test time the retriever conditions on its own retrievals, which soon contain mistakes: by the third step, about half or more of the states it visits hold a wrong passage, and these passages lower its chance of retrieving a missing gold passage well beyond what the difficulty of the states explains. We propose on-policy state training, which trains the encoder on the states it reaches itself. At every epoch the current encoder retrieves chains for the training questions; each state it visits, including those built on its own mistakes, is labeled with the earliest gold passage still missing while the other missing gold passages are masked, so that no correct continuation is penalized; and the encoder is updated contrastively on these states mixed with teacher-forced ones, so that its training distribution follows it as it learns. The resulting retriever retrieves complete evidence more often than teacher-forced training and the systems we compare, and it transfers without further training to harder long-chain variants of HotpotQA and 2WikiMultiHopQA that we construct. On the same states, it recovers from its own mistakes far more often, and controlled comparisons attribute the gain to training on the states the encoder currently reaches.
Figures & tables
Figure 1. Overview. (a) Teacher forcing trains on gold histories, while at test time the retriever’s own mistakes stay in the query and its states drift. (b) On-policy state training rolls in the current encoder, labels every visited state with the earliest missing gold passage, and trains on a mix of gold and visited states, refreshed every epoch.
Method
MuSiQue
HotpotQA-Long
2Wiki-Long
R@5
R@10
C@5
C@10
R@5
R@10
C@5
C@10
R@5
R@10
C@5
C@10
Simple Baselines
BM25
43.4
50.8
9.8
15.5
23.3
25.5
0.0
0.0
27.9
31.5
0.0
0.0
Dense
55.1
62.4
18.9
27.1
29.0
31.9
0.3
1.0
31.3
34.5
0.0
2.0
Structure-Augmented Retrieval
G-Retriever
36.5
43.7
9.2
15.9
16.1 †
20.5 †
0.0 †
2.0 †
18.9 †
22.2 †
0.0 †
1.7 †
Table 1. Retrieval results. Our retriever is trained on MuSiQue and evaluated on the long-chain benchmarks without further training; its results are the mean over five seeds, with the standard deviation in gray. Dense is the frozen Qwen3-Embedding-0.6B encoder with the question as the query; the structure-augmented baselines, Search-R1 and our retriever use the same encoder. Search-R1 calls a 7B language model at every step. The best result in each column is in bold and the second best is underlined. † HippoRAG 2 and G-Retriever on the long-chain benchmarks are run on reduced corpora (all gold passages plus the frozen encoder’s top 100 per question), an easier setting than the full corpora used by the other rows (Appendix D ).
Figure 2. Retrieval by question type, with the number of questions under each column. Colors are scaled within each column, and the best result of each column is framed. Our retriever is the mean over the five seeds of Table 1 ; † reduced corpora.
Passages from
EM
F1
Dense
25.9
36.3
G-Retriever
21.2
29.2
HippoRAG 2
28.2
39.4
LinearRAG
22.3
30.5
Q-RAG
32.4
42.0
Search-R1
35.5
44.8
Table 2. Question answering on MuSiQue with a fixed reader; trained retrievers use the shared seed.
MuSiQue
HotpotQA-L
2Wiki-L
Method
R@5
C@5
R@10
C@10
R@10
C@10
On-policy state training
74.2
50.5
56.4
28.0
67.0
34.7
w/o refresh
71.8
48.4
52.6
22.7
64.6
28.3
w/o on-policy states
69.9
44.6
51.5
22.3
61.3
24.3
w/o chain
54.6
18.9
29.8
1.0
31.4
1.7
Table 3. Ablations; all rows are trained with the same seed.
Figure 3. (a) Share of the teacher-forced retriever’s states whose history contains a wrong passage, at each step of four-hop MuSiQue questions (gold passage still missing). (b) Rate at which each encoder retrieves a missing gold passage next on the same states, by the number of wrong passages in the history. All encoders are trained with the same seed.
Figure 4. Query trajectories of the two retrievers on six MuSiQue questions, each on its own chain. Gold passages sit on the top row in hop order and the non-gold passages retrieved by either chain on the bottom row. The query vector at step t is drawn at the mean of these positions weighted by the softmax of its cosine to each passage at the training temperature τ=0.05 , so it sits next to the passages it points at; filled nodes are steps that retrieve a gold passage.
Other half
Roll-in
New states
R@5
C@5
Gold (teacher forcing)
–
–
69.9
44.6
Synthetic
none
✗
72.1
47.9
Synthetic
none
✓
71.9
47.1
Visited
once
✗
71.8
48.4
Visited
once
✓
71.8
47.0
Visited (ours)
every epoch
✓
74.2
50.5
Table 4. What in the visited states matters on MuSiQue; “new states” marks a half that changes at every epoch. All rows are trained with the same seed and recipe.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Next-step success
Pools
Training
C@5
0
1
2+
Per state
Teacher forcing
44.6
67.0
25.0
14.5
On-policy
50.5
69.0
35.8
29.2
Shared
Teacher forcing
45.0
65.4
26.1
20.8
On-policy
48.8
67.9
32.8
28.7
Appendix
Table 5. Per-state and shared negative pools on MuSiQue, all trained with the same seed.
Figure 5. Mixing ratio between gold and visited states on MuSiQue, all trained with the same seed; 0% is teacher forcing on gold states.
Method
HotpotQA-Long
2Wiki-Long
Dense
31.9 / 1.0
34.5 / 2.0
HippoRAG 2
33.4 / 1.3
35.7 / 2.3
On-policy state training
64.2 / 38.6
69.2 / 36.8
Appendix
Table 6. Results on the reduced corpora (R@10 / C@10); ours is the five-seed mean of Table 1 .
Type
Question (answer)
Gold passages in chain order
HotpotQA-Long, 3 hops
Who voices the title character of the animated television series that features Kuvira as a fictional character? (Janet Varney)
Kuvira → The Legend of Korra → Korra
HotpotQA-Long, 5 hops
Hagen Site sits in a county whose seat was headquarters for a rail division that reached a town on a river; how long is that river? (692 mi)
Hagen Site → Dawson County, Montana → Glendive, Montana → Livingston, Montana → Yellowstone River
2Wiki-Long, 4 hops
Where was the father of the maternal grandmother of Prince Andrew of Greece and Denmark born? (Hildburghausen)
Prince Andrew of Greece and Denmark → Olga Constantinovna of Russia → Princess Alexandra of Saxe-Altenburg → Joseph, Duke of Saxe-Altenburg
2Wiki-Long, comparison
Among The Lambeth Walk, Dikshul, and Ezhu Rathrikal, which entity has the composer with the latest date of death? (Ezhu Rathrikal)
Multi-hop retrieval-augmented generation (RAG) requires an agent to retrieve evidence for a sequence of dependent sub-questions. When a wrong answer lacks a required passage, the passage may be inaccessible to the retriever, or the agent may have failed to issue a query that would retrieve it. Standard retrieval evaluations conflate these two causes. We study how much apparent retrieval failure is instead caused by search drift: the search no longer tracks an unanswered sub-question. Holding the retriever, corpus, and generation model fixed, we introduce ExecutionRAG, a training-free requirement register that records open sub-questions and their supporting evidence. The register changes search guidance but not evidence access, so each gold passage it recovers certifies that the passage was reachable by the original retriever. On 1,000 MuSiQue questions, the number of wrong answers with unretrieved required evidence falls from 382 to 234 with all-MiniLM-L6-v2 and from 383 to 274 with Qwen3-Embedding-0.6B. These counts correspond to 69% to 44% and 66% to 48% of each system's wrong answers, respectively. Crucially, evidence recovery does not produce a proportional reduction in total error: premature resolutions and abstentions increase as missing-evidence failures decrease. The central finding is therefore a shift in the remaining bottleneck, from guiding search toward required evidence to verifying that the retrieved evidence is sufficient.
Jiaqi Wei, Wenxuan Huang, Chengdong Xu +2
Zhejiang University Hangzhou, China · Shanghai AI Laboratory Shanghai, China · Fudan University Shanghai, China +3
Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own internal representations. We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation. By construction, INTRA unifies retrieval and generation, eliminating the retriever-generator mismatch typical of RAG pipelines. This design also amortizes context encoding by reusing precomputed encoder states across queries. On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality. Our results demonstrate that attention-based models already possess a retrieval mechanism that can be elicited, rather than added as an external module.
Elad Hoffer, Yochai Blau, Edan Kinderman +3
NVIDIA · Department of Electrical Engineering, Technion, Haifa, Israel
Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.