RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

Deniz Qian; Hung-Ting Chen; Eunsol Choi

cs.CLFeb 20, 2026

RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

Authors: Deniz Qian, Hung-Ting Chen, Eunsol Choi

Organizations: Courant Institute School of Mathematics, Computing, and Data Science, New York University

Abstract

Comprehensively retrieving diverse documents is crucial to address queries that admit a wide range of valid answers. We introduce retrieve-verify-retrieve (RVR), a multi-round retrieval framework designed to maximize answer coverage. Initially, a retriever takes the original query and returns a candidate document set, followed by a verifier that identifies a high-quality subset. For subsequent rounds, the query is augmented with previously verified documents to uncover answers that are not yet covered in previous rounds. RVR is effective even with off-the-shelf retrievers, and fine-tuning retrievers for our inference procedure brings further gains. Our method outperforms baselines, including agentic search approaches, achieving at least 10% relative and 3% absolute gain in complete recall percentage on a multi-answer retrieval dataset (QAMPARI). We also see consistent gains on two out-of-domain datasets (QUEST and WebQuestionsSP) across different base retrievers. Our work presents a promising iterative approach for comprehensive answer recall leveraging a verifier and adapting retrievers to a new inference scenario.

Figures & tables

Figure 1: Overview of our Retrieve-Verify-Retrieve framework. Each query

q

aims to retrieve documents to get multiple answers

(y_{1},y_{2},y_{3})

. The initial retriever takes a query and returns document sets, and the verifier examines each document, identifying two valid answers

y_{1},y_{2}

. The subsequent retriever takes the query and documents containing identified answers as input, targeting to retrieve complementary answer

y_{3}

	Base Retriever Model
	Contriever-MSMARCO		Qwen3-Embedding-0.6B		INF-Retriever-v1-1.5B
	MR@100	R@100	MR@100	R@100	MR@100	R@100
Base $f_{i}$	19.00	54.17	16.70	52.94	26.10	62.34
FT $(D_{i})$	28.60	63.19	26.90	63.48	29.30	65.99
Tongyi (w/ Base $f_{i}$ )	6.60	35.96	16.20	52.22	20.40	57.03
Tongyi (w/ FT $(D_{i})$ )	9.80	42.09	22.00	60.67	21.30	58.13

Table 1: Main Experimental Results on QAMPARI test set (N=1000). We report MRecall@100 and Recall@100. Our proposed methods outperform both the base retrievers (Base) and retrievers fine-tuned in-domain (FT (

D_{i}

)). RVR approaches also achieve much higher performance than agentic baselines. For our results, statistical significance is tested using bootstrap resampling with 10,000 trials at

\alpha=0.05

\dagger

indicates statistically significant improvement over FT

(D_{i})

. * indicates statistically significant improvement over FT

(D_{i})

+ FT

(D_{i})

	Contriever-MSMARCO				Qwen3-Embedding-0.6B				INF-Retriever-v1-1.5B
	# Calls $\downarrow$	Ret. $\downarrow$	Ver. $\downarrow$	Sum $\downarrow$	# Calls $\downarrow$	Ret. $\downarrow$	Ver. $\downarrow$	Sum $\downarrow$	# Calls $\downarrow$	Ret. $\downarrow$	Ver. $\downarrow$	Sum $\downarrow$
Base	1	1.91	-	1.91	1	1.34	-	1.34	1	3.75	-	3.75
Tongyi	14.94	97.72	92.32	190.04	15.85	232.79	83.60	316.39	16.42	264.30	80.40	344.70
SearchR1	2.92	19.43	2.41	21.84	2.96	22.95	2.42	25.38	2.93	39.86	2.54	42.40
RVR (w/ T=2, B=1)	2	3.80	0.07	3.87	2	5.05	0.06	5.11	2	7.55	0.07	7.62
RVR (w/ T=2, B=50)	2	3.80	0.53	4.33	2	5.05	0.52	5.57	2	7.55	0.53	8.08

Table 2: Efficiency comparison. We report the number of retrieval calls (# Calls) per query, as well as the seconds taken for retrieval (Ret.), the seconds taken for verification (Ver.) and total (sum of Ret. and Ver.). RVR methods are significantly more efficient than agentic baselines but still achieve better performance.

	Contr. (0.48)		Qwen3 (1.50)		INF (5.43)
	LLM	Index	LLM	Index	LLM	Index
Base	-	73.98	-	98.63	-	147.96
Tongyi	61.06	74.30	61.06	99.35	61.06	139.01
SearchR1	17.41	73.98	17.41	98.63	17.41	147.95
RVR ( $f_{i}$ == $f_{r}$ )	56.93	73.98	56.93	98.63	56.93	147.96
RVR ( $f_{i}$ != $f_{r}$ )	56.93	147.96	56.93	197.26	56.93	295.92

Table 3: Memory requirement comparison (all in GB), for LLM used

\downarrow

and retriever index

\downarrow

. We report the size of each retriever model next to its name in the first row.

	QUEST						WebQuestionsSP
	Contriever		Qwen3		INF		Contriever		Qwen3		INF
	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100
Base	3.24	23.79	3.13	21.38	4.75	26.60	62.16	77.38	61.00	76.06	62.47	77.39
FT $(D_{i})$	3.13	18.67	2.43	18.38	4.75	26.31	49.45	65.72	46.68	61.50	51.60	67.06
Tongyi	0.93	10.51	2.49	19.52	3.30	21.95	52.33	65.84	57.56	71.80	58.29	72.37
SearchR1	0.81	9.54	3.19	20.83	3.53	23.01	54.91	68.92	61.49	76.67	61.98	76.82

Table 4: Out-of-Domain Generalization. Results on QUEST (N=1727) and WebQuestionsSP (N=1639) test sets. We report MRecall@100 (MR) and Recall@100 (R). We report performances of different base retrieval models, agentic baselines, and RVR methods. For RVR, we use a verifier budget of 100. For our results, statistical significance is tested using bootstrap resampling with 10,000 trials at

\alpha=0.05

\dagger

indicates statistically significant improvement over Base. * indicates statistically significant improvement over Base + Base

Verifier	Precision	Recall	Accuracy
Random Baseline	6.71	6.42	64.85
GPT-5-nano	50.25	54.70	82.29
Qwen3-4B-Inst	43.62	52.57	77.63
Qwen3-30B-Inst	34.06	74.05	68.43

Table 5: Average verifier performance on top-100 retrieved documents on QAMPARI test set. Qwen3-30B-A3B-Instruct-2507 performs the best in terms of recall and is used as our verifier for the main experiments.

Verifier	Contriever	Qwen3	INF
Oracle	33.80	33.60	36.30
LLM (Qwen3-30B)	31.60	31.40	33.70
TopK	26.90	27.50	28.50

Table 6: Comparing the performance of using oracle vs. LLM verifier (Qwen3-30B) on QAMPARI test set. We evaluate MRecall@100 in the RVR setting FT

(D_{i})

+ FT

(D_{r})

, with a verifier budget of 100. Using LLM (Qwen3) verifier comes close to using an oracle verifier (upper bound) and outperforms TopK (baseline).

Figure 2: Multi-turn Generalization Results. This figure illustrates the change in Recall@100 and MRecall@100 across five iterations with a verifier budget of 100. Left panels show results with LLM verifier (Qwen3-30B), while right panels show results with oracle verifier that selects documents containing unique answer strings to be used as input. Performance with the LLM verifier plateaus after the second iteration, whereas the oracle verifier shows continued improvement, indicating substantial headroom for better verification mechanisms.

Figure 3: Varying the verifier budget. We evaluate MR@100 across three new verifier budgets on QAMPARI dataset. RVR is shown in green and blue, while our one-round baseline in red. See Appendix B.1 for Recall@100 results.

Figure 4: The performance (MRecall@100) with varying number of input documents at inference time (Context Budget

M

). We compare models fine-tuned with different maximum document counts (3, 6, and 12 docs) for INF and Qwen3. Different colors denote fine-tuning with different number of documents, and shapes indicate the retrievers used. Other metric results (Recall@100) are provided in Appendix B.2 .

+ FT( $D_{i}$ + $D_{r}$ )	31.70	66.12	26.30	61.24	32.40	68.04
	Contriever		Qwen3		INF
	MR	R	MR	R	MR	R
Base + Base	21.30	58.10	20.20	58.12	28.50	65.29
FT( $D_{i}$ ) + FT( $D_{i}$ )	28.80	63.59	30.30	66.80	31.10	66.76
FT( $D_{i}$ + $D_{r}$ )	31.70	66.12	26.30	61.24	32.40	68.04
Base + FT ( $D_{r}$ )	30.70	67.25	29.20	66.36	34.20	69.29

Table 7: Ablation study on QAMPARI test set. Each row represents a RVR configuration, denoted in the format of

f_{i}

f_{r}

(initial + subsequent retriever). Base : off-the-shelf pretrained retriever; FT(

D_{i}

) : fine-tuned on initial retrieval data; FT(

D_{r}

) : fine-tuned on subsequent retrieval data with document context; FT(

D_{i}

D_{r}

) : fine-tuned on both objectives jointly.

	# New Gold Docs.		# New Uniq. Ans.
	Turn 1	Turn 2	Turn 1	Turn 2
Contriever
FT ( $D_{i}$ ) + FT ( $D_{i}$ )	49.59	17.60	7.12	0.33
FT ( $D_{i}$ ) + FT ( $D_{r}$ )	49.59	22.83	7.12	0.73
Qwen3
FT ( $D_{i}$ ) + FT ( $D_{i}$ )	50.94	22.35	7.17	0.66

Table 8: Contribution analysis of first and second stage retrieval on QAMPARI test set. We report the average number of new gold documents retrieved and the average number of new answers covered per question in each stage, with verifier budget

B

=100.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

	# examples			# answers	# gold docs
Dataset	Train	Dev	Test	Avg.	Avg.
QAMPARI	34,425	6,191	1,000	14.43	16.58
QUEST	—	—	1,727	10.50	—
WebQuestionsSP	—	—	1,639	8.75	—

Appendix

Table 9: Dataset statistics showing the number of questions split across train, development, and test sets, along with the average number of answers and gold documents per question.

Figure 5: This figure shows the results of lowering verifier budget beyond 100 for Recall@100.

Figure 6: This figure shows the results of increasing the number of input documents on Recall@100 for INF and Qwen3 models fine-tuned with different maximum document counts: 3, 6, and 12. We evaluate across 7 different context budgets.

Figure 7: Multi-turn Generalization Results. This figure illustrates the change in Recall@100 and MRecall@100 across five iterations with a verifier budget of 50.

		# New Gold Docs.		# New Uniq. Ans.
		Turn 1	Turn 2	Turn 1	Turn 2
Contr-	FT ( $D_{i}$ ) + FT ( $D_{i}$ )	27.93	10.69	5.68	0.51
iever	FT ( $D_{i}$ ) + FT ( $D_{r}$ )	27.93	13.98	5.68	1.05
Qwen3	FT ( $D_{i}$ ) + FT ( $D_{i}$ )	28.95	13.53	5.84	0.89
Qwen3	FT ( $D_{i}$ ) + FT ( $D_{r}$ )	28.95	13.61	5.84	0.93
INF	FT ( $D_{i}$ ) + FT ( $D_{i}$ )	30.98	13.50	6.13	0.66

Appendix

Table 10: Contribution analysis of first-stage and second-stage retrieval on QAMPARI test set. We report the average number of gold documents retrieved and the average number of new answers covered per question in each stage with verifier budget of 50.

Verifier	Contriever	Qwen3	INF
Oracle	29.60	30.50	33.50
LLM (Qwen3-30B)	29.40	29.80	32.00
TopK	26.90	27.50	28.50

Appendix

Table 11: Comparing the performance of using oracle verifier vs. LLM verifier on QAMPARI test set. We evaluate MRecall@100 in RVR setting FT

(D_{i})

+ FT

(D_{r})

with a verifier budget of 50.

	QUEST						WebQuestionsSP
	Contriever		Qwen3		INF		Contriever		Qwen3		INF
	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100	MR@100	R@100
Base	3.24	23.79	3.13	21.38	4.75	26.60	62.16	77.38	61.00	76.06	62.47	77.39
FT ( $D_{i}$ )	3.13	18.67	2.43	18.38	4.75	26.31	49.45	65.72	46.68	61.50	51.60	67.06
Ours: Retrieve-Verify-Retrieve
Base + Base	2.72	21.98	2.95	20.21	4.11	24.47	62.22	76.94	61.73	76.93 †	63.15	77.90 †

Appendix

Table 12: Results with Verifier Budget of 50. Results on QUEST (N=1727) and WebQuestionsSP (N=1639) test sets. We report MRecall@100 (MR) and Recall@100 (R). We report performances of different base retrieval models and RVR methods. For RVR, we use a verifier budget of 100. For our results, statistical significance is tested using bootstrap resampling with 10,000 trials at

\alpha=0.05

\dagger

indicates statistically significant improvement over Base. * indicates statistically significant improvement over Base + Base

Figure 8:

Figure 9:

Figure 10:

Figure 11:

Figure 12:

Question Answering

Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.

See all 260 papers

Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval
May 7, 2026Zeyu Yang, Qi Ma, Jason Chen +1Agentic Retrieval Information Retrieval
Meta Superintelligence Labs · Rice University · Work done at Meta
PAR $^2$ -RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering
Mar 30, 2026Xingyu Li, Rongguang Wang, Yuying Wang +5Retrieval-Augmented Reasoning Multi-Document QA
Oracle AI
KohakuRAG: A simple RAG framework with hierarchical document indexing
Mar 8, 2026Shih-Ying Yeh, Yueh-Feng Ku, Ko-Wei Huang +1Retrieval-Augmented Generation Structured RAG
National Tsing Hua University · Comfy Org Research · Kohaku-Lab

May 7, 2026J/K move · Enter open · S save

Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval

Zeyu Yang, Qi Ma, Jason Chen, Anshumali Shrivastava

Meta Superintelligence Labs · Rice University · Work done at Meta

Agentic Retrieval Information Retrieval Query Expansion IR Evaluation

Open paper AskPDF arXiv

Retrieval-augmented agents are increasingly the interface to large knowledge bases, yet most treat retrieval as a black box: they issue exploratory queries, inspect snippets, and reformulate until evidence emerges. This resembles how a newcomer searches an unfamiliar database rather than how an expert navigates it with strong priors about terminology and likely evidence, causing extra retrieval rounds, latency, and poor recall. We introduce Superintelligent Retrieval Agent (SIRA), which casts superintelligence in retrieval as compressing multi-round exploratory search into a single corpus-discriminative retrieval action. SIRA does not merely ask which terms are relevant; it asks which terms separate the desired evidence from corpus-level confusers. Offline, an LLM enriches each document with missing search vocabulary; at query time, it predicts evidence vocabulary the query omits; and corpus statistics serve as tool calls that filter terms that are absent, overly common, or unlikely to create retrieval margin. The final step is a single weighted BM25 call combining the query with the validated expansion. Across ten BEIR benchmarks, SIRA achieves the strongest average retrieval performance in our comparison, beating dense retrievers, learned sparse retrievers, and LLM search-agent baselines while using no relevance labels or retriever fine-tuning. On downstream QA, its retrieval-only answer coverage exceeds recent RL-trained agentic QA systems on NQ and HotpotQA. We also introduce BrowseComp-Wikipedia, a hard-search benchmark of 232 BrowseComp-derived queries over a 25,587,229-document Wikipedia index. Even without index-time enrichment, using only grounded Wikipedia categories, SIRA outperforms multi-round Perplexity agents at every budget, reaching 9.70% Recall@1, 15.27% Recall@10, and 36.14% Recall@100.

Abstract

Question Answering

Explore similar work

Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval

PAR2^22-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering

KohakuRAG: A simple RAG framework with hierarchical document indexing

PAR $^2$ -RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering