Beyond Confidence: Test-Time Scaling for Multi-Turn Search Agents via Retrieval Grounding

Hyunho Kook; Junhyuk So; Tianyu Fu; Haizhong Zheng; Beidi Chen

cs.AIAug 25, 2026

Beyond Confidence: Test-Time Scaling for Multi-Turn Search Agents via Retrieval Grounding

Authors: Hyunho Kook, Junhyuk So, Tianyu Fu, Haizhong Zheng, Beidi Chen

Organizations: University of Southern California · Pohang University of Science and Technology (POSTECH) · Carnegie Mellon University

Abstract

Confidence-based voting aggregates parallel LLM rollouts by weighting each with internal signals such as token log probabilities, and has been actively studied for single-turn reasoning. However, modern LLMs increasingly act as multi-turn search agents that retrieve and condition on external documents. In this paper, we show that confidence-based voting transfers poorly to this multi-turn setting, and identify the underlying failure reason as copy inflation: when retrieved documents are appended to an agent's context, tokens copied from those documents receive systematically inflated log probabilities. This flattens confidence scores within each question and weakens the resulting weighted vote. To address this issue, we propose Retrieval-Grounded Voting (RGV), which scores each rollout by the lexical overlap between its final answer and the documents it retrieved. By computing the signal outside the contaminated context, RGV sidesteps both token log probabilities and additional LLM calls. Across four search-agent benchmarks and five LLMs, RGV consistently outperforms confidence-based voting, with gains of up to +5.4% accuracy and +35% on minority-correct questions, where the correct answer appears in only 1-2 of 8 rollouts.

Figures & tables

Figure 1: Overview. (a) Each rollout is one trajectory of the ReAct-style search agent, looping over tool calls (search, page visit) and emitting a predicted answer; $N$ parallel rollouts may differ. (b) The weighted vote sums per-rollout weights over predicted answers; argmax wins. (c) Confidence-based voting weights each rollout by its token logprobs, inflated when the model parrots context. RGV (ours) weights each rollout by the lexical overlap between its answer prose and the retrieved documents—a signal read outside the contaminated context.

Dataset	Model	Single Avg.	Simple Maj.	DeepConf	RGV (Ours)	Oracle
BrowseComp-Plus	Tongyi-DeepResearch	51.6	62.0	65.7	71.1	74.2
	OSS-120B	47.3	53.3	54.7	56.0	66.7
	MiniMax-M2.7	67.8	75.3	75.3	78.7	83.3
	Kimi-K2.5	80.0	80.7	80.7	81.3	88.7
	GLM-5.1	74.9	80.7	80.7	82.0	90.0
GAIA	Tongyi-DeepResearch	70.0	79.6	79.6	80.6	92.2

Table 1: Headline voting accuracies (%) across 4 multi-turn search agent benchmarks

\times

5 LLMs,

N{=}8

rollouts per question.

Panel of Figure 2 — Figure 2: Why RGV works (BrowseComp-Plus Tongyi-DeepResearch). $y$ -axis: C–W mean gap (mean RGV score of correct rollouts minus that of wrong rollouts). ( 2(a) ) Rare-entity tokens carry the signal. ( 2(b) ) The signal survives a within-Q doc shuffle, collapses $3\times$ across questions, and $53\times$ under a random-token null.

Figure 3: Voting accuracy by question difficulty (# correct of

N{=}8

) on BrowseComp-Plus -DeepResearch. Most of the gap concentrates on the

1{-}2/8

bucket.

Figure 4: Accuracy vs. rollout budget

N

on BrowseComp-Plus with Tongyi-DeepResearch. RGV is uniformly above DeepConf.

Reduction	Accuracy (%)
Simple Majority (no RGV)	62.0
$\min_{d}$	69.3
$\mathrm{mean}_{d}$	70.5
$\max_{d}-\min_{d}$	70.5
$\max_{d}$ (headline RGV)	71.1

Table 2: Doc-set reduction ablation (

N{=}8

, BrowseComp-Plus, Tongyi-DeepResearch).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	gpt-oss-120b	MiniMax-M2.7	GLM-5.1	Kimi-K2.5	Tongyi-DeepResearch
rollout temperature	0.85	0.85	0.85	0.85	0.85
top_p	1.0	1.0	1.0	1.0	1.0
presence penalty	0.0	0.0	0.0	0.0	1.1
max output tokens / turn	8,192	8,192	8,192	8,192	8,192
max total output budget	30,000	30,000	30,000	30,000	30,000
max tool calls	200	200	200	200	200

Appendix

Table 3: Per-model sampling parameters. All models use the same system prompt (Appendix M ), confidence solicitation protocol, and Qwen3-32B judge at temperature

0.0

Dataset	Model	Simple Maj.	DeepConf	RGV	Oracle
BrowseComp-Plus	Tongyi-DeepResearch	$62.0{\pm}1.1$	$65.7{\pm}1.0$	$\textbf{71.1}{\pm}1.9$	$74.2{\pm}0.5$
	OSS-120B	$53.3{\pm}4.7$	$54.7{\pm}4.1$	$\textbf{56.0}{\pm}2.8$	$66.7{\pm}4.1$
	MiniMax-M2.7	$75.3{\pm}6.8$	$75.3{\pm}7.4$	$\textbf{78.7}{\pm}8.2$	$83.3{\pm}1.9$
	Kimi-K2.5	$80.7{\pm}5.0$	$80.7{\pm}4.1$	$\textbf{81.3}{\pm}5.7$	$88.7{\pm}0.9$
	GLM-5.1	$80.7{\pm}4.5$	$80.7{\pm}5.2$	$\textbf{82.0}{\pm}4.8$	$90.0{\pm}1.6$
GAIA	Tongyi-DeepResearch	$79.6{\pm}5.8$	$79.6{\pm}5.8$	$\textbf{80.6}{\pm}3.9$	$92.2{\pm}4.2$

Appendix

Table 4: Headline accuracies (%) with fold-mean

\pm

std under a 3-fold split of each benchmark’s question pool.

Dataset	Model	SM	DeepConf	Verb	RGV	DeepConf+RGV	Verb+RGV
BrowseComp-Plus	Tongyi-DeepResearch	62.0	65.7	68.0	71.1	70.7	71.3
	OSS-120B	53.3	54.7	56.7	56.0	56.0	58.0
	MiniMax-M2.7	75.3	75.3	74.0	78.7	78.0	78.0
	Kimi-K2.5	80.7	80.7	80.0	81.3	81.3	82.0
	GLM-5.1	80.7	80.7	82.0	82.0	82.0	82.7
GAIA	Tongyi-DeepResearch	79.6	79.6	73.8	80.6	80.6	79.6

Appendix

Table 5: Voting accuracies (%) including the verbalised-confidence weight ( Verb ).

Figure 5: DeepConf overconfidence across all 20 benchmark

\times

model cells (cf. Figure a). Each panel stratifies questions by oracle difficulty and reports the mean DeepConf score as a percentage of its peak. On every configuration, impossible questions (

0\%

correct, red) receive

82

–

97\%

of the peak score. Rows: datasets; columns: models.

Family	Cell	copy %	logprob gap
GPT-OSS	FRAMES	82.1	$+0.15$
GPT-OSS	BrowseComp-Plus	91.9	$+0.07$
Tongyi	FRAMES	85.2	$+0.69$
Tongyi	BrowseComp-Plus	91.9	$+0.11$
GLM	FRAMES	87.8	$+0.16$
Kimi	FRAMES	87.7	$+0.20$

Appendix

Table 6: Copy-inflation severity by model family. The gap is positive in every cell; copying accounts for

78

–

92\%

of generated tokens across all six families (the

78\%

lower end is the wiki-18 cell of Appendix H ).

Dataset	Model	prose-recall	Jaccard	ROUGE-2	ROUGE-L	BM25	TF-IDF
BrowseComp-Plus	Tongyi-DeepResearch	71.1	71.1	70.2	71.3	71.7	69.0
	OSS-120B	56.0	56.0	56.7	55.3	55.3	52.7
	MiniMax-M2.7	78.7	77.3	74.0	76.0	75.3	75.3
	Kimi-K2.5	81.3	80.0	79.3	79.3	77.3	78.7
	GLM-5.1	82.0	79.3	80.0	80.7	79.3	78.7
GAIA	Tongyi-DeepResearch	80.6	70.9	72.8	71.8	71.8	73.8

Appendix

Table 7: Voting accuracy (%) across lexical-overlap variants on the main cells. All variants use the same max-over-docs reduction; only

w_{i}^{\text{RGV}}

changes. The highest value in each row is bolded.

		Lowest-Group ( $\min$ )			Bottom-10%			Tail
Dataset	Model	1024	2048	4096	1024	2048	4096	1024	2048	4096
BrowseComp-Plus	Tongyi-DeepResearch	65.7	65.4	64.8	65.7	65.5	64.9	61.8	61.9	61.4
	OSS-120B	54.7	54.0	53.3	54.7	54.0	53.3	53.3	52.7	52.0
	MiniMax-M2.7	75.3	74.7	74.0	75.3	74.7	73.3	73.3	72.7	72.0
	Kimi-K2.5	80.7	80.0	79.3	80.7	80.0	79.3	78.0	78.0	77.3
	GLM-5.1	80.7	80.0	79.3	80.7	80.0	79.3	78.7	78.0	77.3

Appendix

Table 8: DeepConf voting accuracy (%) over the 3 reductions

\times

3 window sizes grid (generated tokens). The Lowest-Group

\times

W{=}1024

cell matches the DC headline in Table 1 . The highest value in each row is bolded.

Token subset	$\|\mathrm{copy}\|$	$\|\overline{\mathrm{copy}}\|$	gap
all alnum (baseline)	936 332	84 584	$+0.51$
stopword-removed	558 491	74 714	$+0.54$
IDF-weighted	558 491	74 714	$+0.69$
digits / cap-leading	232 855	21 690	$+0.44$
length $\geq 8$	92 233	25 935	$+0.62$

Appendix

Table 9: Copy-vs-non-copy logprob gap under five token-subset rules. All gaps positive at

p\ll 0.001

(paired bootstrap over rollouts).

Retrieval stack	Model	Single	SM	DeepConf	RGV ${}_{\text{prose}}$	RGV ${}_{\text{Jac}}$	Oracle
BM25 $\leftarrow$ dense (BCP+)	gpt-oss-120b	31.4 (47.3)	38.7	40.7	37.3	44.0	52.7
BM25 $\leftarrow$ dense (BCP+)	Tongyi-DeepResearch	48.8 (58.9)	58.7	59.3	60.7	61.3	79.3
wiki-18 + E5, HotpotQA	gpt-oss-120b	72.7	74.7	75.3	73.3	72.7	80.7
wiki-18 + E5, FRAMES questions	gpt-oss-120b	59.6 (78.7)	58.7	58.0	60.0	59.3	72.0

Appendix

Table 10: Voting accuracy (%) under degraded retrieval. Parenthesised values are the single-rollout accuracy of the corresponding stock cell, i.e. how far the swap moved the agent. We report both members of the overlap family (Appendix E ). Length-weighted voting scores

38.0

56.0

74.7

and

60.7

on the four rows respectively.

Corpus / cell	Model family	copy %	DC	DC ${}_{\text{masked}}$	DC	DC ${}_{\text{masked}}$	RGV
			rollout AUC		within- $Q$ variance share
FRAMES	GPT-OSS	82.1	0.68	0.49	0.35	0.52	0.46
FRAMES	Tongyi	85.2	0.59	0.37	0.49	0.57	0.57
FRAMES	GLM	87.8	0.78	0.75	0.16	0.30	0.31
FRAMES	Kimi	87.7	0.67	0.47	0.46	0.59	0.53
FRAMES	MiniMax	84.8	0.69	0.57	0.51	0.50	0.58

Appendix

Table 11: Masking copied tokens restores variance but not validity. copy % is the median fraction of generated tokens that are copies. rollout AUC separates judge-correct from judge-wrong rollouts; within-

Q

variance share is the quantity weighted voting consumes (§ 3 ). Masking raises the variance share in

8/10

cells (up to

2.4\times

on BrowseComp-Plus / GPT-OSS,

0.18\!\to\!0.42

) while AUC falls in

9/10

Artifact	Type	Licence	Note on use
BrowseComp-Plus ( Chen et al., 2025 )	benchmark	MIT	questions, corpus, gold-evidence qrels
BrowseComp ( Wei et al., 2025 )	benchmark	MIT	via openai/simple-evals
FRAMES ( Krishna et al., 2025 )	benchmark	Apache-2.0	questions and gold answers
GAIA ( Mialon et al., 2023 )	benchmark	gated, none stated	validation split; redistribution not permitted
HLE ( Phan et al., 2025 )	benchmark	MIT	authors ask that it not be re-uploaded
HotpotQA ( Yang et al., 2018 )	benchmark	CC BY-SA 4.0	Appendix H only

Appendix

Table 12: External artifacts and their licences.

Explore similar work

Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG
Sep 29, 2026Pranav Handa, Ariful AzadRetrieval-Augmented Generation LLM Reliability
Texas A&M University
Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search
Jun 15, 2026Sidhaarth Murali, João Coelho, Jingjie Ning +3Agentic Search Test-Time Scaling
Carnegie Mellon University, United States · Instituto Superior Técnico and INESC-ID, University of Lisbon, Portugal · NOVA LINCS, NOVA School of Science and Technology, Portugal
CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback
Jun 1, 2026Bin Chen, Xinye Liao, Yiming Liu +2RL for Language Model Reasoning Credit Assignment in RL

Sep 29, 2026J/K move · Enter open · S save

Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG

Pranav Handa, Ariful Azad

Texas A&M University

Retrieval-Augmented Generation LLM Reliability Multi-Turn Dialogue Evaluation RAG Evaluation RAG Robustness

Open paper AskPDF arXiv

When conversing with large language models (LLMs), users often begin with a simple question and build towards a multi-hop question through follow-up turns. Retrieval-augmented generation (RAG) and its graph-based variant (GraphRAG) have become the dominant approaches for grounding LLM responses in external evidence, yet both are evaluated almost exclusively on single-turn, fully specified queries. We systematically investigate this evaluation mismatch through a large-scale simulation study. Building on prior work on multi-turn LLM evaluation, we transform questions from multi-hop question answering (QA) benchmarks into underspecified conversations and evaluate ten LLM assistants with eight retrieval systems across 1.5 million simulated conversations. Our findings reveal that multi-turn interaction causes widespread performance degradation, incurring relative performance drops of up to 21% and increasing unreliability by 47%, making RAG systems simultaneously less accurate and less reliable. We identify two distinct failure modes behind this degradation. Systems are either lost in translation, where conversational rephrasing distorts the retrieval query, or lost in conversation, where retrieval succeeds but the LLM fails to synthesize evidence distributed across turns.

Abstract

Explore similar work

Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG

Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search

CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback