How Specific Do I Need to Be? Effects of User Query Underspecification on Deep Research Agents

Yi Wen; Xiangjue Dong; Bingxuan Li; Hangxiao Zhu; Leo Zhicheng Liu; Yu Zhang; James Caverlee

cs.CLOct 29, 2026

How Specific Do I Need to Be? Effects of User Query Underspecification on Deep Research Agents

Authors: Yi Wen, Xiangjue Dong, Bingxuan Li, Hangxiao Zhu, Leo Zhicheng Liu, Yu Zhang, James Caverlee

Organizations: University of Maryland, College Park · Texas A&M University · University of Illinois, Urbana-Champaign

Abstract

Deep research agents (DRAs) conduct long-horizon reasoning, search, and literature synthesis to provide long-form answers to complex, domain-specific questions. These agents are typically trained and evaluated on fully specified user queries. However, real-world user queries can often be underspecified. We introduce UNDERSPEC-DR, a framework for systematically evaluating how underspecified queries affect agents' final answer quality and intermediate search behavior. Across three DRAs we evaluated, underspecification substantially reduces how well DRA's answers fulfill users' information needs, even though they remain grounded in cited evidence. Our analysis of intermediate search behavior suggests a possible explanation: DRAs conduct broader searches that draw less on the original user query but rely more on their learned prior during training, which can be misaligned with users' intent. Our findings reveal the vulnerability of current DRAs to underspecified user queries and inform future efforts to build DRAs that proactively recognize when additional user input is needed to reliably complete deep research tasks.

Figures & tables

Figure 1: Overview of UnderSpec-DR . (a) From 176 initial–final query pairs in researchers’ conversations with DRAs, we identify three omitted information types: scope ( $q_{s}$ ), terminology ( $q_{t}$ ), and preference ( $q_{p}$ ). We then rewrite benchmark queries into matched variants with an LLM followed by human verification. (b) Each agent answers the complete query $q$ or a variant $q_{r}$ by alternating search queries $s_{k}$ and retrieved documents $D_{k}$ . Highlighted search-query tokens come from the user query (blue), earlier search queries (yellow), retrieved documents (green), or parametric knowledge (red). (c) We analyze answer quality (RQ1), whether agents recognize underspecification (RQ2), and search behavior (RQ3).

Rubric	Removed / Obscured Information	Example	# Query Variants
			RQA	SQA	DRB
R1 Scope ( $q_{\textsc{s}}$ )	Time, place, population, domain	Which regions and countries saw widespread acceptance and implementation of home delivery services for antiretroviral therapy (ART) during the COVID-19 pandemic ?	274	129	74
R2 Term ( $q_{\textsc{t}}$ )	Precise technical terms (replaced by generic words)	How do non-aspirin NSAIDs pain relievers and selective COX-2 enzyme inhibitors differ in their associations with aneurysmal subarachnoid hemorrhage risk, and what is known about the heterogeneity across studies for these drug classes?	238	126	45
R3 Preference ( $q_{\textsc{p}}$ )	Stated criteria, comparisons, or requirements on the answer	Which Obsidian plugins can effectively replicate Notion’s multi-view database functionality (including Table, Kanban, Calendar, and List views)? Please provide a detailed comparison of the strengths and weaknesses of these plugins.	27	65	58
Fully-Underspecified ( $q_{\textsc{u}}$ )	R1–R3 jointly	How does the incidence of imminent fractures differ depending on the site of the (index) initial fracture according to pooled meta-analyses ?	499	287	100

Table 1: Underspecification rubrics, examples, and number of constructed variants in ResearchQA (RQA), ScholarQA-Bench (SQA), and DeepResearch Bench (DRB). Colors distinguish scope , terminology , and preference edits; struck text is removed and bold text denotes replacements.

Figure 2: Effects of query underspecification on answer quality across three benchmarks and DRAs. Top: Mean paired score changes (underspecified variant minus complete query). Filled and outlined bars indicate significant and nonsignificant differences, respectively, based on paired Wilcoxon signed-rank tests ( ${}^{*}p<.05$ , ${}^{**}p<.01$ , ${}^{***}p<.001$ ). Bottom: Proportions of matched queries with lower scores (below zero) or higher scores (above zero), with colors indicating the magnitude of change. Segment labels are percentages; score changes within $\pm 0.02$ are omitted.

Model	Metric	Fully underspec.	w/o preference	w/o scope	w/o term
DR Tulu	Document Utilization	$+2.85\%^{***}$	$+0.01\%$	$+1.51\%^{*}$	$+0.07\%$
DR Tulu	Citation Precision	$+0.45\%$	$+1.40\%$	$-2.88\%$	$+2.17\%$
GPT-5.4 w/ Tools	Document Utilization	$-0.44\%$	$+1.98\%$	$+2.11\%^{**}$	$-0.45\%$
GPT-5.4 w/ Tools	Citation Precision	$-2.14\%$	$+4.68\%$	$-0.48\%$	$+0.83\%$

Table 2: Mean paired changes in document utilization in the final answer and citation precision (underspecified variant minus complete query), pooled across datasets within each model. Values are displayed as percentages and represent percentage-point differences. Significance is assessed using unadjusted paired Wilcoxon signed-rank tests. Stars denote

{}^{*}p<0.05

{}^{**}p<0.01

, and

{}^{***}p<0.001

Figure 7: Changes in document novelty and context-driven term adoption under underspecification. Values show mean paired differences with 95% confidence intervals. Positive novelty changes indicate more newly retrieved documents; negative adoption changes indicate less use of retrieved content. Significance markers follow Fig. 6 .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Agent	$q$	$q_{\textsc{p}}$	$q_{\textsc{s}}$	$q_{\textsc{t}}$	$q_{\textsc{u}}$	Rel. change $q_{\textsc{u}}$
ResearchQA	DR Tulu-8B	.791	.694 ns	.615 ***	.565 ***	.294 ***	$-$ 62.8%
	GPT-5.4 with tools	.610	.496 *	.466 ***	.449 ***	.215 ***	$-$ 64.8%
	Tongyi-DR	.840	.763 **	.681 ***	.624 ***	.334 ***	$-$ 60.2%
ScholarQA-Bench	DR Tulu-8B	.477	.504 ns	.431 ns	.438 **	.308 ***	$-$ 35.5%
	GPT-5.4 with tools	.524	.495 ns	.489 ns	.496 ns	.377 ***	$-$ 25.8%
	Tongyi-DR	.438	.414 ns	.357 **	.395 ns	.207 ***	$-$ 52.8%

Appendix

Table 3: Average answer quality for different types of query variants. Significance markers indicate differences from the fully specified condition (

q

). The last column reports the relative change from

q

to the fully underspecified condition (

q_{\textsc{u}}

), calculated as

(q_{\textsc{u}}-q)/q

		ResearchQA			ScholarQA-Bench			DeepResearch Bench
Agent	Variant	Ask	Assume	Unaware	Ask	Assume	Unaware	Ask	Assume	Unaware
DR Tulu-8B	$q_{\textsc{p}}$	0.0	11.1	88.9	0.0	18.4	81.6	0.0	37.9	62.1
	$q_{\textsc{s}}$	0.0	6.6	93.4	0.0	10.1	89.9	0.0	41.7	58.3
	$q_{\textsc{t}}$	0.4	15.5	84.0	0.0	15.1	84.9	0.0	53.3	46.7
	$q_{\textsc{u}}$	0.2	12.6	87.2	1.0	12.9	86.1	1.0	51.0	48.0
GPT-5.4 with tools	$q_{\textsc{p}}$	0.0	8.7	91.3	2.1	12.8	85.1	1.9	36.5	61.5

Appendix

Table 4: Clarification behavior (% of runs): asks for clarification (Ask), recognizes the ambiguity and states an assumption (Assume), or proceeds without recognizing the ambiguity (Unaware).

Figure 8: Paired changes in query drift between the first and final search queries (underspecified minus complete query), with 95% confidence intervals. Positive values indicate greater drift under underspecification. One, two, and three stars denote $p<.05$ , $p<.01$ , and $p<.001$ , respectively; open circles indicate nonsignificant differences.

	$q_{\textsc{p}}$		$q_{\textsc{s}}$		$q_{\textsc{t}}$		$q_{\textsc{u}}$
Agent	$q$	$q_{\textsc{p}}$	$q$	$q_{\textsc{s}}$	$q$	$q_{\textsc{t}}$	$q$	$q_{\textsc{u}}$
DR Tulu-8B	56.39	56.40 ns	55.97	57.48 *	56.43	56.50 ns	56.59	59.44 ***
GPT-5.4 with tools	21.66	23.64 ns	20.82	22.93 **	21.29	20.85 ns	21.38	20.94 ns
Both agents	42.36	43.16 ns	41.12	42.88 ***	42.27	42.13 ns	44.23	45.92 ***

Appendix

Table 5: Mean document utilization (%,

\#_{\mathrm{cited}}/\#_{\mathrm{retrieved}}

) for each underspecified variant and its paired complete query

q

, pooled across ResearchQA, ScholarQA-Bench, and DeepResearch Bench. Both agents pools the paired questions of DR Tulu-8B and GPT-5.4 with tools. Significance markers come from paired Wilcoxon signed-rank tests against

q

	$q_{\textsc{p}}$		$q_{\textsc{s}}$		$q_{\textsc{t}}$		$q_{\textsc{u}}$
Agent	$q$	$q_{\textsc{p}}$	$q$	$q_{\textsc{s}}$	$q$	$q_{\textsc{t}}$	$q$	$q_{\textsc{u}}$
DR Tulu-8B	43.95	45.35 ns	45.43	42.54 ns	50.87	53.04 ns	50.45	50.90 ns
GPT-5.4 with tools	70.52	75.20 ns	70.59	70.11 ns	75.78	76.61 ns	70.93	68.79 ns

Appendix

Table 6: Mean citation precision (%,

\#_{\mathrm{supporting}}/\#_{\mathrm{cited}}

) for each underspecified variant and its paired complete query

q

, pooled across ScholarQA-Bench and DeepResearch Bench. Only questions with a final answer and at least one cited document under both conditions are included. Significance markers come from paired Wilcoxon signed-rank tests against

q

Deep Research Agents

Deep research agents plan long-horizon searches, synthesize cited evidence, and produce reports for complex open-ended questions.

See all 116 papers

Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents
Sep 1, 2026Ruochen Zhou, Zhengyu Chen, Luan Zhang +3Inference-Time Search Deep Research Agents
City University of Hong Kong · Meituan · Independent +1
DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
Jan 13, 2026Ruizhe Li, Mingxuan Du, Benfeng Xu +3Deep Research Agents LLM Agent Evaluation
University of Science and Technology of China · Metastone Technology, Beijing, China
DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute
Apr 26, 2026Nishant Balepur, Malachi Hamada, Varsha Kishore +9Deep Research Agents LLM Agent Evaluation
University of Maryland · New York University · Allen Institute for Artificial Intelligence (Ai2)

Sep 1, 2026J/K move · Enter open · S save

Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents

Ruochen Zhou, Zhengyu Chen, Luan Zhang, Siyang Gao+2

City University of Hong Kong · Meituan · Independent+1

Inference-Time Search Deep Research Agents Agentic Search

Open paper AskPDF arXiv

Deep-research agents answer complex questions by interacting with search and browsing tools, yet they often search along a single evolving trajectory. Our trajectory-level analysis reveals a common failure mode in which the agent may encounter an early search state with several plausible directions, but follow one direction before collecting enough comparative evidence. Once this happens, subsequent tool calls tend to reinforce the same path, increasing the chance of failure when the initial direction is misleading. We further find that successful trajectories reduce this risk through two behaviors: grounding vague exploration in concrete candidates and shifting directions when the current path is weak or incomplete. Based on these findings, we propose HypoSearch, which generates lightweight hypotheses as soft search hints, explores them through bounded independent branches, and compares branch-level evidence before commitment. Across four deep-research benchmarks and three backbone models, HypoSearch consistently outperforms single-trajectory search and standard parallel baselines, improving Qwen3.5-122B from 46.7 to 60.0 on BC-small while using fewer tool calls than five independent trajectories. A pilot supervised fine-tuning study further shows that these behavioral signals can curate compact training trajectories and reduce degradation from unfiltered data.

Abstract

Deep Research Agents

Explore similar work

Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents

DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports

DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute