AgentIR: Reasoning-Aware Retrieval for Deep Research Agents

Zijian Chen; Xueguang Ma; Shengyao Zhuang; Jimmy Lin; Akari Asai; Victor Zhong

cs.CLMar 4, 2026

AgentIR: Reasoning-Aware Retrieval for Deep Research Agents

Authors: Zijian Chen, Xueguang Ma, Shengyao Zhuang, Jimmy Lin, Akari Asai, Victor Zhong

Organizations: University of Waterloo · University of Queensland · Carnegie Mellon University

Abstract

Deep Research agents are rapidly emerging as primary consumers of modern retrieval systems. Unlike human users who issue and refine queries without documenting their intermediate thought processes, Deep Research agents generate explicit natural language reasoning before each search call, revealing rich intent and contextual information that existing retrievers entirely ignore. To exploit this overlooked signal, we introduce: (1) Reasoning-Aware Retrieval, a retrieval paradigm that jointly embeds the agent's reasoning trace alongside its query; and (2) DR-Synth, a data synthesis method that generates Deep Research retriever training data from standard QA datasets. We demonstrate that both components are independently effective, and their combination yields a trained embedding model, AgentIR-4B, with substantial gains. On the challenging BrowseComp-Plus benchmark, AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch, compared to 50% with conventional embedding models twice its size, and 37% with BM25. Code and data are available at: https://texttron.github.io/AgentIR/.

Figures & tables

Figure 1: Reasoning-Aware Retrieval ( AgentIR-4B ) vs. conventional retrieval (Qwen3-Embedding-4B) for a task from BrowseComp-Plus, paired with the Tongyi-DR agent. The task has been simplified for display.

Figure 2: Oracle reranking procedure used in DR-Synth (Section 3.3 )

LLM	Retriever	Accuracy	Recall	Search Calls
Tongyi-DR	BM25	33.98	46.83	32.92
	Qwen3-Embed-4B	48.67	59.90	31.02
	Qwen3-Embed-8B	50.72	61.78	30.43
	ReasonIR-8B	51.03	63.62	31.15
	Reason-Rewriter + Reason-Embed-8B	31.08	40.15	34.64
	Agentic-R	44.70	47.67	31.05

Table 1: End-to-end evaluation on BrowseComp-Plus. For Tongyi-DR with the visit tool, Search Calls are reported as (search + visit). LLM-Rerank refers to listwise reranking the top-20 results with Qwen3-8B.

Agent	Method	Accuracy	Recall	Search Calls
Tongyi-DR	Qwen3-Embed-4B	48.67	59.90	31.02
	AgentIR-4B (w/o Training)	55.54	66.13	29.21
	AgentIR-4B (w/o Reasoning)	59.40	70.02	27.97
	AgentIR-4B	66.27	78.86	25.91
oss-120b-high	Qwen3-Embed-4B	47.59	58.15	29.14
	AgentIR-4B (w/o Training)	51.33	63.05	27.72

Table 2: Component ablation. All methods use Qwen3-Embed-4B as the backbone. “ AgentIR-4B (w/o Training)” prepends reasoning traces without additional fine-tuning. “ AgentIR-4B (w/o Reasoning)” trains on DR-Synth -generated WebShaper data (Section 3.3 ) but embeds only the query.

Agent	$f(\mathcal{H}_{t})$	Accuracy	Recall	Search Calls
Tongyi-DR	None	59.40	70.02	27.97
	Current Reasoning ( AgentIR-4B )	66.27	78.86	25.91
	Global Question	63.25	70.98	26.90
	Prior Queries	63.13	71.55	27.12
	Prior Queries & Reasonings	63.13	74.20	26.48
	Prior Queries & Reasonings & Docs	60.00	70.73	27.18

Table 3: Ablation over alternative signals. All models are fine-tuned from Qwen3-Embedding-4B using DR-Synth (Section 3.3 ), varying only the trajectory transformation

f(\mathcal{H}_{t})

used as the retriever’s input. “None” embeds only the query, corresponding to the AgentIR-4B (w/o Reasoning) entry in Table 2 .

Figure 3: Effect of embedding $k$ history turns. We fix the Agent to Tongyi-DR. Plot (a) shows the end-to-end accuracy of embedding past $k$ turns, where “None” denotes the “ AgentIR-4B w/o Reasoning” entry in Table 2 . Plot (b) shows the ratio of unique clues covered by using $k$ most recent reasonings, among all clues that have been covered. This is averaged across all trajectories for the $k=\text{all}$ setting (Section 5.3 ).

Figure 4: (a) Reasoning for the query in Figure 1 after identifying the candidate artist, Otto Knows from Sweden. (b) Average number of correct vs. incorrect claims (hypotheses) using $k$ most recent reasonings. This is averaged across trajectories for the $k$ = all setting (Section 5.3 )

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: The prompt template used to embed for AgentIR-4B . At turn $t$ , we fill in {reasoning} with $\tau_{t}$ and {query} with $q_{t}$ . Note that the duplicate “Query:” is intentional due to Qwen3-Embedding-4B Zhang et al. (2025) ’s instruction format.

Panel of Figure 6 — Figure 6: Prompt template used for the oracle reranker (Section 3.3 ).

Panel of Figure 7 — Figure 7: Prompt used for the “Prior Queries” ablation.

Panel of Figure 8 — Figure 8: Prompt used for the “Prior Queries & Reasonings” ablation.

Panel of Figure 9 — Figure 9: Prompt used for the “Prior Queries & Reasonings & Docs” ablation.

Panel of Figure 13 — Figure 13: Prompt used to extract the ground truth multi-hop answer list (Step 1).

Panel of Figure 14 — Figure 14: Prompt used to extract the number of correct vs. incorrect claims (Step 2).

Explore similar work

DeepResearch Agent System
Jul 30, 2026Yong Huang, Yulu Huang, for the team CollaborationDeep Research Agents LLM Inference Acceleration
QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks
May 22, 2026Jian Xie, Tianhe Lin, Zilu Wang +16Deep Research Agents LLM Agent Training
1The Ohio State University · 2Amazon AGI SF Lab
Question's Gambit: The First Move Matters in Agentic Deep Search
Sep 13, 2026Radin Hamidi Rad, Amin Bigdeli, Negar Arabzadeh +4Deep Research Agents Agentic Retrieval
University of Toronto · Mila – Quebec AI Institute · University of Waterloo +2

Jul 30, 2026J/K move · Enter open · S save

DeepResearch Agent System

Yong Huang, Yulu Huang, for the team Collaboration

Deep Research Agents LLM Inference Acceleration Autonomous Research Agents

Open paper AskPDF arXiv

The DeepResearch Agent System is a large language model system engineered for deep information retrieval, multi-step reasoning, and autonomous research tasks. Built upon a sparse activation architecture with 30 billion total parameters of which only 3 billion are activated per token, the system achieves state-of-the-art performance on multiple agent search benchmarks while delivering 3.2 times faster inference compared to dense counterparts of equivalent scale. The system supports a 128K-token context window with hierarchical attention mechanisms that yield 18.7% accuracy and 23.4% recall improvements over standard long-context approaches. A dual-mode reasoning engine provides both a ReAct paradigm for basic multi-step problem solving and an IterResearch mode for high-performance iterative research with up to 20 reasoning steps, collectively delivering a 31.2% accuracy improvement over single-pass baselines. Multi-tool coordination integrates retrieval, computation, web search, and file parsing modules to achieve 92.1% tool-use accuracy. A reinforcement learning optimization framework based on the GRPO algorithm provides token-level policy gradients that improve training stability by 35% and accelerate convergence by 42%. An automated data synthesis pipeline with seed-based expansion achieves a 92.5% usability rate. Benchmark results include 87.3% on Humanity's Last Exam, 85.3% on BrowserComp Chinese, and 91.2% on WebWalkerQA. The system is fully open-sourced, including data synthesis, training, and inference code, and supports applications in academic research, business analysis, R&D support, and education.

Abstract

Explore similar work

DeepResearch Agent System

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

Question's Gambit: The First Move Matters in Agentic Deep Search