Compute Allocation for Reasoning-Intensive Retrieval Agents

Sreeja Apparaju; Nilesh Gupta

cs.IRMar 15, 2026

Compute Allocation for Reasoning-Intensive Retrieval Agents

Authors: Sreeja Apparaju, Nilesh Gupta

Abstract

As agents operate over long horizons, their memory stores grow continuously, making retrieval critical to accessing relevant information. Many agent queries require reasoning-intensive retrieval, where the connection between query and relevant documents is implicit and requires inference to bridge. LLM-augmented pipelines address this through query expansion and candidate re-ranking, but introduce significant inference costs. We study computation allocation in reasoning-intensive retrieval pipelines using the BRIGHT benchmark and Gemini 2.5 model family. We vary model capacity, inference-time thinking, and re-ranking depth across query expansion and re-ranking stages. We find that re-ranking benefits substantially from stronger models (+7.5 NDCG@10) and deeper candidate pools (+21% from $k$ =10 to 100), while query expansion shows diminishing returns beyond lightweight models (+1.1 NDCG@10 from weak to strong). Inference-time thinking provides minimal improvement at either stage. These results suggest that compute should be concentrated on re-ranking rather than distributed uniformly across pipeline stages.

Figures & tables

Figure 1: Example of simplified LLM-augmented retrieval pipeline for reasoning-intensive queries

Model Variant	NDCG@10	Recall@10	Cost ($/query)
No Enhancement (BM25 only)	14.52	33.76	0.000
Flash-Lite	28.87	57.19	0.0018
Flash (No-Think)	29.63	58.56	0.0093
Flash (Think)	30.23	57.73	0.0141
Pro	30.01	58.01	0.0489

Table 1: BM25 retrieval with different QE Model Variants

Configuration	NDCG@10	Recall@10	Cost ($/query)	Latency (s/query)
QE (Flash) + RR (Flash-Lite)	36.54	38.76	0.021	20.25
QE (Flash) + RR (Flash-no-think)	39.36	41.11	0.045	45.31
QE (Flash) + RR (Flash-Think)	39.92	42.03	0.054	63.28
QE (Flash) + RR (Pro)	41.19	43.57	0.17	79.25

Table 2: Impact of Re-ranker Model Strength (

k=

100;

QE=

Flash No-Think)

Figure 2: Impact of re-ranking depth across query expansion strategies. Increasing the candidate pool from $k=10$ to $k=100$ consistently improves NDCG@10 by approximately 20% across all query expansion models, demonstrating that episodic retrieval depth is a high-leverage compute allocation mechanism.

Inference-Time Compute Allocation

Inference-time compute allocation decides where, when, and how much computation models spend during prediction to improve quality under resource limits.

See all 82 papers

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models
Apr 29, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuRetrieval-Augmented Generation Retrieval-Augmented Reasoning
The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China
Natural Language Query to Configuration for Retrieval Agents
May 26, 2026Melissa Z. Pan, Negar Arabzadeh, Mathew Jacob +3Retrieval-Augmented Generation Cost-Aware Inference
UC Berkeley · University of Washington · Microsoft Azure Research - Systems
ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition
Jan 7, 2026Muyang Zhao, Chunhao Tian, Qi Qi +1Cost-Aware Inference Metacognition in Language Models
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

Apr 29, 2026J/K move · Enter open · S save

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models

Dongxin Guo, Jikun Wu, Siu Ming Yiu

The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China

Retrieval-Augmented Generation Retrieval-Augmented Reasoning Multi-Hop Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large reasoning models such as DeepSeek-R1 and OpenAI o1 generate extended chains of thought spanning thousands of tokens, yet their integration with retrieval-augmented generation (RAG) remains fundamentally misaligned. Current RAG systems optimize for providing context before reasoning begins, while reasoning models require evidence injection during multi-step inference chains. We introduce ReaLM-Retrieve, a reasoning-aware retrieval framework that addresses this mismatch through three key innovations: (1) a step-level uncertainty detector that identifies knowledge gaps at reasoning-step granularity rather than token or sentence level; (2) a retrieval intervention policy that learns when external evidence maximally benefits ongoing reasoning; and (3) an efficiency-optimized integration mechanism that reduces per-retrieval overhead by 3.2x compared to naive integration. Experiments on MuSiQue, HotpotQA, and 2WikiMultiHopQA demonstrate that ReaLM-Retrieve achieves on average 10.1% absolute improvement in answer F1 over standard RAG (range: 9.0-11.8% across the three benchmarks) while reducing retrieval calls by 47% compared to fixed-interval approaches like IRCoT (all improvements significant at p<0.01, paired bootstrap). On the challenging MuSiQue benchmark requiring 2-4 hop reasoning, our method achieves 71.2% F1 with an average of only 1.8 retrieval calls per question. Analysis shows that ReaLM-Retrieve also improves retrieval quality itself, achieving 81.3% Recall@5 with consistently higher precision and MRR than fixed-interval baselines on supporting evidence, establishing new state-of-the-art efficiency-accuracy trade-offs for reasoning-intensive retrieval tasks.

Abstract

Inference-Time Compute Allocation

Explore similar work

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models

Natural Language Query to Configuration for Retrieval Agents

ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition