Learning When to Think: Utility-Guided Adaptive Reasoning for Efficient Text Retrieval

Yingtao Luo; Kangyi Zhao; Shiyi Du; Yubo Li

cs.IROct 15, 2026

Learning When to Think: Utility-Guided Adaptive Reasoning for Efficient Text Retrieval

Authors: Yingtao Luo, Kangyi Zhao, Shiyi Du, Yubo Li

Organizations: Carnegie Mellon University · University of Pittsburgh

Abstract

Reasoning-conditioned embeddings can improve retrieval by progressively refining semantic constraints. However, additional reasoning may introduce over-generalized or semantically noisy abstractions that weaken retrieval specificity. Existing methods largely supervise reasoning through final retrieval outcomes, providing limited guidance over the refinement process itself. We propose AURA, a framework that directly supervises intermediate reasoning states according to how well they separate relevant documents from hard distractors. AURA further trains a lightweight utility estimator over prefix-conditioned embeddings to adapt reasoning depth at inference time. On the reasoning-intensive BRIGHT benchmark, AURA-4B achieves 34.7 average nDCG@10 while using 2.8 reasoning steps and 96 generated tokens per query. Under matched deployment conditions, adaptive stopping reduces latency from 4.6 to 0.76 seconds relative to full-trajectory reasoning. Results on MS MARCO further suggest that this specialization need not substantially degrade conventional in-domain retrieval.

Figures & tables

Figure 1: Retrieval specificity across reasoning steps on BRIGHT. Lines and shaded regions show the mean and query-level variability normalized to Step 1. Bars indicate the fraction of queries that have already passed their query-specific optimal step.

Figure 2: Motivation behind AURA. Existing methods supervise only the final retrieval outcome, while AURA identifies the peak-utility reasoning state before over-reasoning weakens retrieval specificity.

Figure 3: Overview of AURA. Training: A warm-up stage first establishes a retrieval-oriented embedding space. Specificity labels are then constructed from the warmed-up model, the utility estimator is trained on frozen embeddings, and utility-aware weighting is enabled for subsequent joint optimization. Inference: The utility estimator evaluates each state online and terminates reasoning according to the predicted utility, returning the selected embedding for retrieval.

Retriever	Original	+ Rewrite
BM25	14.5	12.9
SBERT	14.9	12.6
Qwen3-4B	5.6	5.5
OpenAI-3-large	17.9	15.2

Table 2: Effect of Qwen3-4B external query rewriting on BRIGHT (average nDCG@10).

Figure 4: Inference-time scaling of AURA on BRIGHT. Greedy selects the best continuation at each step using utility estimator; best-of-

k

samples

k

full trajectories and selects the one with highest estimated utility.

Text Embedding Models

Text embedding models map words, sentences, or documents to vectors for semantic search, retrieval, clustering, and downstream prediction.

See all 126 papers

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models
Apr 29, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuRetrieval-Augmented Generation Retrieval-Augmented Reasoning
The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China
Retrieval Grounding Latent Reasoning for Dense Retrieval
Aug 14, 2026Gang Zhou, Xiongxi Yu, Hu Tian +5Dense Retrieval Implicit Reasoning in Language Models
School of Artificial Intelligence, Beijing University of Posts and Telecommunications · Meituan LongCat Interaction Team · School of Management Science and Engineering, Central University of Finance and Economics +1
GEM: A Generative Embedding Model Bridging Reasoning and Retrieval
Aug 13, 2026Zhili Shen, Craig MacdonaldText Embedding Models
University of Glasgow

Apr 29, 2026J/K move · Enter open · S save

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models

Dongxin Guo, Jikun Wu, Siu Ming Yiu

The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China

Retrieval-Augmented Generation Retrieval-Augmented Reasoning Multi-Hop Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large reasoning models such as DeepSeek-R1 and OpenAI o1 generate extended chains of thought spanning thousands of tokens, yet their integration with retrieval-augmented generation (RAG) remains fundamentally misaligned. Current RAG systems optimize for providing context before reasoning begins, while reasoning models require evidence injection during multi-step inference chains. We introduce ReaLM-Retrieve, a reasoning-aware retrieval framework that addresses this mismatch through three key innovations: (1) a step-level uncertainty detector that identifies knowledge gaps at reasoning-step granularity rather than token or sentence level; (2) a retrieval intervention policy that learns when external evidence maximally benefits ongoing reasoning; and (3) an efficiency-optimized integration mechanism that reduces per-retrieval overhead by 3.2x compared to naive integration. Experiments on MuSiQue, HotpotQA, and 2WikiMultiHopQA demonstrate that ReaLM-Retrieve achieves on average 10.1% absolute improvement in answer F1 over standard RAG (range: 9.0-11.8% across the three benchmarks) while reducing retrieval calls by 47% compared to fixed-interval approaches like IRCoT (all improvements significant at p<0.01, paired bootstrap). On the challenging MuSiQue benchmark requiring 2-4 hop reasoning, our method achieves 71.2% F1 with an average of only 1.8 retrieval calls per question. Analysis shows that ReaLM-Retrieve also improves retrieval quality itself, achieving 81.3% Recall@5 with consistently higher precision and MRR than fixed-interval baselines on supporting evidence, establishing new state-of-the-art efficiency-accuracy trade-offs for reasoning-intensive retrieval tasks.

Abstract

Text Embedding Models

Explore similar work

When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models

Retrieval Grounding Latent Reasoning for Dense Retrieval

GEM: A Generative Embedding Model Bridging Reasoning and Retrieval