CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting

Zixi Zhu; Jiayuan Su; Jian Zhang; Yu Lin; Hongwei Wang

cs.AIAug 21, 2026

CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting

Authors: Zixi Zhu, Jiayuan Su, Jian Zhang, Yu Lin, Hongwei Wang

Organizations: ZJU-UIUC Institute, Zhejiang University. · Tencent Inc. · State Key Laboratory of CAD&CG, Zhejiang University.

Abstract

Search Agents face a severe reliability crisis during reinforcement learning (RL) fine-tuning. Heuristic Top-K retrieval often causes critical evidence loss or noise inclusion, while over-confidence induced by progressive RL leads to hallucinated answers and redundant searches. To build highly reliable agents, we introduce Conformal Prediction (CP) and propose Conformalized Agentic Search (CAS). This framework establishes reliability guarantees on both the retrieval and training sides: on the retrieval side, an Adaptive Prediction Set (APS), a specific CP realization, translates statistical coverage into dynamic document truncation to construct prediction sets that are adaptive in size; on the training side, Adaptive Conformal Inference (ACI), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage to quantify answer confidence, which is then used to penalize low-confidence trajectories within the Group Relative Policy Optimization (GRPO) objective, ensuring the model learns only from reliable ones. Experiments across single-hop and multi-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations, establishing a highly reliable and efficient agent paradigm. Our code is available at https://github.com/S1llyBird/CAS.

Figures & tables

Figure 1: The CAS framework. (A) APS dynamically sizes the retrieved document set based on local query difficulty. (B) ACI modulates the GRPO policy loss by penalizing low-confidence trajectories ( $\mathrm{NLL}>q_{1-\alpha_{t}}$ ) to enforce reliable policy optimization.

Methods	General QA			Multi-Hop QA				Average
Methods	NQ †	TriviaQA ⋆	PopQA ⋆	HotpotQA †	2WikiMultiHopQA ⋆	Musique ⋆	Bamboogle ⋆	Average
Direct Inference	0.106	0.288	0.108	0.149	0.244	0.020	0.024	0.134
CoT	0.023	0.032	0.005	0.021	0.021	0.002	0.000	0.015
RAG	0.348	0.544	0.387	0.255	0.226	0.047	0.080	0.270
IRCoT	0.111	0.312	0.200	0.164	0.171	0.067	0.240	0.181
Search-o1	0.238	0.472	0.262	0.221	0.218	0.054	0.320	0.255

Table 1: The main results on seven datasets.

\dagger/\star

represents in-domain/out-of-domain datasets. The best and second best performances are set as bold and underlined, respectively.

Qwen2.5-3B-Instruct
Methods	General QA	Multi-Hop QA	Overall
Search-R1	0.451	0.249	0.336
Ours	0.498	0.328	0.401
Improvement ( $\Delta$ )	+0.047	+0.079	+0.065
Qwen3-8B
Search-R1	0.496	0.329	0.400

Table 2: Performance improvements of our method compared to Search-R1 and Search-R2 on General QA (single-hop) and Multi-Hop QA.

\Delta

denotes the absolute performance gain.

Methods	General QA	Multi-Hop QA	Overall
Ours (Default)	0.498	0.328	0.401
-ACI	0.481	0.311	0.384
-APS	0.490	0.313	0.389
$\rho=0.1$	0.497	0.302	0.386
$\rho=0.4$	0.495	0.273	0.368
$\alpha_{\text{APS}}=0.35$	0.476	0.294	0.372

Table 3: Ablation study on Qwen2.5-3B-Instruct. The table presents the unablated framework, component-wise ablations, and sensitivity analyses for

\rho

and

\alpha_{\text{APS}}

.Full results across all individual datasets are detailed in Table 8 .

Panel of Figure 2 — Figure 2: Training dynamics and sensitivity analyses on the Qwen2.5-3B-Instruct backbone. (a) Evolution of average search calls during training across different ablation configurations. (b) The ratio of low-confidence samples penalized by the ACI weight under varying target error rates ( $\rho$ ). (c) The dynamic adaptation of the target error rate ( $\alpha_{t}$ ), demonstrating stable convergence to the preset $\rho$ values.

$\alpha_{\text{APS}}$	Avg. Retrieved Documents
$0.20$ (Default)	3.4
$0.35$	2.4
$0.05$	4.8

Table 4: Average number of retrieved documents under different APS significance levels (

\alpha_{\text{APS}}

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Prompt 1: Template for CAS
You are Qwen, created by Alibaba Cloud. You are a helpful assistant. Answer the given question. You must conduct reasoning inside <think> and </think> first every time you get new information. After reasoning, if you find you lack some knowledge, you can call a search engine by <search>query</search> and it will return the top searched results between <information> and </information> . You can search as many times as you want. If you find no further external knowledge needed, you can directly provide the answer inside <answer> and </answer> , without detailed illustrations. For example, <answer> Beijing </answer> .

Question: [Question]

Appendix

Table 5: Template for CAS reasoning process.

Prompt 2: Query Decomposition
You decompose QA tasks into hop-level search query-answer pairs. Return strict JSON. Given a QA sample, split it into single-hop query-answer pairs.

Rules:
1) If single-hop, return one pair.
2) If multi-hop, return one pair per hop.
3) answer should be concise and factual.

Appendix

Table 6: Prompt for decomposing multi-hop queries into single-hop sub-queries.

Prompt 3: Retrieval Relevance Judge
You are a precise retrieval relevance judge for QA reasoning. You are given a retrieval query, its target answer, and retrieved documents. Decide which document(s) can support reasoning to the target answer.

Return strict JSON with schema:
{"golden_doc_indices":[0,1], "best_golden_doc_index":0, "reason":"..."}

Rules:

Appendix

Table 7: Judge prompt for locating the most relevant documents.

Panel of Figure 3 — Figure 3: Visualization of training metrics and the impact of the thinking mode. Panels (a) and (b) illustrate the progression of accuracy and tool invocation frequency during training. Panels (c) and (d) demonstrate how the thinking mode influences the final EM scores and the efficiency of the search process.

Methods	General QA			Multi-Hop QA				Average
Methods	NQ †	TriviaQA ⋆	PopQA ⋆	HotpotQA †	2WikiMultiHopQA ⋆	Musique ⋆	Bamboogle ⋆	Average
- ACI	0.417	0.598	0.430	0.378	0.372	0.143	0.352	0.384
- APS	0.435	0.603	0.433	0.393	0.374	0.143	0.344	0.389
$\rho=0.1$	0.441	0.601	0.449	0.382	0.375	0.132	0.320	0.386
$\rho=0.4$	0.438	0.596	0.452	0.356	0.351	0.113	0.272	0.368
$\alpha_{\text{APS}}=0.35$	0.420	0.584	0.432	0.372	0.370	0.146	0.288	0.372

Appendix

Table 8: Detailed ablation study and sensitivity analysis on Qwen2.5-3B-Instruct. The table presents component-wise ablations and sensitivity analyses for

\rho

and

\alpha_{\text{APS}}

Question: What BBC television drama serial starring Jim McManus was based on Sarah Waters’ debut novel of the same name?

Golden Answer: [’Tipping the Velvet’]

<think>

The question asks for a BBC television drama serial based on Sarah Waters’ debut novel of the same name. I recall that Sarah Waters’ debut novel is probably "Tipping the Velvet", but I need to confirm it, especially because Jim McManus may create ambiguity in search results.

</think>

<search> Jim McManus Tipping the Velvet </search>

Appendix

Table 9: Case Study: Complex Single-Hop QA.

Question: Who won the Men’s Singles French Open Tennis Tournament in 2018?

Golden Answer: [’Rafael Nadal’]

<think>

I need to find out who won the Men’s Singles French Open Tennis Tournament in 2018. I remember Rafael Nadal was dominant at the French Open around that time, but I should verify the exact 2018 result.

</think>

<search> 2018 French Open Men’s Singles winner </search>

Appendix

Table 10: Case Study: Simple Single-Hop QA.

Question: Which hockey team does the younger brother of Sean James Pronger currently work for?

Golden Answer: [’the Florida Panthers’]

<think>

I need to identify Sean James Pronger’s younger brother first, then determine which hockey team that person currently works for. This requires at least two steps: family relation lookup, then current team lookup.

</think>

<search> Sean James Pronger younger brother </search>

Appendix

Table 11: Case Study: Multi-Hop QA.

Explore similar work

MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG
Aug 25, 2026Qiuyi Qi, Tian Liang, Jiamu Wang +7Agentic RAG RL for LLM Agents
Zhejiang University · Ant Group · City University of Hong Kong
GRASP: GRanularity-Aware Search Policy for Agentic RAG
Jul 11, 2026Varun Gandhi, Jaewook Lee, Shantanu Todmal +4Multi-Hop QA Retrieval-Augmented Generation
University of Massachusetts Amherst · Adobe Research
Contextual Information Policy Optimization for Search Agents
Aug 6, 2026Xingyu Guo, Wei Chen, Linlin Yang +1Evidence-Grounded Reasoning RL for Language Model Reasoning
National College for Excellent Engineers, Beihang University, Beijing, China · School of Artificial Intelligence, Beihang University, Beijing, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China

Aug 25, 2026J/K move · Enter open · S save

MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG

Qiuyi Qi, Tian Liang, Jiamu Wang, Jinjian Zhang+6

Zhejiang University · Ant Group · City University of Hong Kong

Agentic RAG RL for LLM Agents Knowledge-Intensive QA RL for IR

Open paper AskPDF arXiv

Agentic retrieval-augmented generation (RAG) requires language models to decide when to continue searching and when to answer. Existing RL-based methods rely on external supervision and overlook the agent's internal belief about whether the current evidence is sufficient. To address this problem, we reformulate the search decision quality as belief-action alignment and propose MetaRAG, a belief-action aligned policy optimization framework for agentic RAG. MetaRAG uses Verify-first Action Generation to elicit an explicit verification process before each actual action, and Internal Belief Probing to estimate the policy model's own answerability belief from the same question-history context. Based on these, MetaRAG derives a consistency reward that is further gated by answer correctness, avoiding reinforcement of internally consistent but incorrect trajectories. The belief probe is used only during training and introduces no inference-time overhead. Experiments on seven public QA benchmarks show that MetaRAG consistently improves the accuracy-efficiency trade-off over strong RL-based agentic RAG baselines, with gains that transfer to deep research settings, different optimizers, and multiple model backbones.

Abstract

Explore similar work

MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG

GRASP: GRanularity-Aware Search Policy for Agentic RAG

Contextual Information Policy Optimization for Search Agents