DeepSearch agents achieve strong performance on complex information-seeking tasks through iterative search and reasoning. However, they often issue intent-misaligned, under-constrained, or redundant search queries, resulting in poor evidence coverage and accumulating retrieval noise. To mitigate these limitations, we propose QREF, a plug-and-play Query Refiner and Evidence Filter. Specifically, the Query Refiner rewrites search queries according to the current reasoning intent while ensuring complementary coverage across each search batch. The Evidence Filter retains evidence relevant to both the current search batch and the reasoning objective, reducing distracting retrieval results. We initialize the Query Refiner with supervised fine-tuning and optimize it through reinforcement learning based on interactions with a real-world web search environment, reducing ineffective queries. Comprehensive experiments demonstrate that our approach improves the performance of DeepSearch agents across four challenging benchmarks, achieving average relative Pass@1 improvements of 33.55% on WebSailor and 11.04% on TongyiDeepResearch without modifying architectures or retraining their reasoning policies. Compared with the original DeepSearch backbone without QREF, the RL-trained refiner reduces the fraction of query batches that lack discriminative retrieval anchors by 95%. Compared with the SFT-trained refiner, it also lowers the filter-based ineffective query ratio.
Figures & tables
Figure 1: Overview of the QREF integrated into a tool-augmented DeepSearch agent.
Figure 2: Analysis of ineffective queries on WebSailor. (a) Occurrence rates of the three ineffective-query categories before/after query refinement. (b) Cumulative ineffective-query counts for successful and unsuccessful trajectories.
Figure 3: Case study illustrating how the proposed modules refine queries and filter retrieved evidence. (a) Vanilla Search Agent with ineffective intermediate queries. (b) Search Agent with Query Refiner. (c) Search Agent with Query Refiner and Evidence Filter.
Backbone
Method
BrowseComp
BrowseComp-ZH
GAIA
HLE
P@1 ↑
Turns ↓
IQR ↓
P@1 ↑
Turns ↓
IQR ↓
P@1 ↑
Turns ↓
IQR ↓
P@1 ↑
Turns ↓
IQR ↓
Tongyi DeepResearch
Baseline
39.89
46.33
-
39.66
18.10
-
74.36
10.60
-
27.61
11.53
-
+Qwen3-8B
30.65
58.49
59.28
38.41
28.18
34.06
70.69
11.36
51.01
25.53
14.01
55.06
+SFT
44.39
29.57
62.22
48.10
15.59
37.22
74.36
7.77
57.11
28.08
8.81
59.10
+RL(Ours)
45.62
20.46
36.41
49.48
11.50
29.36
77.78
7.51
31.21
27.73
6.07
37.25
+GPT-5.2
46.29
22.23
52.76
51.21
10.15
35.26
79.49
5.97
45.15
27.90
5.67
45.57
Table 1: Main results and search efficiency. P@1 denotes Pass@1. Turns indicates the average number of search turns for WebSailor and search actions for TongyiDeepResearch. IQR denotes the ineffective query ratio (%). Bold and underlined numbers denote the best and second-best results, respectively. ↑ indicates higher is better, and ↓ indicates lower is better.
Method
BrowseComp-ZH
BrowseComp
WebSailor
19.03
10.82
w/o Query Refiner
15.22
13.51
w/o Evidence Filter
22.84
12.88
Full Method
25.26
17.61
Table 2: Ablation study of the proposed modules on WebSailor. w/o Query Refiner retains only the Evidence Filter, whereas w/o Evidence Filter retains only the Query Refiner.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Fine-tuning type
Full-parameter SFT
Maximum length
20,480 tokens
Batch size
2 per device
Gradient accumulation
2 steps
Learning rate
1×10−5
Training epochs
1
Appendix
Table 3: Main hyperparameters used for supervised fine-tuning.
Setting
Value
RL algorithm
PPO with GAE
Training batch size
64
Prompt / response length
2,048 / 2,048 tokens
Actor / critic learning rate
1×10−6 / 1×10−5
PPO mini-batch size
64
Actor / critic micro-batch
1 / 2 per GPU
Appendix
Table 4: Main hyperparameters used for PPO training. TP, PP, and CP denote tensor, pipeline, and context parallelism, respectively.
Method
BC-EN
BC-ZH
HLE
GAIA
Avg@3
18.07
23.99
15.06
48.43
Pass@3
30.15
41.18
25.81
65.51
Appendix
Table 5: Additional WebSailor results with the Query Refinement Module, reported using Avg@3 and Pass@3 .
Trajectory Group
Incorrect
Correct
Total
No failure batches
30
11
41
At least one failure batch
243
16
259
Total
273
27
300
Appendix
Table 6: Contingency table for the χ2 analysis of trajectory correctness versus the presence of ineffective query batches.
Figure 4: Pairwise Pearson correlation matrix for the three ineffective-query categories. The heatmap summarizes the extent to which different query-failure modes co-occur across trajectories.