RAG-FGO: Dynamic Optimization of Query-Rewriting Agents for Generative RAG
Authors: Qigan Sun, Xudong Wang, Chaoning Zhang, Chenghao Li, Jiaquan Zhang, Jiehui Xie, Sungyoung Lee, Jie Zou, Ning Xie, Peng Wang, Yang Yang, Heng Tao Shen
Organizations: School of Computing, Kyung Hee University, Yongin-si, South Korea · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Computer Science and Technology, Tongji University, Shanghai, China
In retrieval-augmented generation (RAG), query rewriting serves as a core interface that aligns user intent with external knowledge bases, and its effectiveness directly determines overall system performance. However, generative retrieval faces three critical challenges in real-world deployment: noisy retrieval feedback, the difficulty of jointly optimizing discrete prompt templates and continuous decoding-control parameters, and the limited generalization of static rewriting strategies under semantically complex scenarios. To address these issues, we propose RAG-FGO, a dynamic optimization framework for generative retrieval strategies. RAG-FGO formulates query rewriting as a non-differentiable black-box optimization problem in a mixed discrete-continuous space, and represents prompt templates and decoding parameters as adaptively optimizable query-rewriting agents. The framework performs global search to enable semantic jumps and structural reorganization that escape local optima, while applying local perturbations for fine-grained alignment and precise refinement. In addition, RAG-FGO introduces an elite-preserving memory pool that accumulates high-quality strategies across iterations, thereby mitigating the impact of stochasticity and noisy feedback on optimization stability. Experimental results on a diverse suite of benchmarks demonstrate the effectiveness and robustness of the proposed method, supporting its applicability to complex semantic retrieval tasks.
Figures & tables
Fig. 1 : Conceptual illustration of the FGO-inspired search dynamics in RAG-FGO. Hyphae grow from low-efficiency (info-sparse) zones toward high-efficiency (info-rich) zones in the retrieval space. (i) Local search enables fine-grained exploitation within promising regions; (ii) global search facilitates large semantic jumps to escape local optima and discover new resource-rich areas.
Fig. 2 : Overview of the main components of RAG-FGO. (a) A query-rewriting agent consists of a prompt template and continuous decoding parameters. (b) Variation optimization evolves agents via local search (synonym-level or minor phrasing edits) and global search (LLM-driven prompt rewriting). (c) Candidate agents are scored, ranked, and filtered to select elites. (d) The elite-preserving memory pool retains elite agents and prunes inferior ones across iterations for subsequent refinement.
Fig. 3 : Training process for optimizing query-rewriting agents driven by RAG-FGO. Query-rewriting agents, consisting of a prompt template and continuous decoding parameters, iteratively rewrite queries for retrieval. Retrieved contexts are ranked by semantic similarity to reference evidence, and the corresponding elite agents are stored in an elite-preserving memory pool. The memory-retained agents are then evolved through local search (synonym-level or minor phrasing edits) and global search (LLM-driven prompt rewriting) for the next iteration.
Model
Method
LongBench
HotpotQA
2Wiki
MuSiQue
NQ
SQuAD
AVG
GPT-4o-mini
Direct
58.1
51.1
48.9
40.1
36.4
51.2
47.6
ReAct
62.7
65.6
52.3
53.6
48.9
55.3
56.4
Self-Act
68.9
73.1
69.2
67.3
62.4
69.5
68.4
Aflow
61.0
73.5
65.7
71.6
67.9
73.7
68.9
RAG-FGO
71.5
81.5
77.3
79.4
73.8
79.2
77.1
GPT-4.1-nano
Direct
56.3
48.5
45.3
38.2
33.7
49.3
45.2
TABLE I : Comparison with retrieval-based baselines across six QA benchmarks using five LLM generators. Metrics are answer-level F1 scores (%). Best results are highlighted in bold.
Method
HotpotQA
StrategyQA
AmbigQA
PopQA
EM
F1
Acc
EM
F1
Acc
EM
F1
Acc
EM
F1
Acc
LLM Rewrite
25.8
34.3
30.6
61.4
61.4
65.4
33.4
44.0
48.6
9.2
23.2
52.8
HyDE
32.8
43.8
38.6
67.2
67.2
71.0
47.2
60.4
64.6
9.0
22.7
52.8
Query2Doc
30.8
41.3
36.8
63.6
63.6
67.6
48.6
60.8
65.2
8.2
19.5
44.2
Crafting
28.8
39.3
35.4
65.2
65.2
69.0
42.8
55.1
60.8
7.6
18.4
44.8
SFT
29.8
39.4
36.2
63.6
63.6
67.6
43.2
56.0
62.2
9.0
22.3
54.0
TABLE II : Comparison of RAG-FGO with query rewriting baselines on QA benchmarks using Qwen2.5-7B-Instruct as the generator LLM. Best results are highlighted in bold.
Method
MATH500
MMLU-pro
GPQA
DROP
HotpotQA
MuSR-op
MuSR-ta
Avg.
(Acc.)
(Acc.)
(Acc.)
(EM/F1)
(EM)
(Acc.)
(Acc.)
Zero-shot CoT
76.4
63.0
43.0
77.6/85.6
31.5
58.1
56.2
58.0
Few-shot CoT
75.2
63.0
41.3
76.8/83.1
34.0
59.4
77.0
61.0
USC
78.6
62.5
42.4
78.8/85.8
36.6
59.8
76.4
62.2
CoT-WP
77.8
64.2
42.4
77.6/83.1
34.0
57.0
79.6
61.8
LEAP
74.5
63.2
43.9
75.8/83.0
34.0
59.8
74.4
60.8
TABLE III : Comparison of RAG-FGO with baseline methods on seven reasoning benchmarks using GPT-4o-mini as the generator LLM. Best results are highlighted in bold.
Method
HotpotQA
MuSiQue
NQ
Avg.
AutoPrompt [ 33 ]
60.2
54.5
61.3
58.6
Genetic Prompt Search [ 63 ]
58.7
63.9
67.9
63.5
RLPrompt [ 35 ]
65.1
59.2
65.4
63.2
RAG-FGO
74.5
66.1
73.1
71.2
TABLE IV : Comparison with gradient-free prompt optimization baselines. All results are reported in F1 (%).
Method
LongBench
HotpotQA
2WikiMultihopQA
PSO [ 66 ]
66.2
72.9
69.1
GA [ 67 ]
65.5
71.3
67.8
SA [ 68 ]
61.0
67.2
63.5
FOA [ 69 ]
68.8
76.4
72.5
TLBO [ 70 ]
63.3
69.0
65.4
HS [ 71 ]
64.1
70.2
66.7
TABLE V : Performance comparison of different heuristic optimizers under identical RAG configurations, using GPT-4o-mini as the generator LLM. Results are reported in F1 (%).
Target Dataset
Target-tuned
HotpotQA → Target
Δ
NQ
73.8
71.1
-2.7
SQuAD
79.2
77.3
-1.9
AmbigQA
63.9
59.8
-4.1
StrategyQA
70.2
64.5
-5.7
TABLE VI : Cross-task generalization from HotpotQA using GPT-4o-mini as the generator LLM. Target-tuned: performance with target-specific optimization; HotpotQA → Target: performance with the agent transferred from HotpotQA; Δ : drop in F1 score. Results are reported in F1 (%)
Fig. 4 : Performance (F1, %) as a function of mutation count ( mc ) on six QA benchmarks using GPT-4o-mini.
Fig. 5 : Total token consumption under different mc settings across six QA datasets. Each subplot corresponds to a dataset. The x-axis denotes the mutation count mc , and the y-axis reports the total tokens (in millions, M).
Fig. 6 : Effect of elite-preserving memory pool on saturation during RAG-FGO optimization process.
Configuration
Hotpot
MuSiQue
2Wiki
NQ
SQuAD
RAG-FGO (full)
81.5
79.4
77.3
73.8
79.2
w/o Memory Pool
75.2 (-6.3)
72.8 (-6.6)
70.9 (-6.4)
66.9 (-6.9)
72.1 (-7.1)
w/o Local Search
76.4 (-5.1)
74.2 (-5.2)
72.1 (-5.2)
69.0 (-4.8)
74.3 (-4.9)
w/o Global Search
78.3 (-3.2)
75.4 (-4.0)
74.0 (-3.3)
71.2 (-2.6)
76.1 (-3.1)
TABLE VII : Component ablation of RAG-FGO across datasets (GPT-4o-mini). We report answer-level F1 (%), and the relative change w.r.t. the full model in parentheses.
Param
Value
Avg F1
Cost
δ
0.50
↓ 0.8–1.5%
↓ 2–4%
0.75
↓ 0.1–0.5%
↓ 1–2%
1.00
baseline
baseline
1.25
↑ 0.3–0.9%
↑ 1–3%
1.50
↓ 0.5–1.2%
↑ 1–3%
σl
0.50
↓ 0.6–1.2%
↓ 2–4%
TABLE VIII : Sensitivity analysis of search-related hyperparameters δ,σl,σg,kl,kg on HotpotQA. We report normalized changes in answer-level F1 and optimization cost (token usage) relative to the default configuration ( δ=1.00,σl=1.00,σg=0.50,kl=0.50,kg=0.10 ).