PatternBloom: Empowering Agentic RAG with Externalized RL-Distilled Graph Patterns

Yi Ge; Heng Huang

cs.AINov 7, 2026

PatternBloom: Empowering Agentic RAG with Externalized RL-Distilled Graph Patterns

Authors: Yi Ge, Heng Huang

Organizations: Department of Computer Science, University of Maryland, College Park

Abstract

Reinforcement-learning-trained agents for retrieval-augmented generation retrieve content (passages, knowledge-graph fragments, or hyperedges) and supervise training with outcome rewards on the final answer. Across these paradigms the reasoning route through retrieved content never leaves the language model's hidden state, conflating declarative knowledge with the procedural knowledge that recurs across queries of the same logical type; content-only methods consequently plateau as reasoning depth grows. We introduce PatternBloom, an agentic RAG framework that shifts the unit of retrieval from content (what to read) to logic (how to reason). A first RL stage scores the agent's constructed evidence graph by a frozen oracle's information gain through the Information-Density Reward (IDR), a size-normalized mutual-information surrogate. High-reward trajectories are distilled into a Graph Pattern Memory (GPM) of type-abstracted reasoning skeletons, and a second RL stage uses the Pattern-Augmented Reward (PAR) to couple policy to memory, turning procedural reasoning into a learnable, externalized structural prior. A 7B PatternBloom outperforms $27$ baselines by $+15.0$ Avg-OOD EM, with the gain compounding with reasoning depth precisely because procedural knowledge, once distilled, is reused rather than re-derived at every query. We release our code at https://github.com/EllenYiGe/PatternBloom.

Figures & tables

Figure 1: Content Retrieval paradigm versus Logic Retrieval paradigm . Left : prior paradigms retrieve content units (passages, knowledge-graph fragments, or hyperedges), leaving the reasoning route implicit in policy weights. Right : PatternBloom retrieves a typed reasoning skeleton from an externalized Graph Pattern Memory and traverses it as an explicit logic path.

Figure 2: PatternBloom pipeline. Top: training with intermediate pattern distillation between Stage I (IDR) and Stage II (PAR). Bottom: deployment uses only trained policy and frozen pattern memory.

	Multi-Hop QA								Single-Hop QA						Avg-IID			Avg-OOD
	2Wiki IID		HotpotQA IID		MuSiQue OOD		Bamboogle OOD		NQ OOD		TriviaQA OOD		PopQA OOD
Method	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	CA	EM	F1	CA
Vanilla inference-only Baselines (w/ Qwen2.5-7B-Instruct)
Direct (no retrieval)	21.7	27.7	14.2	22.8	2.8	9.4	4.0	14.4	4.6	13.7	28.0	38.5	9.2	14.4	18.0	25.3	25.8	9.7	18.1	22.2
$\boxplus$ Naive RAG (E5 chunk top-5)	29.7	39.5	24.3	34.9	3.7	10.5	12.0	20.2	10.9	22.7	43.8	55.3	28.0	37.4	27.0	37.2	44.0	19.7	29.2	41.6
Prompt-Engineered RAG

Table 1: Main results on seven QA benchmarks, organized as four Multi-Hop datasets (HotpotQA, 2WikiMHQA, MuSiQue, Bamboogle) and three Single-Hop factoid datasets (NQ, TriviaQA, PopQA). IID marks the two in-distribution training datasets; OOD marks the five held-out datasets. Retrieval representation:

\boxplus

chunk,

\blacktriangledown

knowledge graph,

\square

hypergraph,

\odot

adaptive structure,

\diamond

procedural memory. ‡ marks reported data; remaining rows are our reproduction s.

Variant	IDR	PAR	GPM	HotpotQA	2Wiki	MuSiQue	Bam.	Avg-F 1	Avg-EM	Sch. $\downarrow$
(a) Module on/off
PatternBloom	✓	✓	✓	55.6	58.6	48.0	61.2	55.9	46.6	4.7
$-$ Stage I	✗	✓	✓	54.9	54.4	41.3	60.9	52.9 $\downarrow$ 3.0	42.5 $\downarrow$ 4.1	4.2
$-$ Stage II	✓	✗	✓	41.7	41.7	26.9	66.0	44.1 $\downarrow$ 11.8	34.9 $\downarrow$ 11.7	4.7
$-$ GPM	✓	✓	✗	44.8	44.6	28.2	68.8	46.6 $\downarrow$ 9.3	37.0 $\downarrow$ 9.6	5.9 $\uparrow$ 1.2
(b) Reward shaping weight $\lambda_{p}$ in PAR

Table 2: Ablation studies on

7

B PatternBloom across (a) IDR , PAR , and GPM ; (b) reward shaping weight

\lambda_{p}

in PAR; (c) Graph Pattern Memory pool size. Sch. : avg. # of retrieval calls per query.

Method	RPF $\uparrow$	P-Cov $\uparrow$
PatternBloom	0.886	0.051
$-$ Stage I	0.231	0.025
$-$ Stage II	0.414	0.044

Table 3: Pattern quality.

Panel of Figure 3 — Figure 3: Performance and reasoning quality across three views: (a) cost-quality trade-off, (b) reasoning-depth scaling on MuSiQue, (c) human evaluation.

Method	Bridge	Comp.	1-hop	Multi-3 $+$
$\square$ Graph-R1	5.4	5.3	4.9	5.8
$\square$ $\diamond$ PatternBloom	4.7	4.9	4.2	4.9

Table 4: Average #Turns per question category. Full EM/F1 breakdown in Appendix G.11 .

Figure 4: Case study on a 4-hop query. Structure-based retrieval fabricates both directors’ names, and chunk-based retrieval confuses the film Antonieta with the writer Antonieta Rivas Mercado, while PatternBloom’s typed skeleton fixes what to look up for each film.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Detection rule	Example
EMPTY	empty or whitespace-only string	""
YEAR	^\d{4}$	1989
DATE	^\d{1,2}[/-]\d{1,2}[/-]\d{2,4}$	18-March-1965
NUMBER	^-?\d+(.\d+)?$ , not matching YEAR / DATE	42.7
BOOLEAN	case-folded match in {true, false, yes, no}	Yes
ENTITY	all other non-empty strings	Leslie H. Martinson

Appendix

Table 5: Type-abstraction rules. Detection is rule-based and parameter-free, applied in priority order so each value is assigned a single type.

Metric (abbrev.)	What it measures	Better	Reported in
EM, F1, CA	Answer correctness (exact / token-overlap / lenient match)	$\uparrow$	Tab. 1 , 14
R@5, EvR	Top- $5$ recall, evidence recall on gold supporting facts	$\uparrow$	Tab. 15 , 8
RPF, P-Cov	Pattern-trajectory faithfulness (PB-novel)	$\uparrow$	Tab. 3
Index Tok	One-time corpus-indexing LLM tokens (M)	$\downarrow$	Tab. 8
#Turns	Average number of retrieval calls per query	$\downarrow$	Tab. 4 , 14 , 13
6-axis LLM judge $+$ $\kappa$	Sonnet/Haiku/Gemini ensemble Likert ( $1$ – $5$ ) $+$ agreement	$\uparrow$	Tab. 17

Appendix

Table 6: Reading guide for the metrics used throughout § 4 . Higher is better unless marked

\downarrow

. PB-novel metrics are introduced in this work; all others follow established conventions.

Offline (one-time, cached)	Online (per-query)
Chunking $\mathcal{C}\to\{c_{i}\}$	Query embedding $q\to\mathbf{e}_{q}$ (BGE-large)
LLM entity/relation extraction ( gpt-4o-mini )	FAISS top- $k$ hyperedge retrieval
Hyperedge construction $E_{\text{hyper}}$	Signature extraction $\sigma(q)$
BGE-large embedding of $E_{\text{hyper}}$	FAISS top- $3$ pattern lookup in $\mathcal{M}_{0}$
FAISS index over $E_{\text{hyper}}$	Multi-turn agent rollout $\pi_{\theta}(\cdot\mid q)$
Stage I RL training (Oracle in the loop)	—

Appendix

Table 7: Pipeline phases of PatternBloom: which artifacts are produced once offline, and which calls are made online (per-query).

Knowledge-graph construction (LLM-extracted)
Method	Retrieval Paradigm	Index Tok (M) $\downarrow$	R@5 $\uparrow$	EvR $\uparrow$
GFM-RAG	KG triple + GNN reasoning	4.0	81.3	68.0
LightRAG	KG dual-level (low+high)	86.7	–	–
Adaptive structure (per-query)
StructRAG	adaptive structure	$8.3$	67.9	40.0
PatternBloom (ours)

Appendix

Table 8: Indexing cost vs retrieval quality across paradigms on multi-hop QA. Index Tok : total LLM tokens (M) for one-time graph construction over 2WikiMHQA; StructRAG performs all LLM work at query time, so its entry sums tokens over

1{,}000

test queries. R@5 , EvR : top-

5

retrieval recall and total evidence recall, averaged over the four multi-hop benchmarks. best , second .

Method	LC	Comp.	Faith.	Reuse	Avg	$\boldsymbol{\alpha}$
$\boxplus$ R1-Searcher	3.95	3.80	3.90	3.75	3.85	0.66
$\square$ Graph-R1	3.85	3.70	3.95	3.20	3.68	0.63
$\odot$ Structure-R1	4.10	3.95	4.05	3.85	3.99	0.67
$\square$ $\diamond$ PatternBloom	4.65	4.55	4.50	4.70	4.60 $\uparrow$ 0.85	0.74

Appendix

Table 9: Human study of reasoning-trajectory quality. Three annotators rate

30

random queries on a

1

–

5

Likert scale across four axes: LC Logical Coherence, Comp. Completeness, Faith. Faithfulness, Reuse Pattern Reusability.

\alpha

is Krippendorff inter-annotator agreement. All methods use Qwen2.5-7B-Instruct. best , second .

Sample from 2WikiMultiHopQA

Question: Which film has the director who was born later, The Silver Treasure or Taxi To Paradise ? Ground Truth: Taxi To Paradise

PatternBloom (Q2.5-7B)

Retrieved pattern from

\mathcal{M}_{0}

via

\sigma(q)=

compare(by[birth_year])[FILM] :

\quad(\textsc{film}_{1})\xrightarrow{\texttt{directed\_by}}(\textsc{person}_{1})\xrightarrow{\texttt{born\_in\_year}}(\textsc{year}_{1})

\quad(\textsc{film}_{2})\xrightarrow{\texttt{directed\_by}}(\textsc{person}_{2})\xrightarrow{\texttt{born\_in\_year}}(\textsc{year}_{2})

\quad\text{result}=\arg\max_{i}\textsc{year}_{i}

Turn 1. <query>director of The Silver Treasure</query>

\to

Rowland V. Lee ; <graph>The Silver Treasure | directed_by | Rowland V. Lee</graph> Turn 2. <query>director of Taxi To Paradise</query>

\to

Adrian Brunel ; <graph>Taxi To Paradise | directed_by | Adrian Brunel</graph> Turn 3. <query>Rowland V. Lee born in year</query>

\to

1891 ; <graph>Rowland V. Lee | born_in_year | 1891</graph> Turn 4. <query>Adrian Brunel born in year</query>

\to

1892 ; <graph>Adrian Brunel | born_in_year | 1892</graph> Turn 5. <think>Brunel (1892) is later than Lee (1891), so Taxi To Paradise.</think> <answer>Taxi To Paradise</answer>

Appendix

Table 10: Case illustration on a 2WikiMultiHopQA comparison query. PatternBloom retrieves the typed reasoning skeleton compare(by[birth_year])[FILM] from

\mathcal{M}_{0}

at the start of the rollout, then traverses two parallel director-birthyear paths to a correct comparison. Graph-R1, lacking a procedural prior, conflates the two films and emits the wrong answer.

Sample from MuSiQue

Question: What country was the director of Black Gold (1962) from? Ground Truth: United States

PatternBloom (Q2.5-7B)

Retrieved pattern from

\mathcal{M}_{0}

via

\sigma(q)=

lookup_chain(person)[COUNTRY] :

\quad(\textsc{film})\xrightarrow{\texttt{directed\_by}}(\textsc{person})\xrightarrow{\texttt{nationality}}(\textsc{country})

Turn 1. <query>director of Black Gold 1962</query>

\to

Leslie H. Martinson ; <graph>Black Gold (1962) | directed_by | Leslie H. Martinson</graph> Turn 2. <query>Leslie H. Martinson nationality</query>

\to

American ; <graph>Leslie H. Martinson | nationality | United States</graph> Turn 3. <think>The director is American, so the country is the United States.</think> <answer>United States</answer>

R1-Searcher (Q2.5-7B)

Turn 1. <query>Black Gold 1962 country</query>

\to

retrieves chunks about a 2011 film of the same name. Turn 2. <query>Black Gold 1962 director origin</query>

\to

retrieves a Texas-oil-history chunk that does not mention the director. Turn 3. <think>The film is set in Texas, so probably the United States.</think> <answer>United States</answer> ( correct, but reached via a fragile keyword path; R1-Searcher fails when the same chain query is rephrased with a less common film ) Counter-example: For “What country was the director of Vesna na Zarechnoy Ulitse from?” , R1-Searcher’s keyword path returns a Soviet-cinema retrospective chunk that lists the city but not the director’s nationality, and the model confidently emits the wrong country. PatternBloom traverses the same chain skeleton and returns the correct Russia .

Appendix

Table 11: Case illustration on a MuSiQue 3-hop chain query. PatternBloom retrieves the chain skeleton lookup_chain(person)[COUNTRY] and traverses it in a fixed order, while R1-Searcher loses track of the intermediate entity after Turn 2 and cannot recover.

Sample from Bamboogle

Question: Who was the commander for the space mission that had the first spacewalk? Ground Truth: Pavel Belyayev

PatternBloom (Q2.5-7B)

Retrieved pattern from

\mathcal{M}_{0}

via

\sigma(q)=

compare(by[date])[ENTITY] (top-1 similarity

1.00

, distilled from a HotpotQA training trajectory and re-used here on Bamboogle):

\quad(\textsc{mission}_{1})\xrightarrow{\texttt{achieved}}(\textsc{event})\xrightarrow{\texttt{date}}(\textsc{year}_{1})

\quad(\textsc{mission}_{2})\xrightarrow{\texttt{achieved}}(\textsc{event})\xrightarrow{\texttt{date}}(\textsc{year}_{2})

\quad\text{select }i^{*}=\arg\min_{i}\textsc{year}_{i}\;\Rightarrow\;(\textsc{mission}_{i^{*}})\xrightarrow{\texttt{commander}}(\textsc{person})

Turn 1. <query>space mission with the first spacewalk</query>

\to

retrieves several mission summaries, including Voskhod 2 , Apollo 17 , and Gemini 4 . Turn 2. <think>The first spacewalk was during the Voskhod 2 mission on 8 March 1965.</think> <query>Voskhod 2 mission</query>

\to

launched with Belyayev and Leonov on board on 18 March 1965 . Turn 3. <query>commander of the Voskhod 2 mission</query>

\to

Pavel Belyayev was the first commander of the cosmonaut corps . Turn 4. <think>The knowledge clearly states that Pavel Belyayev was the commander of the Voskhod 2 mission.</think> <graph>Pavel Belyayev | commander_of | Voskhod 2</graph> <answer>Pavel Belyayev</answer> (

\textsc{em}=1.0

)

Appendix

Table 12: Cross-distribution pattern reuse. The compare(by[date])[ENTITY] pattern, distilled from HotpotQA-7K + 2WikiMHQA-7K training, drives a four-turn rollout to the correct answer on an out-of-distribution Bamboogle query.

# gold facts	# queries	Search-R1	R1-Searcher	Graph-R1	Search-R1 (Q3)	PatternBloom	PatternBloom turns
2 hops	425	21.0	20.3	6.6	17.1	52.6 $\uparrow$ 32.3	2.4
3 hops	312	6.4	17.0	4.4	13.1	34.0 $\uparrow$ 17.0	3.1
4 hops	178	2.8	8.6	1.8	6.2	24.5 $\uparrow$ 15.9	4.2
5 $+$ hops	85	1.1	3.9	1.5	2.8	17.4 $\uparrow$ 13.5	5.5
MuSiQue (overall, $1{,}000$ )	1,000	11.5	15.8	4.6	12.7	38.8 $\uparrow$ 23.0	3.2

Appendix

Table 13: Performance vs reasoning depth on MuSiQue. Queries bucketed by number of gold supporting facts (proxy for hop count). Per-method power-law fit

\mathrm{EM}=a\,h^{\alpha}

: PatternBloom attains

\alpha=-1.10

(Search-R1

\alpha=-2.91

2.6\times

steeper).

\Delta

shows per-bucket EM gain over R1-Searcher.

Figure 5: Cross-paradigm performance on HotpotQA (2-hop) vs Bamboogle (4-hop). PatternBloom attains the highest absolute Bamboogle EM (

52.0

) and the largest HotpotQA-to-Bamboogle gain (

\Delta=+10.4

); structure-aware Structure-R1 alone collapses on the harder benchmark (

\Delta=-13.2

). Bars: lighter = HotpotQA, darker = Bamboogle within each paradigm group.

	Bridge		Comparison		Single-hop		Multi-3 $+$		Cat-Avg		DS-Avg		#Turns per category
Method	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	Bri.	Cmp.	Sng.	M3 $+$
$\boxplus$ R1-Searcher	39.5	49.4	44.8	50.2	41.0	49.4	22.9	31.5	37.1	45.1	34.1	42.3	3.9	4.6	4.5	4.6
$\square$ Graph-R1	15.6	22.7	36.5	40.7	24.6	31.6	10.1	17.3	21.7	28.1	21.7	29.2	5.4	5.3	4.9	5.8
$\square$ $\diamond$ PatternBloom	40.6	51.2	61.7	67.8	49.5	59.3	44.8	54.2	49.2	58.1	48.6	58.0	4.7	4.9	4.2	4.9

Appendix

Table 14: Per-category EM/F1 across four logical categories aggregated over the seven datasets: Bridge (2-hop), Comparison , Single-hop (NQ/TriviaQA/PopQA), Multi-3

+

+

-hop chains). Cat-Avg : four-category equal-weight mean. DS-Avg : seven-dataset mean (matching Tab. 1 ). #Turns : average retrieval calls per query.

	HotpotQA		2WikiMHQA		MuSiQue		Bamboogle		Avg
Method	R@5	EvR	R@5	EvR	R@5	EvR	R@5	EvR	R@5	EvR
$\blacktriangledown$ HippoRAG 2	80.5	67.2	78.4	65.8	73.2	60.5	80.0	67.0	78.0	65.1
$\blacktriangledown$ GFM-RAG	83.6	70.5	82.0	68.7	76.4	63.8	83.2	69.2	81.3	68.0
$\blacktriangledown$ LinearRAG	76.8	62.0	74.5	60.3	70.0	56.4	76.0	61.5	74.3	60.0
$\boxplus$ Search-R1	73.2	58.4	71.6	56.7	65.0	51.0	70.5	56.0	70.1	55.5
$\boxplus$ R1-Searcher	78.0	63.5	76.4	61.8	70.2	55.6	75.5	60.5	75.0	60.4

Appendix

Table 15: Retrieval quality on the four multi-hop benchmarks. R@5 : top-

5

recall (queries with at least one gold fact retrieved). EvR : Evidence Recall (fraction of all gold facts covered across the trajectory). A retrieved item counts as a hit if its entity set covers a gold supporting fact.

\boxplus

chunks,

\blacktriangledown

KG,

\square

hypergraph,

\diamond

procedural memory.

Pool size and uniqueness
Quantity	Value
# patterns	27
# unique signatures	27
# unique relations	369
Avg. pattern length (# triples / pattern)	2.48
Distributional entropy

Appendix

Table 16: Descriptive statistics of the learned Graph Pattern Memory built on HotpotQA-7K + 2WikiMHQA-7K.

H_{\mathrm{type}}

H_{\mathrm{sig}}

H_{\mathrm{rel}}

: Shannon entropies (bits) over type, signature, and relation distributions of stored patterns.

Figure 6: Six-dimension quality radar (LC, Faith, Reason, Reuse, Concise, Comp). (a) LLM-judge ensemble (Sonnet 4.6 + Haiku 4.5 + Gemini 2.5 Flash) on

200

queries. (b) Human annotators on

30

queries. PatternBloom ranks first on every axis in both panels. Numerical values: Tab. 17 and Tab. 9 .

Method	LC	Faith	Reason	Reuse	Concise	Comp	Avg	$\boldsymbol{\kappa}$
$\square$ Graph-R1	2.35	1.90	2.05	1.50	2.50	1.95	2.04	0.59
$\boxplus$ R1-Searcher	2.65	2.30	2.20	1.85	2.85	2.35	2.37	0.68
$\odot$ Structure-R1	2.75	2.55	2.50	2.10	2.70	2.55	2.53	0.69
$\square$ $\diamond$ PatternBloom	2.85	2.95	2.75	3.10	2.95	2.85	2.91	0.71

Appendix

Table 17: LLM-judge quality ensemble. Average score (

1

–

5

Likert) from three judges (Sonnet 4.6, Haiku 4.5, Gemini 2.5 Flash) over

200

random queries on six axes: LC Logical Coherence, Faith Faithfulness, Reason Reasoning Clarity, Reuse Pattern Reusability, Concise Conciseness, Comp Completeness.

\kappa

is Krippendorff agreement across judges. All methods use Qwen2.5-7B-Instruct.

Category	Symptom	Addressing extension
R 1 : signature mismatch	$\sigma(q)$ similarity to every stored pattern $<\theta_{\mathrm{sim}}{=}0.5$ ; no pattern hint is injected and the agent reasons from chunks alone.	Inference-time pattern admission (§ I ).
R 2 : entity disambiguation	Retrieved hyperedge surface form matches the query mention but disambiguates to the wrong real-world entity (e.g., two films sharing a title).	Coreference-aware reranking before <query> -block dispatch (generic graph-RAG extension).
R 3 : numerical reasoning	The retrieved skeleton and traversal are correct, but the arithmetic step (year subtraction, attribute ranking) is wrong.	External calculator tool invoked when the retrieved skeleton ends in a numerical comparison (generic LLM extension).
R 4 : predicate canonicalization	The retrieved hyperedge contains the right entities but the predicate string at <graph> does not match the gold annotation.	Predicate canonicalization layer between extraction and graph emission (generic graph-RAG extension).

Appendix

Table 18: Residual error categories on the four multi-hop benchmarks. R 2 –R 4 are generic graph-RAG challenges shared by every baseline; only R 1 is specific to the procedural-memory mechanism and is addressed by an inference-time pattern-admission extension (§ I ).

Component	Stage I (IDR)	Stage II (PAR)
Backbone	Qwen2.5-7B-Instruct [ 33 ]	Stage-I best checkpoint
Optimizer	AdamW	AdamW
Learning rate	$5\times 10^{-7}$	$2\times 10^{-7}$
GRPO group size	$5$	$5$
Batch size	$128$ prompts	$128$ prompts
Mini-batch size	$64$	$64$

Appendix

Table 19: Training hyperparameters for both stages.

Method	Representation	Build-time	Retrieval unit	Reasoning path
GraphRAG [ 11 ]	entity-relation KG	static (LLM)	community summary	implicit
HippoRAG [ 12 , 13 ]	entity-relation KG	static (LLM)	PPR-traversed nodes	implicit
LightRAG [ 15 ]	entity-relation KG	static (LLM)	dual-level summary	implicit
LinearRAG [ 14 ]	hierarchical entity	static (NER)	relation-free chunks	implicit
GFM-RAG [ 16 ]	query-conditioned KG	static (foundation)	GFM-scored entities	implicit
HyperGraphRAG [ 24 ]	hypergraph	static (LLM)	hyperedge	implicit

Appendix

Table 20: Structured-RAG methods along five axes. Static = built once and frozen; Dynamic = constructed per query or evolved at training/inference. PatternBloom is the only method whose retrieval unit is a typed reasoning skeleton with an explicit directed path through the evidence rather than a content fragment.

Method	Reward signal	Retrieval target	Training scale
Search-R1 [ 5 ]	EM on answer	Wikipedia (E5 retriever)	${\sim}170$ k examples
R1-Searcher [ 7 ]	multi-stage curriculum, EM/F 1	search engine	${\sim}8$ k
FrugalRAG [ 10 ]	EM + budget penalty	search engine	$1{,}000$ examples
Graph-R1 [ 6 ]	F 1 + entity-presence proxy	hypergraph index	${\sim}14$ k
Structure-R1 [ 18 ]	EM + format-validity proxy	format-conditional	${\sim}14$ k
PatternBloom (ours)	IDR (info-theoretic) + PAR (pattern coverage)	hypergraph + GPM	${\sim}14$ k (HotpotQA $+$ 2Wiki)

Appendix

Table 21: RL-for-RAG methods compared on reward design and retrieval target. Prior methods use surface-form (token-level) signals on the final answer with at most lightweight proxies on the constructed graph; PatternBloom introduces a process-level reward that interrogates the constructed evidence graph itself (Stage I) and aligns the trajectory with retrieved reasoning patterns (Stage II).

Agentic RAG

Agentic retrieval-augmented generation lets language models plan searches, use tools, and refine evidence before producing grounded answers.

See all 257 papers

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning
Aug 23, 2026Jun Chen, Yongchao Liu, Pengyu Qiu +6Multi-Hop QA Agentic RAG
1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China +1
AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing
Aug 30, 2026Xinke Jiang, Yue Fang, Zhibang Yang +12Retrieval-Augmented Generation Agentic RAG
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +3
AutoPrunedRetriever: Persistent Reasoning Graphs for Efficient and Extensible Retrieval-Augmented Generation
Feb 4, 2026Ning Wang, Kuanyan Zhu, Daniel Yuehwoon Yee +4RAG Context Compression Retrieval-Augmented Generation
Cornell University · University of Cambridge · The University of Hong Kong +2

Aug 23, 2026J/K move · Enter open · S save

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning

Jun Chen, Yongchao Liu, Pengyu Qiu, Jiajun Zheng+5

1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China+1

Multi-Hop QA Agentic RAG RL for RAG RL for IR Retrieval-Augmented Reasoning

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) enables LLMs to access external knowledge for answering knowledge-intensive questions. For complex multi-hop questions, multi-turn retrieval-augmented reasoning extends RAG into an iterative process that repeatedly searches for and integrates evidence across documents. However, existing reinforcement-learning (RL) approaches for agentic RAG are typically optimized with final-answer rewards, which provide sparse supervision and overlook whether the model actually retrieves the required evidence chain. We present GTA-RAG, a graph-trajectory-augmented RL framework for multi-turn retrieval-augmented reasoning. From an entity--document graph, we sample connected document paths, synthesize multi-hop QA trajectories, and validate them with the deployed retriever to obtain executable trajectory-level supervision. We then optimize the retrieval policy with Group Relative Policy Optimization (GRPO) and a trajectory-guided reward that encourages both accurate answers and acquisition of target evidence documents, followed by answer-reward training on natural QA instances. Experiments on three multi-hop and two simple QA benchmarks show that \method{} consistently outperforms RL-based RAG baselines with both Qwen2.5-3B and Qwen2.5-7B backbones, while substantially improving evidence-chain coverage. Our code is available at https://github.com/cjcj46262/GTA-RAG.

Abstract

Agentic RAG

Explore similar work

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning

AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing

AutoPrunedRetriever: Persistent Reasoning Graphs for Efficient and Extensible Retrieval-Augmented Generation