SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning

Zhaohan Meng; Zaiqiao Meng; Siwei Liu; Hao Xu; Ke Yuan; Iadh Ounis

cs.CLAug 22, 2026

SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning

Authors: Zhaohan Meng, Zaiqiao Meng, Siwei Liu, Hao Xu, Ke Yuan, Iadh Ounis

Organizations: School of Computing Science, University of Glasgow · Brigham and Women’s Hospital, Harvard Medical School, Harvard University · Language Technology Lab, University of Cambridge · School of Natural and Computing Science, University of Aberdeen ·

Abstract

Biomedical multi-hop question answering (QA) requires models to connect evidence across intermediate entities such as diseases, drugs, proteins, and phenotypes. Existing agents typically rely on static retrieval workflows or coarse-grained prompt rewriting, which can lead to instruction drift when reasoning procedures need to be updated. We propose SSE-Bio, a structured self-evolving agent with an agentic retrieval policy for multi-hop biomedical reasoning. Instead of globally rewriting agent instructions, SSE-Bio maintains a structured state, selectively retrieves knowledge triplets and prior templates through a trainable proxy policy, and improves its reasoning memory through fine-grained template editing. To optimise retrieval decisions, we introduce a proxy-training strategy based on group relative policy optimization, where the proxy is improved through decision-contrastive groups over alternative retrieval choices. Experiments on three biomedical multi-hop QA benchmarks show that SSE-Bio consistently outperforms existing baselines, achieving an improvement of 6.56 absolute points over the strongest self-evolving baseline on BioHopR.

Figures & tables

Figure 1: Left: The Manager maintains a structured state and template memory. Based on the current state, the Proxy decides whether to retrieve knowledge triplets and/or prior templates. The Manager then constructs a plan for the Dev agent, while the Critic returns structured feedback for iterative refinement. Right: Decision-contrastive trajectory generation explores alternative retrieval actions from the same state, and GRPO updates only the Proxy with an answer-grounded reward combining correctness and evidence support.

Family	Method	Single Answer				Multi Answer
		Prec H1	Prec H2	Both cor $\uparrow$	Both wr $\downarrow$	Prec H1	Prec H2	Both cor $\uparrow$	Both wr $\downarrow$
LLM	Llama-3.1-8B	0.13	0.04	0.00	99.83	0.07	0.01	0.00	99.92
LLM	Llama-3.1-70B	26.41	9.45	4.94	69.06	19.82	6.94	3.24	76.51
LLM	GPT-4o	32.86	14.59	7.84	60.43	25.48	10.71	5.25	69.03
Medical LLM	HuatuoGPT-70B	0.15	0.00	0.00	99.84	0.09	0.00	0.00	99.91
Medical LLM	HuatuoGPT-8B	0.21	0.03	0.00	99.76	0.11	0.01	0.00	99.88

Table 1: Performance (%) comparison on BioHopR . Prec H1 and Prec H2 evaluate the linked 1-hop and 2-hop questions, while Both cor and Both wr measure whether both questions in a pair are jointly correct or jointly wrong. † indicates a significant improvement over STELLA (paired t-test,

p<0.05

Figure 2: (1) Zero-shot accuracy (%) comparison with agentic baselines on HLE. (2) Zero-shot accuracy (%) comparison with agentic baselines on MedHop. (3) Performance (%) of different proxy backbones on BioHopR.

Model / Setting	Backbone	Single Answer				Multi Answer
		Prec H1	Prec H2	Both cor $\uparrow$	Both wr $\downarrow$	Prec H1	Prec H2	Both cor $\uparrow$	Both wr $\downarrow$
Effect of Structured State Tracking and Fine-grained Template Evolution
Free-form state + rewriting	Qwen-72B	41.58	16.93	11.72	53.21	32.94	12.58	8.04	62.52
Structured state + rewriting	Qwen-72B	44.12	18.92	13.86	50.82	35.07	14.18	9.66	60.41
Free-form state + editing	Qwen-72B	45.03	19.47	14.52	50.02	35.88	14.86	10.28	59.54
Structured state + editing	Qwen-72B	47.21	21.38	16.52	47.93	38.36	16.24	11.73	57.13

Table 2: Ablation study and boundary analysis on BioHopR .

Relation Type	STELLA				SSE-Bio
	Single Answer		Multi Answer		Single Answer		Multi Answer
	Prec H1	Prec H2	Prec H1	Prec H2	Prec H1	Prec H2	Prec H1	Prec H2
Same Query and Bridge
Disease:Drug:Phenotype	48.27	22.83	38.64	10.58	59.12 †	34.26 †	47.03 †	16.82 †
Disease:Drug:Protein	46.75	10.36	37.42	4.37	52.18 †	12.41 †	41.02 †	5.06 †
Drug:Disease:Phenotype	50.18	20.67	33.15	8.74	58.74 †	27.35 †	37.91 †	11.96 †

Table 3: Relation-type breakdown on BioHopR . † indicates a significant improvement (paired t-test,

p<0.05

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Example of the structured state used as short-term memory for retrieval control and planning.

Panel of Figure 4 — Figure 4: Example of long-term template memory and its fine-grained update after a successful case. The grey row shows the original field implicated by the failure, and the blue row shows the updated field. Other fields remain unchanged.

Memory	Field	Role
State $u_{t}$	Task type	template retrieval query
State $u_{t}$	Information gap	triplet retrieval query
State $u_{t}$	Feedback	Critic guidance for refinement
State $u_{t}$	Retrieval state	retrieved evidence summary
Template $x$	Task type	template retrieval key
Template $x$	Reasoning flow	reusable reasoning steps

Appendix

Table 4: Field schema of the structured state and template memory in SSE-Bio.

System Prompt	Single Answer				Multi Answer
	Prec H1	Prec H2	Both cor	Both wr	Prec H1	Prec H2	Both cor	Both wr
Empty	46.38	20.71	16.08	48.62	37.64	15.61	11.21	58.04
Basic	46.72	20.96	16.27	48.31	37.91	15.87	11.44	57.68
SSE-Bio	47.21	21.38	16.52	47.93	38.36	16.24	11.73	57.13

Appendix

Table 5: Prompt-sensitivity analysis on BioHopR. Replacing role-specific system prompts causes only small drops, indicating that SSE-Bio is not driven by brittle prompt wording.

Manager / Critic	Dev	Single Answer				Multi Answer
		Prec H1	Prec H2	Both cor	Both wr	Prec H1	Prec H2	Both cor	Both wr
Gemini-2.5-Pro	Claude-4.5-Sonnet	47.21	21.38	16.52	47.93	38.36	16.24	11.73	57.13
Gemini-2.5-Flash	Claude-4.5-Sonnet	45.96	20.14	15.86	49.08	37.02	15.12	11.04	58.36
GPT-5.5	Claude-4.5-Sonnet	46.54	20.83	16.11	48.57	37.68	15.74	11.38	57.91
GPT-5.5-Mini	Claude-4.5-Sonnet	45.31	19.62	15.42	49.74	36.44	14.83	10.72	58.92

Appendix

Table 6: API-sensitivity analysis on BioHopR. Alternative Manager/Critic APIs lead to moderate but not catastrophic degradation.

Model	Tokens (M)	Both cor
Biomni	23.6	10.45
STELLA	21.2	9.96
SSE-Bio	28.3	16.52

Appendix

Table 7: Efficiency comparison on the BioHopR test split.

Method	Tokens / Ex.	LLM Calls / Ex.	Latency / Ex.	API Cost / Ex.
Biomni	10.3K	5.8	18.4s	$0.039
STELLA	9.3K	4.9	16.7s	$0.035
SSE-Bio	12.4K	6.7	22.1s	$0.047

Appendix

Table 8: Per-example inference cost on BioHopR. SSE-Bio adds moderate overhead relative to STELLA (3.1K tokens, 1.8 calls, 5.4s, and approximately $0.012 per example) while substantially improving paired correctness. The local Proxy adds no commercial API calls; its latency is included in the end-to-end measurement.

Stage	Tokens	Data / Rollouts	GPU-h
Proxy SFT	41.6M	12.6K labelled pairs	42
Proxy GRPO	132.4M	38.5K rollouts	96

Appendix

Table 9: One-time Proxy training cost (138 GPU-hours in total), not incurred during test-time inference.

Figure 5: Effect of the triplet retrieval budget on BioHopR.

Figure 6: Effect of the template retrieval budget on BioHopR.

Hyperparameter	Value
Proxy backbone	Qwen-2.5-72B
SFT learning rate	$1\times 10^{-5}$
SFT batch size	64
SFT max length	2048
SFT # epochs	1
GRPO learning rate	$5\times 10^{-7}$

Appendix

Table 10: Key training and retrieval hyperparameters of SSE-Bio.

Statistic	Value	Interpretation
Initial persistent templates	0	The memory starts empty; no held-out BioHopR, MedHop, or HLE examples are used to seed templates.
Accepted post-success updates	2.47K	Training-only updates accepted after successful trajectories and critic validation.
New-template distillations	612	Updates whose task type does not align with an existing template above $\theta_{\mathrm{tpl}}$ .
Fine-grained edit operations	1.86K	Updates applied to aligned templates without changing the task type.
Final deduplicated templates	512	Compact memory checkpoint used for all held-out evaluations.
Covered task types	24	Twelve BioHopR relation schemas under single-answer and multi-answer variants.

Appendix

Table 11: Template-memory construction and evolution statistics on the BioHopR training split. Counts are rounded after deduplication and memory validation.

Step type	Supported if	Not supported if
Bridge resolution	A retrieved triplet or accepted template step links the query entity to the proposed bridge with the required relation type.	The bridge is introduced only from parametric generation or by a triplet with an incompatible schema.
Answer verification	The final answer is linked to the resolved bridge by a retrieved triplet, or satisfies the retrieved template’s verification criteria.	The answer is plausible but lacks explicit bridge-to-answer evidence.
Multi-answer coverage	Every returned entity has support, and the state records that candidate expansion has been exhausted within the retrieval budget.	The trajectory stops after finding only one supported entity when the task type requires a set.
No-retrieval branch	The existing retrieval state already contains the evidence needed by the current plan.	The branch avoids retrieval while the information gap remains unresolved.

Appendix

Table 12: Evidence-support rules used for

R_{\mathrm{beh}}

. These rules operationalise the behavioural reward in Sec. 3.4 without rewarding retrieval volume by itself.

Statistic	Value	Interpretation
BioHopR training pairs	5.34K	Training split used for trajectory construction and persistent template-memory updates.
Candidate rollouts before pruning	82.4K	Mean 15.4 rollouts per pair, close to the $4^{2}=16$ upper bound from expanding the first two retrieval decisions.
Valid parsed rollouts	76.8K	93.2% of generated rollouts have valid state, action, evidence, and answer fields.
Evidence-bearing non-duplicate rollouts	49.7K	Remaining rollouts after removing duplicate state-action paths and branches with empty selected evidence.
Retained GRPO rollouts	38.5K	Mean group size $K=7.21$ after the $K\leq 8$ cap; median group size is 8.0.
Discarded or pruned rollouts	43.9K	53.3% of generated rollouts are removed by parsing, duplicate/evidence filters, or the group-size cap.

Appendix

Table 13: Construction statistics for SFT pseudo-labels and GRPO trajectory groups on the BioHopR training split. Counts are rounded; each training pair seeds one decision-contrastive rollout trajectory generation.

Retrieval Policy	Single Answer				Multi Answer
	Prec H1	Prec H2	Both cor	Both wr	Prec H1	Prec H2	Both cor	Both wr
Retrieve neither	39.74	15.88	10.62	55.00	30.81	11.92	7.05	64.32
Retrieve KG only	42.36	17.41	12.43	52.84	33.18	13.06	8.52	62.37
Retrieve template only	41.58	16.96	11.86	53.46	32.47	12.64	8.07	62.98
Retrieve KG + template	43.27	18.12	13.18	51.76	34.02	13.58	9.02	61.31
Learned Proxy	47.21	21.38	16.52	47.93	38.36	16.24	11.73	57.13

Appendix

Table 14: Comparison of fixed retrieval strategies on BioHopR. The fixed strategies use the same Manager, Dev, Critic, retriever, and retrieval budget as SSE-Bio, but replace the trainable Proxy with a fixed retrieval decision at each reasoning round. “Retrieve neither” corresponds to the w/o proxy setting in Table 2 .

Setting	Single Answer				Multi Answer
	Prec H1	Prec H2	Both cor	Both wr	Prec H1	Prec H2	Both cor	Both wr
Top-5 retrieved triplets	47.21	21.38	16.52	47.93	38.36	16.24	11.73	57.13
+ 2 random noisy triplets	46.32	20.76	16.02	48.64	37.58	15.68	11.31	57.86

Appendix

Table 15: Noise-injection analysis on BioHopR. Adding two irrelevant triplets reduces single-/multi-answer Both cor by only 0.50/0.42 points, indicating reasonable robustness under this controlled perturbation.

Metric	Coarse rewriting	Fine-grained editing
Changed fields / update $\downarrow$	3.28	1.46
Unchanged-field preservation $\uparrow$	64.0	88.0
Unsupported new instruction $\downarrow$	12.0	4.0
Contradictory instruction $\downarrow$	7.0	2.0

Appendix

Table 16: Template-update quality analysis on 100 randomly sampled memory updates. Rates are percentages except for changed fields per update.

Error Source	Share (%)
Dev reasoning error	31.2
Triplet retrieval failure	24.6
Wrong Proxy action	13.1
Answer-format / coverage error	11.4
Template mismatch	10.8
Critic feedback error	8.9

Appendix

Table 17: Distribution of primary error sources over failed HLE: Biomedicine cases.

Item	STELLA	SSE-Bio
Question pattern	Drug $\rightarrow$ Gene/Protein $\rightarrow$ Disease	Drug $\rightarrow$ Gene/Protein $\rightarrow$ Disease
Question	Name a disease that is related to a gene/protein that is associated with drug (2S)-2-{[HYDROXY(4-IODOBENZYL)PHOSPHORYL]METHYL}PENTANEDIOIC ACID.
Gold bridge	FOLH1
Gold answer	{neurotic disorder, colorectal neoplasm, prostate cancer, schizophrenia, dysthymic disorder, colorectal cancer, familial prostate carcinoma, colorectal carcinoma, unipolar depression, anxiety disorder, prostate carcinoma}
Predicted bridge	AR	FOLH1
Retrieved evidence	Retrieves disease-associated context around prostate cancer and androgen-receptor signalling, but does not establish that the query drug is associated with AR.	Retrieves relation-specific triplets supporting both hops: ((2S)-2-PENTANEDIOIC ACID, associated with, FOLH1) and (FOLH1, related to, prostate cancer) .

Appendix

Table 18: A BioHopR case study comparing STELLA and SSE-Bio.

Biomedical QA

Biomedical question answering develops systems that answer health and life-science questions using clinical knowledge, research literature, and structured data.

See all 51 papers

Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework
Jun 15, 2026Xingyu Tan, Shiyuan Liu, Xiaoyang Wang +5Multi-Hop QA Evidence-Grounded Reasoning
University of New South Wales, Australia · 2CSIRO, Australia · University of Technology Sydney, Australia
Eigen-Agent: Adaptive Multi-Agent Scientific Reasoning with Monitor-Based RAG
Sep 25, 2025Xiangru Tang, Wanghan Xu, Yujie Wang +13Scientific Reasoning in Language Models Multi-Agent Reasoning
Yale University · Shanghai Jiao Tong University · Fudan University +4
SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning
May 16, 2026Yongfeng Huang, Ruiying Chen, James ChengRetrieval-Augmented Generation Healthcare
CSE, The Chinese University of Hong Kong · Wuhan University of Technology

Jun 15, 2026J/K move · Enter open · S save

Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

Xingyu Tan, Shiyuan Liu, Xiaoyang Wang, Qing Liu+4

University of New South Wales, Australia · 2CSIRO, Australia · University of Technology Sydney, Australia

Multi-Hop QA Evidence-Grounded Reasoning Graph Retrieval-Augmented Generation Biomedical QA Knowledge Graph Question Answering Biomedical KGs

Open paper AskPDF arXiv

Biomedical question answering (QA) increasingly requires reasoning over interacting entities, where supporting evidence is scattered across biomedical knowledge graphs, literature documents, and web-accessible resources. However, existing biomedical QA benchmarks mainly focus on exam-style knowledge, literature comprehension, or short-range multi-hop inference, leaving source-conditioned graph reasoning and evidence topology construction underexplored. To fill this gap, we introduce BioMedHop, a multi-source graph-grounded benchmark for evaluating biomedical reasoning over structured evidence topologies. BioMedHop contains 10,045 instances across KG, document, web, and hybrid evidence settings, covering shared-neighbor matching, intersection reasoning, path-based reasoning, and counting, with option-based, open-ended, and numeric count renderings. To support this benchmark, we further propose BioWeave, a source-aware reasoning framework that retrieves biomedical KG paths, gathers supporting clues from documents and web sources, assembles them into a unified evidence graph, and verifies answers through entity-level evidence support. Comprehensive experiments show that BioWeave achieves the best overall performance among compared methods on BioMedHop, outperforming the strong hybrid baseline ToG-2 by 10.5% in the overall average. Moreover, BioWeave consistently improves different LLM backbones and enables smaller models, such as Qwen3-4B, to achieve reasoning performance comparable to GPT-4-Turbo.

Abstract

Biomedical QA

Explore similar work

Weaving Multi-Source Evidence for Biomedical Reasoning: The BioMedHop Benchmark and BioWeave Framework

Eigen-Agent: Adaptive Multi-Agent Scientific Reasoning with Monitor-Based RAG

SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning