GSEM: Graph-based Self-Evolving Memory for Experience Augmented Clinical Reasoning

Xiao Han; Yuzheng Fan; Sendong Zhao; Haochun Wang; Bing Qin

cs.AIMar 23, 2026

GSEM: Graph-based Self-Evolving Memory for Experience Augmented Clinical Reasoning

Authors: Xiao Han, Yuzheng Fan, Sendong Zhao, Haochun Wang, Bing Qin

Organizations: Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China

Abstract

Clinical decision-making agents can benefit from reusing prior decision experience. However, many memory-augmented methods store experiences as independent records without explicit relational structure, which may introduce noisy retrieval, unreliable reuse, and in some cases even hurt performance compared to direct LLM inference. We propose GSEM (Graph-based Self-Evolving Memory), a clinical memory framework that organizes clinical experiences into a dual-layer memory graph, capturing both the decision structure within each experience and the relational dependencies across experiences, and supporting applicability-aware retrieval and online feedback-driven calibration of node quality and edge weights. Across MedR-Bench and MedAgentsBench with two LLM backbones, GSEM achieves the highest average accuracy among all baselines, reaching 70.90% and 69.24% with DeepSeek-V3.2 and Qwen3.5-35B, respectively. Code is available at https://github.com/xhan1022/gsem.

Figures & tables

Figure 1: Two motivating failure cases of experience-augmented clinical reasoning. Boundary Failure (top): an experience is retrieved based on partial condition match, while critical boundary conditions are ignored. Collaboration Failure (bottom): conflicting experiences without joint-use value are co-retrieved, leading to incoherent guidance for the final decision. A detailed clinical case study is provided in Appendix F .

Figure 2: Overall architecture of GSEM, illustrating the three-stage pipeline across Memory Construction, Memory Retrieval, and Memory Evolution.

Model	Method	MedR-Bench		MedAgentsBench					AVG
Model	Method	Diag	Treat	MedBullets	MedQA	MedMCQA	MedExQA	MedXpertQA	AVG
DeepSeek-V3.2	Vanilla	87.87	57.43	48.31	74.62	40.00	28.00	31.00	64.78
	Naïve RAG	92.40	84.46	47.19	74.84	38.00	29.00	27.00	68.56
	GraphRAG	90.27	87.16	42.70	68.93	38.00	27.00	24.00	65.61
	Mem0	86.63	57.43	39.32	73.74	36.00	26.00	29.00	62.96
	Mem0 g	83.28	56.08	50.56	74.62	37.00	29.00	27.00	63.19

Table 1: Main results on MedR-Bench and MedAgentsBench. Acc denotes outcome accuracy (%). All MedAgentsBench results report Pass@1 accuracy (%). Best results are bold and second-best are underlined .

Method	MedR-Bench
Method	Diagnosis / Acc	Treatment / Acc
GSEM	94.22	94.59
w/ SE(50)	97.26	97.30
w/ SE(150)	97.87	95.95
w/ SE(250)	96.96	97.30

Table 2: Effect of the number of online evolution updates on GSEM performance. Results are reported after online memory evolution using different numbers of samples. Bold indicates the best performance.

Setting	Diag.	Treat.	MedBullets
w/o entity_recall	92.71	92.57	24.00
w/o embedding_recall	93.62	83.78	10.00
w/o multi_seed_retrieval	91.79	92.57	23.00
GSEM	94.22	94.59	34.00

Table 3: Ablation results on diagnosis (Diag.) / treatment (Treat.) accuracy and QA benchmark MedBullets. Bold indicates the best performance per task.

Figure 3: Per-dimension reasoning quality (Efficiency, Factuality, Completeness, and R (AVG)) on MedR-Bench diagnosis and treatment tasks. Error bars denote two-sided 95% confidence intervals.

Method	MedR-Bench
Method	Diagnosis / R	Treatment / R
GraphRAG	$86.9_{\pm 3.6}$	$\mathbf{89.0}_{\pm 5.0}$
A-Mem	$78.8_{\pm 4.4}$	$83.9_{\pm 5.9}$
FLEX	$77.0_{\pm 4.5}$	$77.3_{\pm 6.7}$
GSEM	$\mathbf{87.4}_{\pm 3.6}$	$84.3_{\pm 5.9}$

Table 4: Reasoning quality on MedR-Bench (mean

\pm

CI). R denotes the aggregated reasoning quality score. Bold indicates the best performance.

Retriever + Generator	Diag. / Acc	Treat. / Acc
Qwen + Qwen	91.49	66.89
Qwen + DeepSeek	96.66	95.95
DeepSeek + Qwen	87.84	61.49
DeepSeek + DeepSeek	94.22	94.59

Table 5: Effect of model size on GSEM by swapping the generation model and the retrieval model. Bold indicates the best performance.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Distribution of node quality $Q$ in self-evolving stage.

Figure 5: Distribution of relation weight $W$ in self-evolving stage.

Figure 6: Examples of experiences, illustrating the four-field structure

(c_{i},s_{i},z_{i},Q_{i})

defined in Section 3.2 .

Figure 7: Visualization of a subgraph in the entity layer $\mathcal{G}^{ent}$ . Nodes correspond to typed decision entities, and edges capture internal reasoning flows within and across experiences.

Figure 8: Visualization of a subgraph in the experience layer $\mathcal{G}^{exp}$ . Node size encodes experience quality $Q_{i}$ , and edge weights reflect learned inter-experience associations.

Method	Efficiency	Factuality	Completeness	Diagnosis/R (AVG)
GraphRAG	$\mathbf{0.983}_{\pm.014}$	$0.842_{\pm.039}$	$0.782_{\pm.045}$	$0.869_{\pm.036}$
A-Mem	$\mathbf{0.983}_{\pm.014}$	$0.610_{\pm.053}$	$0.770_{\pm.045}$	$0.788_{\pm.044}$
FLEX	$\mathbf{0.983}_{\pm.014}$	$0.588_{\pm.053}$	$0.740_{\pm.047}$	$0.770_{\pm.045}$
GSEM	$0.978_{\pm.016}$	$\mathbf{0.852}_{\pm.038}$	$\mathbf{0.793}_{\pm.044}$	$\mathbf{0.874}_{\pm.036}$

Appendix

Table 6: Per-dimension reasoning quality on MedR-Bench Diagnosis (n=329) with 95% CI.

Method	Efficiency	Factuality	Completeness	Treatment/R (AVG)
GraphRAG	$\mathbf{0.993}_{\pm.013}$	$\mathbf{0.918}_{\pm.044}$	$\mathbf{0.760}_{\pm.069}$	$\mathbf{0.890}_{\pm.050}$
A-Mem	$0.992_{\pm.014}$	$0.815_{\pm.063}$	$0.709_{\pm.073}$	$0.839_{\pm.059}$
FLEX	$0.990_{\pm.016}$	$0.662_{\pm.076}$	$0.667_{\pm.076}$	$0.773_{\pm.067}$
GSEM	$0.989_{\pm.017}$	$0.836_{\pm.060}$	$0.704_{\pm.074}$	$0.843_{\pm.059}$

Appendix

Table 7: Per-dimension reasoning quality on MedR-Bench Treatment (n=148) with 95% CI.

	GSEM	ReMe
Retrieved Exps	exp_0907: history of prior radiotherapy + new progressive deficit in irradiated field; exp_0858: undifferentiated sarcoma with small round cell morphology, pre-treatment planning; + 6 other sarcoma-related experiences	exp1 : interpreting conflicting clinical data (PET-negative lesions vs. known progressive disease); exp2 : mixed biomarker findings (driver mutation + PD-L1) in metastatic cancer
Key Steps	Step 2: Recognizes abscopal effect as immune-mediated response. Step 4: Decides close observation is appropriate while spontaneous regression is ongoing; defers systemic therapy unless progression occurs.	Step 4: exp1 frames spontaneous regression as “conflicting data” and overrides it. Step 5–6: Proceeds to select third-line systemic therapy (pazopanib) despite ongoing regression.
Final Answer	Close surveillance with PET-CT in 6–8 weeks; initiate anti-PD1 immunotherapy only if regression halts or progresses. (Correct)	Initiate pazopanib (800 mg/day) immediately as third-line therapy; refer to immunotherapy clinical trial. (Incorrect)

Appendix

Table 8: Case 1 (PMC11631726): Boundary Failure. A 50-year-old male with metastatic undifferentiated pleomorphic sarcoma (UPS) showing spontaneous regression attributed to the abscopal effect after multiple surgeries, radiotherapy, and two lines of chemotherapy. Ground truth: continue close follow-up leveraging the abscopal effect; do not initiate systemic therapy while regression is ongoing.

	GSEM	ReMe
Retrieved Exps	exp_1315: elderly male with hemodynamic collapse and retroperitoneal bleeding; exp_0769: acute abdomen with pneumoperitoneum and hollow viscus perforation; exp_0796: hospitalized patient with sudden severe abdominal pain and hemodynamic instability; + 3 other abdominal emergency experiences	exp1 : generating surgical steps for acute abdominal emergencies; exp2 : formulating treatment plans when explicit diagnoses are provided; exp3 : dental trauma classification discrepancy (irrelevant); exp5 : IBD with pre-established diagnosis (irrelevant)
Key Reasoning Steps	Step 3: Identifies IVC laceration and gastroduodenal artery as immediate hemorrhage sources. Step 3–4: Recognizes pancreaticoduodenal injury severity and concludes EPD is required for irreparable injury; includes Whipple resection in the definitive surgical plan.	Step 2: exp1 drives immediate surgical step listing without resolving EPD decision. Step 3: exp2 accepts injury list and moves to damage control; pancreaticoduodenal injury is deferred to staged reconstruction rather than addressed on-site.
Final Answer	Emergency damage control laparotomy including IVC repair, right nephrectomy, gastroduodenal artery ligation, and EPD (Whipple resection) for irreparable pancreatic head injury; postoperative ICU management. (Correct)	Emergency exploratory laparotomy with IVC repair, nephrectomy, liver packing; pancreaticoduodenal injury deferred with drainage and staged reconstruction after 24–48 hours. (Incorrect)

Appendix

Table 9: Case 2 (PMC11490744): Collaboration Failure. A 49-year-old male with blunt abdominal trauma, hemodynamic instability, and multiple organ injuries including irreparable pancreatic head and duodenal injuries. Ground truth: perform emergency pancreaticoduodenectomy (Whipple procedure) combined with IVC repair, right nephrectomy, adrenalectomy, cholecystectomy, and liver hemostasis.

Panel of Table 10 — Table 10: System prompt for Indication experience extraction.

Panel of Table 11 — Table 11: Human prompt for Indication experience extraction.

Panel of Table 12 — Table 12: System prompt for Contraindication divergence analysis.

Panel of Table 13 — Table 13: Human prompt for Contraindication divergence analysis.

Panel of Table 14 — Table 14: System prompt for Contraindication experience extraction.

Panel of Table 15 — Table 15: Human prompt for Contraindication experience extraction.

Panel of Table 16 — Table 16: System prompt for entity extraction.

Panel of Table 17 — Table 17: Human prompt for entity extraction.

Panel of Table 18 — Table 18: System prompt for role-edge structure extraction.

Panel of Table 19 — Table 19: Human prompt for role-edge structure extraction.

Panel of Table 20 — Table 20: System prompt for semantic similarity evaluation.

Panel of Table 21 — Table 21: Human prompt for semantic similarity evaluation.

Explore similar work

MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents
Aug 22, 2026Md Asaduzzaman Jabin, Khoa Le, Lin Zhao +1LLM Agent Memory Memory-Augmented Language Model Agents
University of Georgia, Athens, GA, USA · New Jersey Institute of Technology, Newark, NJ, USA
Experience Makes Skillful: Enabling Generalizable Medical Agent Reasoning via Self-Evolving Skill Memory
Jun 8, 2026Haoran Sun, Wenjie Li, Yujie Zhang +8Continual Learning for LLM Agents LLM Agent Memory
Fudan University · Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China +1
EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse
Sep 14, 2026Dongsheng Shi, Yue Li, Xin Yi +1Continual Learning for LLM Agents Multi-Agent LLM Systems
East China Normal University · City University of Hong Kong

Aug 22, 2026J/K move · Enter open · S save

MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents

Md Asaduzzaman Jabin, Khoa Le, Lin Zhao, Tianming Liu

University of Georgia, Athens, GA, USA · New Jersey Institute of Technology, Newark, NJ, USA

LLM Agent Memory Memory-Augmented Language Model Agents Multimodal Agents Agentic AI Continual Learning for LLM Agents

Open paper AskPDF arXiv

Clinical decision-making is inherently experience-driven: physicians progressively refine their reasoning by synthesizing patient history, multimodal observations, and prior diagnostic experiences across interactions. In contrast, current multimodal large language model (MLLM)-based medical AI agents largely operate as stateless inference systems, generating decisions independently for each interaction without retaining or internalizing experiential knowledge. This discrepancy limits their ability to progressively improve reasoning reliability through usage and adapt to longitudinal patient contexts in real-world clinical workflows. In this study, we propose Medical Structured Multimodal Memory (MSM-Mem), an agentic memory framework that enables medical AI agents to evolve through accumulated clinical experiences. MSM-Mem organizes heterogeneous clinical experiences into semantic, episodic, and visual memory and incrementally updates them during inference, allowing the agent to retrieve prior experiences to inform current reasoning and progressively refine decision-making over time. Evaluations on MoE-LLaVA backbones demonstrate consistent performance improve- ments with further gains observed through continued usage. In general, MSM-Mem offers a viable pathway toward medical AI agents capable of evolving their reasoning competence in a manner analogous to the way clinicians learn from practice over time.

Abstract

Explore similar work

MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents

Experience Makes Skillful: Enabling Generalizable Medical Agent Reasoning via Self-Evolving Skill Memory

EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse