Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent

Bingxuan Li; Simo Du; Yue Guo

cs.CLApr 8, 2026

Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent

Authors: Bingxuan Li, Simo Du, Yue Guo

Organizations: University of Illinois Urbana-Champaign · Albert Einstein College of Medicine

Abstract

Clinical expertise improves not only by acquiring medical knowledge, but by accumulating experience that yields reusable diagnostic patterns. Recent LLMs-based diagnostic agents have shown promising progress in clinical reasoning for decision support. However, most approaches treat cases independently, limiting experience reuse and continual adaptation. We propose SEA, a self-learning diagnostic agent with cognitively inspired dual-memory module. We design a reinforcement training framework tailored to our designed agent for joint optimization of reasoning and memory management. We evaluate SEA in two complementary settings. On standard evaluation with MedCaseReasoning dataset, SEA achieves 92.46% accuracy, outperforming the strongest baseline by +19.6%, demonstrating the benefit of jointly optimizing reasoning and memory. On the long-horizon with ER-Reason dataset, SEA attains the best final accuracy (0.7214) and the largest improvement (+0.35 Acc@100), while baseline methods show limited or unstable gains. Expert evaluation further indicates that rules consolidated from SEA show strong clinical correctness, usefulness and trust, suggesting that the induced rules in dual-memory module are reliable and practically meaningful. Overall, SEA improves both diagnostic reasoning ability and continual learning by effectively transforming experience into reusable knowledge.

Figures & tables

Figure 1: Overview of SEA : At each round $t$ , the policy model observes a patient case $x_{t}$ and may invoke memory operations before emitting the final output $o_{t}$ (diagnosis and reasoning). The agent controls a short-term memory cluster that stores recent patient cases with a bounded capacity $K$ ( list / append / pop ) and a long-term memory cluster that consolidates experience into abstracted diagnosis rules ( list / consolidate ). For each round, we sample a group of candidate outputs, score them with round-wise rewards, compute group-wise advantages, and update the policy.

Model	Accuracy (%)	Macro-F1 (%)	Std.
Zeroshot
Qwen3-4b	43.2	40.0	0.025
Qwen3-8b	45.1	41.9	0.025
Qwen3-14B	50.4	42.9	0.024
OLMo3-7B	47.1	43.9	0.025
LLaMa-3.1-8B	42.1	39.8	0.025

Table 1: Performance comparison across methods under the standard evaluation setting.

Figure 2: Accuracy trajectories from 10 to 100 rounds for representative methods.

Method	Final Accuracy (%)	$\boldsymbol{\Delta}$ Acc@50 (%)	$\boldsymbol{\Delta}$ Acc@100 (%)
Zeroshot
Qwen-4b	41.9	-5.0	+8.0
Qwen-8b	44.6	+1.0	+6.0
GPT-5.2	53.2	+8.0	+13.0
Trained Baselines
Qwen-8b (SFT)	56.8	-2.0	+1.0

Table 2: Final accuracy and improvement across different methods.

Method	Ablated Module	Acc.	$\boldsymbol{\Delta}$
Zero-shot	Training & Memory Modules	0.45	-0.47
DiagnosticReward (w/o memory)	All Memory Modules	0.77	-0.15
DiagnosticReward+ShortTerm-memory	LongTerm Memory&MemoryReward	0.82	-0.10
DiagnosticReward+LongTerm-Memory	ShortTerm Memory&MemoryReward	0.72	-0.20
DiagnosticReward+Dual-Memory	Memory Reward	0.47	-0.45
SEA	None (full model)	0.92	—

Table 3: Ablation study on MedCaseReasoning.

\Delta

denotes performance change relative to the full model. Darker color indicates larger performance drop.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Task setup: At each round $t$ , the environment provides a patient case $x_{t}$ together with a candidate diagnosis set $\mathcal{Y}_{t}$ . The agent outputs an action $a_{t}$ consisting of a structured reasoning trace and a final prediction $\hat{y}_{t}\in\mathcal{Y}_{t}$ . The environment then returns feedback $f_{t}$ (e.g., correct/incorrect or graded), which the agent can leverage as experience for subsequent rounds. This interaction repeats for $T$ rounds, modeling continual on-the-job improvement in a streaming setting.

Evaluation aspect	Standard evaluation	Long-horizon evaluation
Dataset	MedCaseReasoning	ER-Reason
Training on the target dataset	Yes, using its training split	No
Test-case organization	Cases are evaluated independently	Cases arrive sequentially as a stream
Feedback during evaluation	No	Yes, after each round
Memory update during evaluation	No feedback-based self-learning	Yes, previous experience is incorporated
Primary purpose	Evaluate static diagnostic competence	Evaluate OOD sequential adaptation and experience reuse

Appendix

Table 4: Comparison of standard and long-horizon evaluation settings.

Clinical Reasoning

Clinical reasoning studies how AI systems interpret patient evidence, form and revise diagnoses, and support safe decisions across care settings.

See all 180 papers

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory
Mar 27, 2026Zhuohan Ge, Haoyang Li, Yubo Wang +3Clinical Decision-Making Multi-Agent LLM Systems
The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
MedExpMem: Adapting Experience Memory for Differential Diagnosis
May 20, 2026Qianhan Feng, Zhongzhen Huang, Yakun Zhu +4Radiology VLMs Healthcare
The Chinese University of Hong Kong · Shanghai Jiao Tong University · Shanghai Innovation Institue
EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse
Sep 14, 2026Dongsheng Shi, Yue Li, Xin Yi +1Continual Learning for LLM Agents Multi-Agent LLM Systems
East China Normal University · City University of Hong Kong

Mar 27, 2026J/K move · Enter open · S save

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

Zhuohan Ge, Haoyang Li, Yubo Wang, Nicole Hu+2

The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Clinical Decision-Making Multi-Agent LLM Systems Medical Diagnosis Clinical Reasoning Memory-Augmented Language Model Agents LLM Agent Planning

Open paper AskPDF arXiv

While Large Language Models (LLMs) have demonstrated potential in healthcare, they often struggle with the complex, non-linear reasoning required for accurate clinical diagnosis. Existing methods typically rely on static, linear mappings from symptoms to diagnoses, failing to capture the iterative, hypothesis-driven reasoning inherent in human clinicians. To bridge this gap, we introduce ClinicalAgents, a novel multi-agent framework designed to simulate the cognitive workflow of expert clinicians. Unlike rigid sequential chains, ClinicalAgents employs a dynamic orchestration mechanism modeled as a Monte Carlo Tree Search (MCTS) process. This allows an orchestrator to iteratively generate hypotheses, actively verify evidence, and trigger backtracking when critical information is missing. The foundation of this framework is a Dual-Memory architecture: a mutable working memory that maintains the evolving patient state for context-aware reasoning, and a static experience memory that retrieves clinical guidelines and historical cases via an active feedback loop. Extensive experiments demonstrate that ClinicalAgents achieves the best performance among evaluated baselines, significantly enhancing both diagnostic accuracy and explainability compared to strong single-agent and multi-agent baselines. Our code is released at https://github.com/ZhuohanGe/ClinicalAgents-Code.

Abstract

Clinical Reasoning

Explore similar work

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

MedExpMem: Adapting Experience Memory for Differential Diagnosis

EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse