cs.CLApr 8, 2026

Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent

Authors: Bingxuan Li, Simo Du, Yue Guo

Organizations: University of Illinois Urbana-Champaign · Albert Einstein College of Medicine

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Clinical expertise improves not only by acquiring medical knowledge, but by accumulating experience that yields reusable diagnostic patterns. Recent LLMs-based diagnostic agents have shown promising progress in clinical reasoning for decision support. However, most approaches treat cases independently, limiting experience reuse and continual adaptation. We propose SEA, a self-learning diagnostic agent with cognitively inspired dual-memory module. We design a reinforcement training framework tailored to our designed agent for joint optimization of reasoning and memory management. We evaluate SEA in two complementary settings. On standard evaluation with MedCaseReasoning dataset, SEA achieves 92.46% accuracy, outperforming the strongest baseline by +19.6%, demonstrating the benefit of jointly optimizing reasoning and memory. On the long-horizon with ER-Reason dataset, SEA attains the best final accuracy (0.7214) and the largest improvement (+0.35 Acc@100), while baseline methods show limited or unstable gains. Expert evaluation further indicates that rules consolidated from SEA show strong clinical correctness, usefulness and trust, suggesting that the induced rules in dual-memory module are reliable and practically meaningful. Overall, SEA improves both diagnostic reasoning ability and continual learning by effectively transforming experience into reusable knowledge.

Figures & tables

Figure 1: Overview of SEA : At each round tt , the policy model observes a patient case xtx_{t} and may invoke memory operations before emitting the final output oto_{t} (diagnosis and reasoning). The agent controls a short-term memory cluster that stores recent patient cases with a bounded capacity KK ( list / append / pop ) and a long-term memory cluster that consolidates experience into abstracted diagnosis rules ( list / consolidate ). For each round, we sample a group of candidate outputs, score them with round-wise rewards, compute group-wise advantages, and update the policy.
ModelAccuracy (%)Macro-F1 (%)Std.
Zeroshot
Qwen3-4b43.240.00.025
Qwen3-8b45.141.90.025
Qwen3-14B50.442.90.024
OLMo3-7B47.143.90.025
LLaMa-3.1-8B42.139.80.025
Table 1: Performance comparison across methods under the standard evaluation setting.
Figure 2: Accuracy trajectories from 10 to 100 rounds for representative methods.
MethodFinal Accuracy (%)Δ\boldsymbol{\Delta} Acc@50 (%)Δ\boldsymbol{\Delta} Acc@100 (%)
Zeroshot
Qwen-4b41.9-5.0+8.0
Qwen-8b44.6+1.0+6.0
GPT-5.253.2+8.0+13.0
Trained Baselines
Qwen-8b (SFT)56.8-2.0+1.0
Table 2: Final accuracy and improvement across different methods.
MethodAblated ModuleAcc.Δ\boldsymbol{\Delta}
Zero-shotTraining & Memory Modules0.45-0.47
DiagnosticReward (w/o memory)All Memory Modules0.77-0.15
DiagnosticReward+ShortTerm-memoryLongTerm Memory&MemoryReward0.82-0.10
DiagnosticReward+LongTerm-MemoryShortTerm Memory&MemoryReward0.72-0.20
DiagnosticReward+Dual-MemoryMemory Reward0.47-0.45
SEANone (full model)0.92—
Table 3: Ablation study on MedCaseReasoning. Δ\Delta denotes performance change relative to the full model. Darker color indicates larger performance drop.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Task setup: At each round tt , the environment provides a patient case xtx_{t} together with a candidate diagnosis set Yt\mathcal{Y}_{t} . The agent outputs an action ata_{t} consisting of a structured reasoning trace and a final prediction y^t∈Yt\hat{y}_{t}\in\mathcal{Y}_{t} . The environment then returns feedback ftf_{t} (e.g., correct/incorrect or graded), which the agent can leverage as experience for subsequent rounds. This interaction repeats for TT rounds, modeling continual on-the-job improvement in a streaming setting.
Evaluation aspectStandard evaluationLong-horizon evaluation
DatasetMedCaseReasoningER-Reason
Training on the target datasetYes, using its training splitNo
Test-case organizationCases are evaluated independentlyCases arrive sequentially as a stream
Feedback during evaluationNoYes, after each round
Memory update during evaluationNo feedback-based self-learningYes, previous experience is incorporated
Primary purposeEvaluate static diagnostic competenceEvaluate OOD sequential adaptation and experience reuse
Appendix
Table 4: Comparison of standard and long-horizon evaluation settings.
More in this topic

Clinical Reasoning

Clinical reasoning studies how AI systems interpret patient evidence, form and revise diagnoses, and support safe decisions across care settings.

See all 180 papers

Explore similar work

CardsList
Mar 27, 2026

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

While Large Language Models (LLMs) have demonstrated potential in healthcare, they often struggle with the complex, non-linear reasoning required for accurate clinical diagnosis. Existing methods typically rely on static, linear mappings from symptoms to diagnoses, failing to capture the iterative, hypothesis-driven reasoning inherent in human clinicians. To bridge this gap, we introduce ClinicalAgents, a novel multi-agent framework designed to simulate the cognitive workflow of expert clinicians. Unlike rigid sequential chains, ClinicalAgents employs a dynamic orchestration mechanism modeled as a Monte Carlo Tree Search (MCTS) process. This allows an orchestrator to iteratively generate hypotheses, actively verify evidence, and trigger backtracking when critical information is missing. The foundation of this framework is a Dual-Memory architecture: a mutable working memory that maintains the evolving patient state for context-aware reasoning, and a static experience memory that retrieves clinical guidelines and historical cases via an active feedback loop. Extensive experiments demonstrate that ClinicalAgents achieves the best performance among evaluated baselines, significantly enhancing both diagnostic accuracy and explainability compared to strong single-agent and multi-agent baselines. Our code is released at https://github.com/ZhuohanGe/ClinicalAgents-Code.

Zhuohan Ge, Haoyang Li, Yubo Wang +3

The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Clinical Decision-MakingMulti-Agent LLM SystemsMedical DiagnosisClinical ReasoningMemory-Augmented Language Model AgentsLLM Agent Planning
AskPDF
May 20, 2026

MedExpMem: Adapting Experience Memory for Differential Diagnosis

Experienced physicians develop diagnostic expertise through clinical practice, acquiring not only disease knowledge but also the ability to differentiate confusable conditions. Current medical vision-language models (VLMs) lack this capability -- their parameters encode static knowledge that does not evolve across diagnostic encounters. We propose MedExpMem, an experience memory framework enabling VLM-based diagnostic agents to accumulate differential diagnosis expertise. Unlike retrieval-augmented generation, which retrieves encyclopedic disease descriptions, MedExpMem memorizes discriminative experience derived from the agent's own diagnostic failures and organizes them as pairwise differential notes encoding key discriminators, actionable decision rules and reasoning error patterns. The framework adopts a two-phase construction process mirroring physician learning: initial practice exposes knowledge gaps, and reflective re-diagnosis refines understanding. When encountering new cases, the agent retrieves experience memory to guide differential reasoning. We evaluate MedExpMem on a radiology benchmark spanning 11 subspecialties. Results demonstrate consistent accuracy improvements, maximum 7.0%, across diverse models and scales. Analytical experiments validate experience quality and robustness, demonstrating MedExpMem as a competitive method addresses medical adaptation needs beyond the reach of parameteric learning.

Qianhan Feng, Zhongzhen Huang, Yakun Zhu +4

The Chinese University of Hong Kong · Shanghai Jiao Tong University · Shanghai Innovation Institue

Radiology VLMsHealthcareVLM AdaptationMedical DiagnosisMedical VLMsMedical Reasoning+2
AskPDF
Sep 14, 2026

EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reuse

Large language model (LLM) driven multi-agent systems have shown promise in complex clinical reasoning, yet existing approaches rely on static strategies and lack persistent clinical memory, preventing self-evolving from prior diagnostic successes and failures. We present EMR, a self-evolving medical multi-agent system via Experience Mining and Reuse. EMR introduces a hierarchical clinical experience library that organizes accumulated knowledge into three levels: clinical principles, diagnostic patterns, and representative cases. During inference, EMR emulates multidisciplinary consultation: a planner agent coordinates domain-specific department agents for specialized reasoning, while a summary agent synthesizes their analyses into a final decision. Critically, EMR automatically extracts correct diagnostic insights and failure-related warnings from multi-agent reasoning trajectories, incrementally updating the experience library to guide future cases. Experiments on medical reasoning benchmarks demonstrate that EMR consistently outperforms state-of-the-art medical multi-agent baselines. Further analysis reveals that the hierarchical experience enables cross-specialty generalization and transfer across diverse LLM backbones, offering a scalable and in

Dongsheng Shi, Yue Li, Xin Yi +1

East China Normal University · City University of Hong Kong

Continual Learning for LLM AgentsMulti-Agent LLM SystemsLLM Agent Self-ImprovementClinical ReasoningMedical LLMsMemory-Augmented Language Model Agents
AskPDF