MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction

Ruoyu Wu; Shenfu Xie; Yinqian Sun; Haibo Tong; Feifei Zhao

cs.AIAug 24, 2026

MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction

Authors: Ruoyu Wu, Shenfu Xie, Yinqian Sun, Haibo Tong, Feifei Zhao

Organizations: Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences · Beijing Key Laboratory of Safe AI and Superalignment · Beijing Institute of AI Safety and Governance · School of Artificial Intelligence, University of Chinese Academy of Sciences ·

Abstract

Interactive clinical agents operate under partial observability, so reliable care depends on reaching the correct diagnosis through evidence-grounded, safe interactions. Yet existing agents struggle to convert experience into reusable process knowledge with explicit provenance and authority. To address this gap, we introduce MediSkill-Evo, which self-evolves governed process knowledge without fine-tuning the backbone. It realizes this self-evolution by updating clinical, process, symbolic, and visual knowledge in four typed banks under type-specific validation and scope rules. The Process-Constrained Preference Harness then turns validated knowledge into action by grounding candidates in evidence and prioritizing safer decisions. We evaluate on 300 MIMIC-IV-derived FullChain encounters, 180 hard-isolation conditions covering six process obligations, and 100 multimodal NEJM image-diagnosis cases. On Qwen FullChain, MediSkill-Evo improves diagnosis accuracy by 7.81% and treatment-intent coverage by 70.67% over the best-performing prior agent, while reducing critical failures by 43.04%. Under stress, it improves the stress-process composite by 7.77% and required-action completion by 12.41% over the best-performing agent for each metric, with stronger patient-fact, temporal-evidence, and triage-red-flag recovery and no controller-scored errors in unavailable-evidence, treatment, and triage safety checks. On multimodal NEJM diagnosis, MediSkill-Evo with optional MedSAM localization improves diagnosis accuracy by 2.56% and core score by 18.96% over the best-performing memory agent. Code is available at https://anonymous.4open.science/r/mediskill-evo_anonymous-68E7.

Figures & tables

Figure 1. Online inference and offline learning and governance in MediSkill-Evo. Left: the online pipeline retrieves four-bank context, generates and verifies candidate actions, executes the selected action, and incorporates environment feedback. Right: completed trajectories are reflected into typed memory operations, validated for label/evidence leakage, controller-defined safety, support, and replayability, and published as the next memory snapshot. Two equally sized MediSkill-Evo diagrams separated by a vertical dashed line. The left panel presents the seven-stage online inference pipeline, and the right panel presents the six-stage offline learning and governance pipeline.

Figure 2. Two frozen-test interactions scored diagnosis-correct by the automatic evaluator. Left: targeted history, available laboratory evidence, an unavailable ultrasound, and a focused physical examination support acute cholecystitis without treating a missing result as negative. Right: medication history, original radiograph and CT evidence, optional MedSAM localization, and a follow-up blood count support nitrofurantoin-induced pulmonary toxicity; this label does not validate anatomy or management. A side-by-side comparison showing the key doctor, patient, and measurement turns in a text-only acute cholecystitis case and a multimodal nitrofurantoin pulmonary toxicity case. The multimodal case displays the original chest radiograph and CT before the corresponding MedSAM overlays.

Dataset	Train	Test	Modality
MIMIC-IV FullChain	700	300	Text
Controlled clinical stress	420	180	Text
NEJM FullChain Interactive	200	100	Text + image

Table 1. Evaluation datasets. FullChain encounters derive from MIMIC-IV ( Johnson et al., 2023 ) and use the AgentClinic interaction protocol ( Schmidgall et al., 2024 ) ; multimodal cases derive from the NEJM image collection ( The New England Journal of Medicine, 2026 ) . Stress counts denote controlled conditions; the 420/180 conditions are derived from 70/30 disjoint underlying clinical cases.

Dimension	Dimension construction	Primary capability under evaluation
Diagnosis difficulty	Delay decisive source evidence until the Doctor uses its registered question, examination, or test channel.	Recovering and using the decisive evidence while avoiding premature closure.
Evidence completeness	Make one source item unavailable while retaining an independent evidence path that preserves case solvability.	Requesting the missing item, treating unavailability as unknown, and using alternative evidence without fabricating a result.
Patient behavior complexity	Partition patient-knowable source facts so that they are released only after focused, relevant questions.	Recovering patient facts through focused, respectful questioning without changing the underlying disease facts.
Treatment and prescription	Gate a source-supported treatment prerequisite behind the corresponding history, examination, or test request.	Verifying prerequisites and choosing conditional treatment, safe deferral, or an alternative rather than an unsafe action.
Temporal dynamics	Delay a source-supported timeline fact until the Doctor asks a temporally targeted question.	Recovering the original timeline and integrating it into reassessment, treatment, monitoring, or escalation.
Triage safety	Withhold a source-supported red flag until the Doctor performs the appropriate symptom or risk screen.	Recovering the red flag, escalating appropriately, and avoiding unsafe reassurance.

Table 2. Controlled-stress dimensions, source-grounded construction, and enforced process obligations. Each condition changes only the visibility or release timing of source-supported information; hidden values remain outside every runtime LLM prompt and are released only by the deterministic controller.

Metric group	Operational definition
Standard and multimodal evaluation
Outcome	Dx $\uparrow$ : accepted final diagnosis; Tx/Rx $\uparrow$ : reference treatment intents covered, with unsafe care penalized.
Evidence acquisition	Hist. $\uparrow$ : required history elicited; Tests $\uparrow$ : required examinations requested; Evid. $\uparrow$ : joint history-and-test coverage.
Risk control	Safety $\downarrow$ : observable safety violation; Critical $\downarrow$ : critical diagnostic, treatment, or triage failure.
Efficiency and completion	Unnec. $\downarrow$ : unjustified examinations; Int.Eff. $\uparrow$ : gated interaction efficiency; Core $\uparrow$ : composite automatic score penalized for critical failure.
Visual measurement	Masks : cases with at least one non-empty MedSAM mask.

Table 3. Evaluation metrics and their operational meanings. Metrics are grouped by evaluation setting and function; arrows indicate the preferred direction.

Backbone	Method	Benchmark outcome		Evidence acquisition			Risk and economy			Int.Eff. $\uparrow$
Backbone	Method	Dx $\uparrow$	Tx/Rx $\uparrow$	Evid. $\uparrow$	Hist. $\uparrow$	Tests $\uparrow$	Safety $\downarrow$	Critical $\downarrow$	Unnec. $\downarrow$	Int.Eff. $\uparrow$
Qwen3.6-Flash	AgentClinic	61.33	33.62	12.31	11.70	13.46	1.67	31.00	15.89	39.60
	Agent-KB	61.00	38.43	14.52	11.26	17.45	0.67	32.33	13.82	41.42
	ExPeL	59.00	35.50	14.58	11.09	17.43	2.33	33.00	14.03	40.28
	MemP	64.00	38.93	15.16	12.45	17.41	1.67	28.67	12.79	41.40
	Reflexion	62.33	38.27	13.38	11.06	15.65	1.33	31.00	13.25	41.18

Table 4. Main automatic operational results (%) on the same 300 FullChain test encounters. Backbones represent the Qwen ( Yang et al., 2025 ) and DeepSeek ( DeepSeek-AI, 2024 ) families; baseline methods are cited in the accompanying text. Higher is better except for Safety, Critical, and Unnec. Bold marks the best result within each backbone; shaded rows denote MediSkill-Evo.

Figure 3. Primary controlled-stress process and safety profile (%). (a) Dimension-aware required-action completion across the six pressure families. (b) The 19 constituent recovery, use, handling, and failure metrics; cell text gives the raw percentage, whereas color is direction-corrected so that darker always denotes better behavior. DTR, HqR, KTR, BTR, TSR, TER, and RFR are controller-grounded recovery endpoints. Fixed-denominator metrics use 30 cases per dimension except HqR and KTR, whose history/test subtypes contain 10 cases each.

\dagger

marks a conditional denominator determined by a visible request or recovered target and must be read with the corresponding recovery row. Bold cell text marks a unique best. A two-panel controlled-stress comparison. The left radar chart compares four agents on six dimension-aware process-completion scores. The right annotated heatmap reports 19 recovery, use, handling, and safety submetrics for the same agents; darker cells indicate better direction-adjusted behavior while printed values retain the raw percentages.

Method	Stress $\uparrow$	Req.Act. $\uparrow$	Core $\uparrow$	Dx $\uparrow$	Tx/Rx $\uparrow$	Safety $\downarrow$	Critical $\downarrow$
AgentClinic	77.75	70.39	75.56	96.67	78.36	0.00	1.11
MemP	72.84	76.81	67.96	87.78	75.31	1.67	7.22
Reflexion	76.06	78.55	71.33	90.56	78.86	1.11	2.78
MediSkill-Evo	83.79	88.30	80.03	93.89	82.40	1.11	2.78

Table 5. Auxiliary cross-dimension aggregates and conventional automatic outcomes on all 180 controlled stress conditions (%). Stress is the registered stress-process composite and Req.Act. is its dimension-aware required-action component. Higher is better except for Safety and Critical.

Agent/condition	Ready	Dx $\uparrow$	Hist. $\uparrow$	Tests $\uparrow$	Unnec. $\downarrow$	Core $\uparrow$	Masks
MemP	100/100	38.00	58.60	59.00	12.75	35.95	0/100
Reflexion	100/100	39.00	58.63	67.00	12.77	40.29	0/100
MediSkill-Evo (raw)	100/100	37.00	75.58	100.00	5.00	44.69	0/100
MediSkill-Evo (+MedSAM)	100/100	40.00	79.50	100.00	6.58	47.93	34/100

Table 6. Four-agent results on 100 multimodal NEJM image cases ( The New England Journal of Medicine, 2026 ) . Ready and Masks are counts; all other values are macro-averaged percentages over the same 100 saved case trajectories, with no error rows. Masks counts cases with at least one non-empty MedSAM mask.

Variant	Dx $\uparrow$	Tx/Rx $\uparrow$	Safe. $\downarrow$	Crit. $\downarrow$	Unnec. $\downarrow$
No memory	67.00	36.50	1.00	30.00	11.50
w/o Clinical	74.00	63.65	1.00	24.00	29.75
w/o Process	71.00	60.70	8.00	36.00	30.50
w/o Symbolic	70.00	68.90	0.00	15.00	19.50
Full	76.00	69.40	0.00	13.00	19.17

Table 7. Component analysis on the same 100 FullChain test cases. Each variant disables the indicated knowledge bank while retaining the registered model, interaction budget, case set, and evaluator. All values are percentages.

Stage	Principal inputs	Gold allowed?	Structured output
Trajectory reflection	Completed training trace, evaluator feedback, case targets	Train only	Reflection and failure attribution
Doctor-bank proposal	Validated reflection, observed symbolic traces, active artifact IDs	Train only	Typed bank mutations
Visual measurement	Original pixels, optional MedSAM artifacts, retrieved Measurement Skills	No	Evidence-only visual report
Candidate and critic	Visible state, process ledger, retrieved banks, action portfolio	No	Validated action scores
Final safety path	Visible trajectory, proposed plan, diagnosis-blind safety frame	No	Rewritten plan and release decision
Offline evaluator	Completed frozen-test trace and hidden scoring targets	Eval only	Case metrics and evidence indexes

Table S1. Prompt inventory and information boundaries. Gold information is permitted only after a training encounter or during offline evaluation.

Dimension	Controller-grounded recovery	Visible-trajectory use and safety
Diagnosis difficulty	Fraction of delayed discriminators released after the correct patient, examination, or test action.	Fraction of recovered discriminators used; premature closure.
Evidence completeness	Whether the permanently unavailable history or test was explicitly requested.	Safe handling of an observed unavailable response; hallucination of its value.
Patient behavior	Fraction of delayed patient facts released after focused questions.	Focused-question adequacy; unrecovered-target failure.
Treatment/prescription	Fraction of delayed medication, allergy, pregnancy, or renal prerequisites released.	Safe conditional treatment, alternative, or deferral; unsafe action.
Temporal dynamics	Fraction of delayed source timeline facts released.	Fraction of recovered timeline facts integrated into assessment or management.
Triage safety	Fraction of delayed source red flags released.	Escalation adequacy; unsafe reassurance.

Table 12

Setting	Backbone alias	Candidates	Doctor ceiling	MediSkill-Evo test-time banks
FullChain	qwen3.6-flash	3	6	C/P/S frozen; M inactive
FullChain	deepseek-v4-flash	3	6	C/P/S frozen; M inactive
Controlled stress	qwen3.6-flash	3	6	C/P/S frozen; M inactive
NEJM, original image	qwen3.6-flash	3	8	C/P/S frozen; condition-specific M frozen
NEJM, optional MedSAM	qwen3.6-flash	3	8	C/P/S frozen; condition-specific M frozen

Table S2. Registered evaluation configuration. All rows use one observed rollout per case–configuration pair. “Inactive” means that no Measurement Bank is queried.

System	Frozen objects	Retrieval cap	Test-time controller
AgentClinic	0	0	Original Doctor, no experience bank.
Agent-KB	249	15	Native structured-database retrieval and Doctor prompt.
ExPeL	258	25	Native cross-trial insight retrieval and Doctor prompt.
MemP	236	15	Native procedural-memory retrieval and Doctor prompt.
Reflexion	264	20	Native reflection-memory retrieval and Doctor prompt.
SkillWeaver	245	20	Native skill retrieval and Doctor prompt.

Table S3. Registered Qwen FullChain comparator ledger. “Native” means the cited method’s memory-to-Doctor prompt adaptation; none of these rows receives the MediSkill-Evo Harness.

Metric	Better	Tie	Worse
Diagnosis	54	215	31
Treatment intent	266	17	17
Evidence recall	287	10	3
History recall	290	10	0
Test recall	133	106	61
Auto safety violation	5	294	1

Table S4. Paired Qwen case transitions from existing frozen traces (

n=300

Self-Evolving Agents

Self-evolving agents improve their skills, memory, tools, policies, or environments by learning from interaction and evaluation feedback.

See all 423 papers

Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
Apr 15, 2026Weixiang Shen, Bailiang Jian, Jun Li +6Medical AI Agents LLM Agent Self-Improvement
Technical University of Munich (TUM) · TUM University Hospital · LMU Munich +3
MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes
Jun 27, 2026Hui ZhangContinual Learning for LLM Agents Long-Horizon Agent Evaluation
Beijing Institute of Technology, Beijing, China
EGIP: An Evidence-Grounded Interaction Protocol for Clinical Agents
Oct 17, 2026Zecheng Ren, Ruichen Cong, Yi Min Chen +8Evidence-Grounded Reasoning Clinical Decision-Making
Waseda University, Tokorozawa, Japan · Shanghai Jiao Tong University, Shanghai, China

Apr 15, 2026J/K move · Enter open · S save

Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve

Weixiang Shen, Bailiang Jian, Jun Li, Che Liu+5

Technical University of Munich (TUM) · TUM University Hospital · LMU Munich+3

Medical AI Agents LLM Agent Self-Improvement LLM Agent Memory Continual Learning for LLM Agents LLM Agent Orchestration Medical Image Analysis

Open paper AskPDF arXiv

Tool-augmented large language model (LLM) agents can orchestrate specialist classifiers, segmentation models, and visual question-answering modules to interpret chest X-rays. However, these agents still solve each case in isolation: they fail to accumulate experience across cases, correct recurrent reasoning mistakes, or adapt their tool-use behavior without expensive reinforcement learning. While a radiologist naturally improves with every case, current agents remain static. In this work, we propose Evo-MedAgent, a self-evolving memory module that equips a medical agent with the capacity for inter-case learning at test time. Our memory comprises three complementary stores: (1)~Retrospective Clinical Episodes that retrieve problem-solving experiences from similar past cases, (2)~an Adaptive Procedural Heuristics bank curating priority-tagged diagnostic rules that evolves via reflection, much like a physician refining their internal criteria, and (3)~a Tool Reliability Controller that tracks per-tool trustworthiness. On ChestAgentBench, Evo-MedAgent raises multiple-choice question (MCQ) accuracy from 0.68 to 0.79 on GPT-5-mini, and from 0.76 to 0.87 on Gemini-3 Flash. With a strong base model, evolving memory improves performance more effectively than orchestrating external tools on qualitative diagnostic tasks. Because Evo-MedAgent requires no training, its per-case overhead is bounded by one additional retrieval pass and a single reflection call, making it deployable on top of any frozen model.

Abstract

Self-Evolving Agents

Explore similar work

Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve

MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes

EGIP: An Evidence-Grounded Interaction Protocol for Clinical Agents