Complex clinical decision making often fails not because a model lacks facts, but because it cannot reliably select and apply the right procedural knowledge and the right prior example at the right reasoning step. We frame clinical question answering as an agent problem with two explicit, retrievable resources: skills, reusable clinical procedures such as guidelines, protocols, and pharmacologic mechanisms; and experience, verified reasoning trajectories from previously solved cases (e.g., chain-of-thought solutions and their step-level decompositions). At test time, the agent retrieves both relevant skills and experiences from curated libraries and performs lightweight test-time adaptation to align the language model's intermediate reasoning with clinically valid logic. Concretely, we build (i) a skills library from guideline-style documents organized as executable decision rules, (ii) an experience library of exemplar clinical reasoning chains indexed by step-level transitions, and (iii) a step-aware retriever that selects the most useful skill and experience items for the current case. We then adapt the model on the retrieved items to reduce instance-step misalignment and to prevent reasoning from drifting toward unsupported shortcuts. Experiments on medical question-answering benchmarks show consistent gains over strong medical RAG baselines and prompting-only reasoning methods. Our results suggest that explicitly separating and retrieving clinical skills and experience, and then aligning the model at test time, is a practical approach to more reliable medical agents.
Figures & tables
Figure 1: Motivation and overview of Skills & Experience Library + TARSE . (Left) A clinical QA example where the key difficulty is goal-conditional disambiguation : cisplatin’s ototoxicity (free radicals → hearing loss) is a side-effect chain, while its antitumor mechanism is the therapeutic chain (covalent DNA binding → DNA cross-linking → apoptosis). Humans solve this by combining experience (recalling a similar bladder-cancer + platinum-chemo case that links the symptom to cisplatin) with skills (reusable guideline/mechanism logical chains) to apply a “mechanism vs. side-effect” gate and select the correct branch (DNA cross-linking, option D). In contrast, a vanilla LLM tends to follow heuristic associations and conflates toxicity with mechanism, and traditional chunk RAG retrieves true but unstructured snippets that are not step-aligned, leading to the same confusion. (Right) Our method operationalizes this workflow: (1) retrieve similar experience traces and relevant skills; (2) perform lightweight test-time adaptation to align reasoning with the question intent; (3) generate a provisional hypothesis/chain; and (4) verify/repair it using retrieved skills to produce the final answer. The bottom-right plot summarizes the accuracy–latency trade-off, showing that TARSE achieves larger gains than CoT-only and conventional RAG baselines at comparable inference time.
Figure 2: TARSE process. Starting from a real clinical diagnosis question (top), the agent first retrieves a similar verified experience trace from an Experience Library (Step 1), then performs experience-conditioned hypothesis generation to propose candidate mechanisms (Step 2). Next, it retrieves Skills from a Skills Library (e.g., guideline/protocol rules) together with supporting experience items to verify key transitions and enforce “gate” checks (Step 3), producing a refined reasoning trace that cleanly separates the side-effect chain (cisplatin → free radicals → ototoxicity/hearing loss) from the therapeutic mechanism chain (cisplatin → DNA cross-linking → apoptosis). The final answer follows the verified therapeutic chain (Step 4).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Ablation study for TARSE. (Top) Effect of the number of retained small-batch samples used in logical chain supervision. (Bottom) Impact of varying the number of parallel users during batch-wise test-time training.
Logical Chain Retriever
73.27
77.49
80.13
81.79
ColBERT Retriever
62.33
65.12
67.48
70.51
Appendix
Table 6: Top- k retrieval accuracy (%) comparison between Logical Chain Retriever and ColBERT Retriever.
Extract Key Points
Based on the question and answer, summarize ten key points that you consider to be the most crucial from the standard answer. Return the response in the following format: {1.2.3….} Here is the question:{question} Here is the answer:{answer} Please do not provide any additional information.
Key Points
1. Multifocal electroretinogram (ERG) showed reduced signal in the right eye throughout the macula, confirming the diagnosis of AZOOR.2. Acute zonal occult outer retinopathy (AZOOR) was first described by Gass in 1993…
Extract Diagnostic Reasoning
Based on the question and answer, please provide a detailed summary of the diagnostic reasoning from the standard answer. Return the response in the following format: {1.2.3….} Here is the question:{question} Here is the answer:{answer} Please do not provide any additional information.
Diagnostic Reasoning
1. The patient is a 7-year-old boy with a slowly growing, asymptomatic lump on the left lower neck since birth.2. Physical examination showed a yellowish, hump-like mass with a hairy surface and cartilage-like consistency near the left sternocleidomastoid muscle…
Extract Evidence
Based on the question and answer, please provide a detailed evidence list which is proposed by correct answer. Return the response in the following format: {1.2.3….} Here is the question:{question} Here is the answer:{answer} Please do not provide any additional information.
Evidence
1. Slowly growing, asymptomatic lump on left lower neck since birth.2. Physical examination revealed a yellowish, hump-like mass with hairy surface and cartilage-like consistency.3. Ultrasonography indicated a hypoechoic, avascular, bulging nodule with an anechoic tubular structure.4. MRI demonstrated a protuberant nodule with diffuse…
Key Points Score
Act as a USMLE evaluator, your role involves assessing and comparing a medical student’s explanation to the provided target answer. Begin the assessment by carefully reviewing the provided target answer. Then, based on following specific criteria, determine the score for the student’s answer. Please judge whether medical student’s answer include these key points(or some other relevant points. But the amount of points must be complete). For example, ground truth have 10 key points, if student answer include one key he will get 0.5 point(if the answer include 5 points so should be 2.5). Medical student’s answer: {answer} Key Points: {Key Point} Please only return a float number(from 0 to 5). You should check each point one by one(shouldn’t judge based on language style such as fluence and so on. Only judge based on whether the student’s answer include correct or relevant and complete key points). Don’t generate any other information.
Appendix
Table 9: Evaluation Pipeline Prompt Example Template.
Question: A 67-year-old man with transitional cell carcinoma of the bladder comes to the physician because of a 2-day history of ringing sensation in his ear. He received this first course of neoadjuvant chemotherapy 1 week ago. Pure tone audiometry shows a sensorineural hearing loss of 45 dB. The expected beneficial effect of the drug that caused this patient’s symptoms is most likely due to which of the following actions? A: Inhibition of proteasome, B: Hyperstabilization of microtubules, C:Generation of free radicals, D: Cross-linking of DNA
Phi-4: To determine the expected beneficial effect of the drug that caused the patient’s symptoms, we need to identify the drug likely responsible for the sensorineural hearing loss and understand its mechanism of action. 1. The patient has transitional cell carcinoma of the bladder and received neoadjuvant chemotherapy. 2. A common chemotherapy drug used for bladder cancer is cisplatin. 3. Cisplatin is known to cause ototoxicity, which can manifest as sensorineural hearing loss. 4 Cisplatin will lead generation of free radicals and lead hearing loss hence the final answer should be C.
Qwen-2.5-14B-Instruct Forms of chemotherapy used in transitional cell carcinoma of the bladder often include platinum-based agents like cisplatin. Cisplatin is known for its ototoxicity, which can lead to sensorineural hearing loss. -> Cisplatin’s mechanism of action involves the generation of free radicals, which can damage cellular components including those in the auditory system. -> Therefore, the generation of free radicals is the most likely action causing the patient’s symptoms. -> The correct answer is C
TARSE:
Small Batch Finetune:
Pretrain Data: MDMA: 3,4-Methylenedioxymethamphetamine (MDMA), commonly known as ecstasy (E) or molly, is a psychoactive drug primarily used for recreational purposes. The desired effects include altered sensations, increased energy, empathy, as well as pleasure. When taken by mouth, effects begin in 30 to 45 minutes and last 3 to 6 hours…
Finetune Data: Scenario: Dr. Smith, a seasoned urologist, is preparing for a surgical procedure to repair an anterior urethral stricture in a 45-year-old male patient. The patient is informed that a surgical resident will be involved in the operation as part of their training. The patient expresses concern about the potential impact of the resident’s involvement on the surgery’s…
Retrieval-Augmented Generation (RAG) is widely employed to mitigate risks such as hallucinations and knowledge obsolescence in medical question answering, yet its predominantly single-round, static retrieval paradigm misaligns with the multi-stage process of clinical reasoning. This compressed workflow induces two structural deficiencies: question-to-query translation often lacks clinically grounded semantic interpretation, and retrieval lacks iterative sufficiency feedback, making it difficult to form reliable evidence chains. We argue that both issues stem from a deeper cause: overloading a single reasoning chain with heterogeneous tasks of interpretation, exploration, and adjudication. The remedy is to reconstruct the workflow via task decoupling and dynamic multi-round exploration. To this end, we propose SEMA-RAG, a Self-Evolving Multi-Agent RAG framework for medical question answering, which assigns these roles to three specialist agents: the Interpreter Agent for clinical schema interpretation, the Explorer Agent for sufficiency-driven self-evolving retrieval, and the Arbiter Agent for evidence adjudication and answer selection. Across five benchmarks and five LLM backbones, SEMA-RAG improves the strongest baseline by +6.46 accuracy points on average, measured per backbone.
Yongfeng Huang, Ruiying Chen, James Cheng
CSE, The Chinese University of Hong Kong · Wuhan University of Technology
Medical agent systems are increasingly expected to support interactive clinical decision making rather than only static question answering. In such settings, effective agents must reuse prior experience across evolving cases, yet existing memory mechanisms often retain raw historical traces that are redundant, noisy, and difficult to govern. More importantly, they rarely distinguish which memories are truly useful for future reasoning. This limits their ability to accumulate compact and reliable experience for long-horizon clinical reasoning. To close this gap, we propose SkeMex, a post-deployment self-evolution framework that improves medical agents through a skill-based memory without updating model weights. SkeMex distills informative interaction trajectories into structured skills that encode reusable procedural knowledge, and organizes them into a multi-branch repository spanning general, task-specific, and action-level experience. To determine which memories should be reused and retained, SkeMex estimates context-dependent utility from environment feedback and uses it to guide value-aware retrieval and repository governance. A closed-loop ``Read--Write--Assess--Govern" lifecycle further supports continual evolution by writing new skills, updating utilities, promoting useful memories, and removing harmful entries. Experiments across diverse clinical tasks show that SkeMex consistently outperforms representative memory-based agents in both offline and online settings. It also generalizes across model backbones and supports transferable skill memory. All data and code will be released publicly.
Haoran Sun, Wenjie Li, Yujie Zhang +8
Fudan University · Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China +1
Clinical reasoning agents based on large language models (LLMs) aim to automate tasks such as intensive care unit (ICU) monitoring and patient state tracking from electronic health records (EHRs). Existing systems typically rely on manually curated clinical tools or skills for concepts such as sepsis detection and organ failure assessment. However, maintaining these tool libraries requires substantial expert effort, while zero-shot querying or code generation often produces inefficient and unreliable reasoning chains, especially under institution-specific clinical policies. We introduce CodeClinic, a benchmark built on MIMIC-IV for evaluating whether LLM agents can synthesize and compose reusable clinical skills instead of relying on fixed toolboxes. The benchmark contains two complementary tasks: longitudinal ICU surveillance and compositional information seeking. The longitudinal setting simulates monitoring patient trajectories with structured decisions every four hours across 25 findings and eight clinical families, while the compositional setting spans 63k instances across 259 tasks in nine domains and is stratified by compositional dependency depth to evaluate increasingly complex multi-step reasoning. We further propose an offline autoformalization pipeline that converts natural-language clinical guidelines into reusable and verified Python skill libraries through iterative LLM refinement. Compared with zero-shot code generation, the resulting libraries improve consistency while reducing per-query token usage by up to 40%.
Timothy Ossowski, Xinchi Liu, Danyal Maqbool +6
University of Wisconsin–Madison · 2Microsoft Research