Organizations: Department of Computer Science, University of Liverpool, Liverpool, UK · School of Computing, University of Georgia, Athens, GA, USA · Beijing Normal University, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China · Department of Eye and Vision, University of Liverpool, Liverpool, UK
While Large Language Models demonstrate immense potential as proactive Medical Agents, their real-world deployment is severely bottlenecked by data scarcity under privacy constraints. To overcome this, we propose State-Enhanced Logical-Skill Memory (SELSM), a training-free framework that distills simulated clinical trajectories into entity-agnostic operational rules within an abstract skill space. During inference, a Query-Anchored Two-Stage Retrieval mechanism dynamically fetches these entity-agnostic logical priors to guide the agent's step-by-step reasoning, effectively resolving the state polysemy problem. Evaluated on MedAgentBench -- the only authoritative high-fidelity virtual EHR sandbox benchmarked with real clinical data -- SELSM substantially elevates the zero-shot capabilities of locally deployable foundation models (30B--32B parameters). Notably, on the Qwen3-30B-A3B backbone, our framework completely eliminates task chain breakdowns to achieve a 100% completion rate, boosting the overall success rate by an absolute 22.67% and significantly outperforming existing memory-augmented baselines. This study demonstrates that equipping models with a dynamically updatable, state-enhanced cognitive scaffold is a privacy-preserving and computationally efficient pathway for local adaptation of AI agents to clinical information systems. While currently validated on FHIR-based EHR interactions as an initial step, the entity-agnostic design of SELSM provides a principled foundation toward broader clinical deployment.
Figures & tables
Fig. 1: Overview of the SELSM framework. (Top-left) Cross-institutional deployment context: heterogeneous hospital systems (EHR, LIS, PACS, etc.) operate under institution-specific protocols, while a locally deployed LLM, guided by medical professionals, issues privacy-preserving operational queries. (Top-right) Logical Skill Distillation (Phase 1): the agent interacts with a system simulator in a closed loop, where it observes the current state s , executes an action a via the policy p=π(a∣s) , and receives the system response o . The Logical Skill Generator G parses each trajectory τ=(s,a,o) through a context encoder and experience decoder to produce entity-agnostic logical skills e=G(τ∣θ) , comprising Operational Logic, canonical examples, and reasoning traces. (Bottom) Query-Anchored Two-Stage Retrieval (Phase 3): upon receiving a new task, the system first performs Task-Level Query Filtering by scoring stored records R=(q,T) via query similarity pq , then executes Transition-Level State Ranking over candidate internal states via state similarity ps , and finally integrates the retrieved skill with the current state to guide the agent toward a better operational action.
Fig. 2: Failure mode distribution and performance improvement analysis. (a)–(c) Each stacked bar decomposes all 300 tasks into four mutually exclusive outcomes: Correct (task completed with the correct answer), Incorrect (task completed but with a wrong answer), Invalid Action (terminated due to an invalid API call), and Task Limit (terminated after exceeding the maximum number of interaction turns) for GLM4-32B, Qwen3-30B-A3B, and Qwen3-32B, respectively. (d) Absolute improvement of our method over the Baseline in percentage points (pp) across three metrics: Overall Success Rate, Query Success Rate, and Action Success Rate.
Fig. 3: Conversation efficiency analysis. (a)–(c) One-Shot Correct Rate (OSR; Eq. 14 ) for each method on GLM4-32B, Qwen3-30B-A3B, and Qwen3-32B, respectively. (d) Average number of conversation turns per task across all three backbone models.
Fig. 4: Multi-dimensional comparison of four methods on the Qwen3-30B-A3B backbone. Five normalized dimensions are shown: Success Rate ( SR/100 ; Eq. 11 ), Completion Rate ( TC/100 ; Eq. 10 ), Error Robustness (ER; Eq. 15 ), Efficiency ( OSR/100 ; Eq. 14 ), and Query-Action Balance (QAB; Eq. 16 ). A larger enclosed area indicates better overall performance.
Fig. 5: Token efficiency analysis. Each point represents one method–model configuration, with the x -axis showing Cˉtok (Eq. 17 ) and the y -axis showing SR (Eq. 11 ). Colors denote methods (Baseline, A-Mem, ExpeL, Ours) and marker shapes denote backbone models (square: GLM4-32B, circle: Qwen3-30B-A3B, triangle: Qwen3-32B). Points closer to the upper-left corner indicate higher accuracy at lower token cost.
Fig. 6: Cross-model generalization analysis. (a)–(c) Overall, Query, and Action Success Rates of each method across three backbone models. (d) Standard deviation of conversation turn counts per task (given comparable completion rates, lower values indicate more consistent agent behavior). All methods are evaluated on the same 300-task benchmark.
We introduce PhysicianBench, a benchmark for evaluating LLM agents on physician tasks grounded in real clinical setting within electronic health record (EHR) environments. Existing medical agent benchmarks primarily focus on static knowledge recall, single-step atomic actions, or action intent without verifiable execution against the environment. As a result, they fail to capture the long-horizon, composite workflows that characterize real clinical systems. PhysicianBench comprises 100 long-horizon tasks adapted from real consultation cases between primary care and subspecialty physicians, with each task independently reviewed by a separate panel of physicians. Tasks are instantiated in an EHR environment with real patient records and accessed through the same standard APIs used by commercial EHR vendors. Tasks span 21 specialties (e.g., cardiology, endocrinology, oncology, psychiatry) and diverse workflow types (e.g., diagnosis interpretation, medication prescribing, treatment planning), requiring an average of 27 tool calls per task. Solving each task requires retrieving data across encounters, reasoning over heterogeneous clinical information, executing consequential clinical actions, and producing clinical documentation. Each task is decomposed into structured checkpoints (670 in total across the benchmark) capturing distinct stages of completion graded by task-specific scripts with execution-grounded verification. Across 13 proprietary and open-source LLM agents, the best-performing model achieves only 46% success rate (pass@1), while open-source models reach at most 19%, revealing a substantial gap between current agent capabilities and the demands of real-world clinical workflows. PhysicianBench provides a realistic and execution-grounded benchmark for measuring progress toward autonomous clinical agents.
Ruoqi Liu, Imran Q. Mohiuddin, Austin J. Schoeffler +10
Clinical large language model (LLM) agents deployed locally can consult more capable remote models, but doing so risks exposing patient information. Privacy-conscious delegation places disclosure decisions with a local agent, yet removing explicit identifiers is insufficient: quasi-identifiers can accumulate across multi-turn consultations and repeated patient visits to enable re-identification. We introduce PrivMeSA, a privacy-aware self-evolving multi-agent system that learns to control disclosure and retains remote expertise for local reuse. A local agent manages each encounter and consults remote specialists that may request additional information. Reinforcement learning balances task accuracy against direct disclosure and registry-based re-identification risk, with privacy evaluated over the complete outbound transcript of each encounter. A local lesson memory distills completed consultations into generalized clinical guidance and retrieves relevant lessons before transmission, allowing subsequent cases to reuse expertise without another remote exchange. Memory grows without additional outcome labels or parameter updates. On an emergency-department benchmark built from MIMIC-IV-ED records, PrivMeSA improves mean task accuracy over delegation by up to 15.8 percentage points. In the same setting, PrivMeSA reduces the disclosure of personal details from 98.0% to 0.2% of cases and the share of cases in which the patient can be narrowed to ten or fewer registry patients from 74% to 0%.
Dannong Wang, Yuran Zhang, Bian Sun +4
Institute of Artificial Intelligence, University of Central Florida · Department of Computer Science and Operations Research, Universit´e de Montr´eal · Department of Medicine, University of Central Florida
Multi-modal large language models (MM-LLMs) have shown strong performance in medical image understanding and clinical reasoning. Recent medical agent systems extend them with tool use and multi-agent collaboration, enabling complex decision-making. However, these systems rely almost entirely on frontier models (e.g., GPT), whose API-based deployment incurs high cost, high latency, and privacy risks that conflict with on-premise clinical requirements. We present Meissa, a lightweight 4B-parameter medical MM-LLM that brings agentic capability offline. Instead of imitating static answers, Meissa learns both when to engage external interaction (strategy selection) and how to execute multi-step interaction (strategy execution) by distilling structured trajectories from frontier models. Specifically, we propose: (1) Unified trajectory modeling: trajectories (reasoning and action traces) are represented within a single state-action-observation formalism, allowing one model to generalize across heterogeneous medical environments. (2) Three-tier stratified supervision: the model's own errors trigger progressive escalation from direct reasoning to tool-augmented and multi-agent interaction, explicitly learning difficulty-aware strategy selection. (3) Prospective-retrospective supervision: pairing exploratory forward traces with hindsight-rationalized execution traces enables stable learning of effective interaction policies. Trained on 40K curated trajectories, Meissa matches or exceeds proprietary frontier agents in 10 of 16 evaluation settings across 13 medical benchmarks spanning radiology, pathology, and clinical reasoning. Using over 25x fewer parameters than typical frontier models like Gemini-3, Meissa operates fully offline with 22x lower end-to-end latency compared to API-based deployment. Data, models, and environments are released at https://github.com/Schuture/Meissa.
Yixiong Chen, Xinyi Bai, Yue Pan +2
Johns Hopkins University, Baltimore, MD, USA · Cornell University, Ithaca, NY, USA