Healthcare

Follow this topic Ask about this topic

Latest papers 533

38 papers in the last four weeks, up 41% on the four weeks before. 0.3% of all new papers.

Feb 1, 2026cs.AI

MedBeads: An AI-Native Clinical Context Graph Built from Immutable Beads and Reconstructable Clinical Links

Generative AI can encode substantial medical knowledge, but patient-specific answers remain constrained by the context supplied at inference time. Electronic health records and FHIR support documentation and interoperability, but they do not by themselves define the complete, current, and auditable context a model should receive. Similarity-based retrieval can find related text, but it neither guarantees collection of clinically connected records nor makes omissions explicit. We introduce MedBeads, an AI-facing clinical record substrate that assembles a declared closure of longitudinal patient information before generation. A Bead is an immutable clinical or knowledge object identified by SHA-256 over canonical content and stored as an append-only frame in a patient-scoped Pod. Structural parent edges form a patient-rooted Merkle DAG. Typed clinical links occupy a separate, reconstructable interpretation layer derived from signed, versioned knowledge rules; they can be recomputed when knowledge changes without rewriting clinical facts. Retrieval follows authorized structural and clinical edges, resolves amendments and retractions, and reports policy or token truncation. An open-source Go implementation uses append-only Pods and reconstructable SQLite projections. File-based conversion of 1,135 synthetic Synthea FHIR bundles produced approximately one million Beads and demonstrated deterministic clinical-link derivation and interpretation-layer reconstruction. These engineering results establish feasibility and reproducibility, not reduced hallucination or improved clinical outcomes. MedBeads reframes grounding as a data-structure problem by delivering a policy-bounded, provenance-bearing clinical subgraph rather than an opaque list of similar fragments.

Takahito Nakajima

Diagnostic Imaging and Interventional Radiology, Institute of Medicine, University of Tsukuba · Center for Cyber Medicine Research, University of Tsukuba

KG Construction LLM Grounding Healthcare Longitudinal EHR Modeling Electronic Health Records

Feb 1, 2026cs.CV

MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection

Medical Anomaly Detection (MedAD) offers a promising direction for medical image analysis with Large Multimodal Models (LMMs). However, progress is limited by fragmented datasets and the tendency of Supervised Fine-Tuning (SFT) to learn superficial image-text correlations rather than verifiable diagnostic reasoning. Consequently, current models often generate fluent explanations that are either insufficiently grounded in the image or inconsistent with their final answers, limiting their reliability in high-stakes medical applications. To address these issues, we introduce MedAD-38K, a large-scale, multimodal, and multicenter benchmark containing structured Visual Question Answering pairs and quality-controlled diagnostic Chain-of-Thought annotations across five core MedAD tasks. Based on this benchmark, we propose a two-stage framework. Cognitive Injection first uses SFT to inject domain-specific medical knowledge and establish a structured think-then-answer format. Consistency Group Relative Policy Optimization (Con-GRPO) then employs an Evidence-Aware Consistency Reward to reinforce reasoning that remains grounded in the image and logically supports the final answer. The resulting MedAD-R1 achieves state-of-the-art performance on MedAD-38K and consistently outperforms all evaluated baselines across five source-disjoint external datasets spanning diverse imaging modalities. Beyond accuracy, it achieves higher reasoning-answer consistency and visual-grounding scores across multiple backbones. With only 0.8B parameters, MedAD-R1 offers practical potential for resource-constrained deployment. These results demonstrate the generality of evidence-aware consistency optimization for interpretable MedAD. Project resources are available at https://github.com/zhtstar/MedAD-R1.

Haitao Zhang, Yingying Wang, Jiaxiang Wang +5

School of Informatics, Xiamen University · Institute of Artificial Intelligence, Xiamen University · Zhejiang Expressway Co., Ltd. +3

Healthcare Group Relative Policy Optimization Medical Image Anomaly Detection Multimodal Large Language Models Medical VQA Multimodal Anomaly Detection+2

Jan 25, 2026cs.AI

Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context

Safety alignment in Large Language Models is critical for healthcare; however, reliance on binary refusal boundaries often results in over-refusal of benign queries or unsafe compliance with harmful ones. While existing benchmarks measure these extremes, they fail to evaluate Safe Completion: the model's ability to maximise helpfulness on dual-use or borderline queries by providing safe, high-level guidance without crossing into actionable harm. We introduce Health-ORSC-Bench, the first large-scale benchmark designed to systematically measure Over-Refusal and Safe Completion quality in healthcare. Comprising 31,920 benign boundary prompts across seven health categories (e.g., self-harm, medical misinformation), our framework uses an automated pipeline with human validation to test models at varying levels of intent ambiguity. We evaluate 30 state-of-the-art LLMs, including GPT-5 and Claude-4, revealing a significant tension: safety-optimised models frequently refuse up to 80% of "Hard" benign prompts, while domain-specific models often sacrifice safety for utility. Our findings demonstrate that model family and size significantly influence calibration: larger frontier models (e.g., GPT-5, Llama-4) exhibit "safety-pessimism" and higher over-refusal than smaller or MoE-based counterparts (e.g., Qwen-3-Next), highlighting that current LLMs struggle to balance refusal and compliance. Health-ORSC-Bench provides a rigorous standard for calibrating the next generation of medical AI assistants toward nuanced, safe, and helpful completions. Furthermore, our benchmark facilitates reproducible evaluation, encourages safety calibration, and supports development of clinically reliable, context-aware, human-aligned medical AI systems. Our code and data are available at: https://github.com/ZhihaoZhang97/Health-ORSC-Bench. Warning: Some contents may include toxic or undesired contents.

Zhihao Zhang, Liting Huang, Guanghao Wu +3

Macquarie University1 · UTS2 · MBZUAI3 +1

LLM Safety Benchmarks Language Model Safety Evaluation Healthcare LLM Refusal LLM Refusal Behavior Over-Refusal+2

Jan 23, 2026cs.AI

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

Large language models (LLMs) deployed in clinical decision support may acquiesce to patient requests for care that conflicts with evidence-based guidelines. We developed SycoEval-EM, a multi-agent simulation framework to evaluate LLM robustness to adversarial patient persuasion in emergency medicine. Across 19 contemporary LLMs and 1,425 simulated clinical encounters spanning three Choosing Wisely scenarios, acquiescence rates ranged from 0% to 100%, revealing a bimodal distribution. Seven models maintained near-perfect guideline adherence, while six acquiesced in the majority of encounters. Vulnerability varied substantially across clinical scenarios. Acquiescence was highest for CT imaging requests, intermediate for antibiotic prescriptions for sinusitis, and lowest for opioid prescriptions for acute back pain. Model scale, recency, and performance on static medical benchmarks did not consistently predict robustness. All five persuasion tactics produced similar acquiescence rates, with no statistically significant differences after correction for multiple comparisons, suggesting a generalized susceptibility rather than tactic-specific weaknesses. LLM-as-judge evaluation was validated against two independent physician raters across 95 matched conversations and demonstrated near-perfect agreement for the primary outcome of acquiescence (Cohens kappa = 0.957). These findings indicate that static medical benchmarks are insufficient to predict safety performance under sustained social pressure and support incorporating multi-turn adversarial testing into clinical AI evaluation. Notably, two models achieved perfect guideline adherence across all encounters, demonstrating that robustness to patient pressure is attainable without sacrificing effective clinical communication.

Dongshen Peng, Yi Wang, Austin Schoeffler +5

UNC Chapel Hill · University of Waterloo · Stanford University

LLM Sycophancy Language Model Safety Evaluation Healthcare Adversarial Robustness of Language Models Clinical Language Model Evaluation Medical LLMs

Jan 20, 2026cs.AI

LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning

Personalized lifestyle health analysis requires long-horizon, multi-dimensional reasoning over heterogeneous lifestyle signals, and recent advances in mobile sensing and large language models (LLMs) make such support increasingly feasible. However, the capabilities of current LLMs in this setting remain insufficiently understood due to the lack of systematic benchmarks. In this paper, we introduce LifeAgentBench, a large-scale QA benchmark for long-horizon, cross-dimensional, and multi-user lifestyle health reasoning, containing 22,573 questions spanning from basic retrieval to complex reasoning. We release an extensible benchmark construction pipeline and a standardized evaluation protocol, deriving verifiable answers through executable queries and programs to support reliable assessment. We then systematically evaluate 13 representative LLMs on LifeAgentBench and identify key bottlenecks in long-horizon aggregation and cross-dimensional reasoning. Motivated by these findings, we propose LifeAgent, a tool-augmented reasoning baseline that decomposes complex queries, performs multi-step evidence retrieval, and invokes tools for deterministic aggregation. LifeAgent substantially enhances LLMs' capabilities on challenging reasoning tasks, achieving clear improvements over widely used baselines and showing potential for health reasoning in everyday scenarios. The benchmark is publicly available.

Ye Tian, Zihao Wang, Onat Gungor +2

University of California San Diego, La Jolla, CA, USA · Google Research, Mountain View, CA, USA

Healthcare Long-Horizon Agent Evaluation AI Agent Benchmarks LLM Tool Use

Jan 6, 2026cs.CL

EpiQAL: Benchmarking Large Language Models in Epidemiological Question Answering and Reasoning

Reliable epidemiological reasoning requires synthesizing study evidence to infer disease burden, transmission dynamics, and intervention effects at the population level. Existing medical question answering benchmarks primarily emphasize clinical knowledge or patient-level reasoning, yet few systematically evaluate evidence-grounded epidemiological inference. We present EpiQAL, to our knowledge the first diagnostic benchmark for epidemiological question answering over research literature, comprising three subsets built from open-access articles across diverse diseases. The three subsets progressively test factual recall, multi-step inference, and conclusion reconstruction under incomplete information, and are constructed through a quality-controlled pipeline combining taxonomy guidance, multi-model verification, and difficulty screening. Experiments on fifteen models spanning open-source and proprietary systems reveal that current LLMs show limited performance on epidemiological reasoning, with multi-step inference posing the greatest challenge. Model rankings shift across subsets, and scale alone does not predict success. Chain-of-Thought prompting benefits multi-step inference but yields mixed results elsewhere. EpiQAL provides fine-grained diagnostic signals for evidence-grounding, inferential reasoning, and conclusion reconstruction.

Mingyang Wei, Dehai Min, Zewen Liu +8

1Emory University · University of Illinois Chicago · 3Microsoft

LLM Evaluation Healthcare Evidence-Grounded Reasoning Biomedical QA Medical Reasoning

Jan 6, 2026cs.CL

MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence

Trustworthy clinical summarization requires every claim to be traceable to its evidence, yet existing attribution often resolves only to the sentence or document, leaving clinicians to scan surrounding text for the few words that matter. We argue that the unit of attribution should match the unit of verification: the precise phrase the reader's eye must land on. We present MAPLE (Medical Aspect-Based Summarization with Phrase-Level Evidence), a human-annotated benchmark that grounds each summarized claim in both cited sentences and contributory phrases within them. Spanning 152 randomized controlled trial (RCT) abstracts and 16 clinically motivated aspects, MAPLE comprises 1,799 aspect-based summaries with two-level evidence. We further introduce a decoupled evaluation framework that separately scores content, traceability, and locatability, together with a proxy for the amount of source text a clinician must inspect to verify a claim. Benchmarking eleven LLMs shows that sentence-level citation is consistently strong (C-F1 up to 90.9%), while phrase-level grounding remains less stable and the most discriminative axis across models (P-F1 66.1-84.5%). These results suggest that the key challenge is not only producing accurate summaries, but localizing their supporting evidence precisely enough for efficient clinical verification. Data and code are available at https://github.com/chubohao/maple.

Bohao Chu, Hendrik Damm, Tabea M. G. Pakull +5

University of Duisburg-Essen · University of Applied Sciences and Arts Dortmund · University Hospital Essen +1

Healthcare Text Summarization Summarization Evaluation Evidence-Grounded Generation Clinical Language Model Evaluation

Jan 5, 2026cs.MM

ELF: A Family of Encoder-Free ECG-Language Models

ECG-Language Models (ELMs) extend recent advances in Multimodal Large Language Models (MLLMs) to automated ECG interpretation. However, most existing ELMs inherit Vision-Language Model (VLM) design choices and rely on pretrained ECG encoders, introducing substantial architectural and training complexity. Inspired by encoder-free VLMs, we introduce ELF, a family of three encoder-free ELMs that remain competitive with, and often outperform, prior state-of-the-art ELMs across two datasets despite substantially simpler architectures and training pipelines. All code and data are available at github.com/ELM-Research/ECG-Language-Models.

William Han, Tony Chen, Chaojing Duan +6

Carnegie Mellon University, USA · Allegheny Health Network, USA · University of California, Los Angeles, USA +1

Healthcare Electrocardiography Multimodal Large Language Models Multimodal Learning

Dec 26, 2025cs.CL

Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions

Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evaluation of their safety boundary maintenance under real-world caregiver pressure remains scarce. We evaluated PediatricSafetyBench-v2, a benchmark of 600 pediatrics health queries comprising 300 authentic caregiver queries sourced from the HealthCareMagic-100k-en physician consultation corpus and 300 matched adversarial variants incorporating six operationalized caregiver pressure patterns, across four consumer AI systems (GPT-4o-mini, Gemini-2.0-Flash, Claude-3.5-Haiku, and Llama-3.1-8B). Safety boundary maintenance was assessed using a validated five-component Safety Composite Score (maximum 15 points; safety-appropriate threshold of 10 or above), validated against independent human raters prior to full-corpus application (mean weighted kappa 0.76; Pearson r = 0.88). The overall safety-appropriate rate was 95.5%. Safety-oriented system prompt deployment improved safety-appropriate rates by 5.9 percentage points across all four models. Counter-intuitively, adversarial caregiver pressure was associated with higher rather than lower Safety Composite Score values for all four models across all ten topic categories and severity levels. False expertise claims were the most vulnerability-inducing pressure pattern, whereas emotional escalation was associated with the highest scores. Consumer AI systems maintain safety boundaries in the large majority of pediatrics health interactions. PediatricSafetyBench-v2 is publicly released for longitudinal safety monitoring.

Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi +2

Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. · Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran. +2

Healthcare LLM Safety Medical QA Adversarial Robustness of Language Models LLM Safety Evaluation

Dec 25, 2025cs.CL

Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data

Online medical consultations contain sensitive health information whose privacy implications depend not only on the entities mentioned but also on how those entities are described in context. Existing classification and grading approaches often map health-information entities directly to predefined sensitivity levels, potentially overlooking whether a condition is confirmed, suspected, negated, hypothetical, or merely planned for investigation. In this study, we formulate sensitive-information grading in online medical dialogues as a context-aware evaluation task. We develop a standard-informed operational framework that incorporates assertion status, experiencer, test-result status, and information granularity. We further design a naturalistic evaluation setting together with contrastive cases that minimally alter negation, uncertainty, experiencer, or granularity, and compare large language models under mention-only and full-context conditions. The study aims to quantify the contribution of contextual information to sensitivity grading and to characterize safety-critical over- and under-grading errors. Our framework provides a reproducible basis for evaluating whether LLMs can distinguish sensitive entity mentions from contextually established sensitive disclosures.

Yiwei Yan, Guanfeng Liu

School of Computing, Macquarie University, Australia

Healthcare Text Classification LLM Safety Evaluation Clinical Language Model Evaluation

Dec 16, 2025cs.AI

Ophiuchus: Incentivizing Tool-augmented "Think with Images" for Joint Medical Segmentation, Understanding and Reasoning

Recent medical MLLMs have made significant progress in generating step-by-step textual reasoning chains. However, they still struggle with complex clinical tasks that necessitate dynamic and iterative focusing on fine-grained visual regions. To close this gap, we introduce Ophiuchus, a versatile, tool-augmented framework that equips an MLLM to (i) decide when fine-grained visual evidence is needed, (ii) determine where to probe and ground within the medical image, and (iii) seamlessly weave the relevant sub-image content back into an interleaved, multimodal chain of thought for precise segmentation and diagnosis. Ophiuchus moves beyond mere tool-calling by tightly fusing the MLLM's inherent grounding and reasoning capabilities with external tools, enabling more accurate and trustworthy decisions. The core of our method is a three-stage training strategy: cold-start SFT for basic tool selection; self-reflection fine-tuning to strengthen decision revision; and agentic tool reinforcement learning to elicit sophisticated, expert-like diagnostic behaviors. Extensive experiments show that Ophiuchus consistently outperforms both closed-source and open-source SOTA methods across diverse medical benchmarks, including VQA, detection, and reasoning-based segmentation. Our project code is available at https://github.com/SII-zyj/Ophiuchus.

Yankai Jiang, Yujie Zhang, Peng Zhang +5

Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute +5

Image Segmentation Tool Use in VLMs Healthcare Medical VQA Medical VLMs Medical Image Segmentation+2

Dec 8, 2025cs.LG

CLARITY: Medical World Model for Guiding Treatment Decisions by Modeling Context-Aware Disease Trajectories in Latent Space

Clinical decision-making in oncology requires predicting dynamic disease evolution, a task current static AI predictors cannot perform. While world models (WMs) offer a paradigm for generative prediction, existing medical applications remain limited. Existing methods often rely on stochastic diffusion models, focusing on visual reconstruction rather than causal, physiological transitions. Furthermore, in medical domain, models like MeWM typically ignore patient-specific temporal and clinical contexts and lack a feedback mechanism to link predictions to treatment decisions. To address these gaps, we introduce CLARITY, a medical world model that forecasts disease evolution directly within a structured latent space. It explicitly integrates time intervals (temporal context) and patient-specific data (clinical context) to model treatment-conditioned progression as a smooth, interpretable trajectory, and thus generate physiologically faithful, individualized treatment plans. Finally, CLARITY introduces a novel prediction-to-decision framework, translating latent rollouts into transparent, actionable recommendations. CLARITY demonstrates state-of-the-art performance in treatment planning. On the MU-Glioma-Post dataset, our approach outperforms recent MeWM by 12%, and significantly surpasses all other medical-specific large language models.

Tianxingjian Ding, Yuanhao Zou, Chen Chen +2

Institute of Artificial Intelligence, University of Central Florida

Healthcare Clinical Decision Support Clinical Decision-Making Oncology Latent World Models

Dec 6, 2025cs.HC

JEEVHITAA -- An HCAI Ecosystem to Support Collective Care

Current mobile health platforms are predominantly individual-centric and lack the support for coordinated, auditable multi-actor workflows. However, in many settings worldwide, health decisions are enacted through multi-actor coordination rather than individual users. We present JEEVHITAA, a cross-platform mobile system enabling role-aware sharing and verifiable information flows within permissioned care circles. JEEVHITAA ingests platform and device data, builds layered profiles from sensors and tiered onboarding, and enforces fine-grained, time-bounded access control across care graphs. Data stays secure both within the application and the cloud. Integrated retrieval-augmented Large Language Models produce structured, role-targeted summaries and action plans, offer evidence-grounded verification with provenance and confidence scores, and support advanced insights on reports. We describe the architecture, connector abstractions, and security primitives, and report robustness evaluations using synthetic, ontology-driven data and findings from a feasibility study with real-life care circles across 9-14 weeks. We outline plans for larger multi-site evaluations focusing on operational alignment, longitudinal trust & literacy impact, and relational friction & efforts to sink into the daily infrastructure.

Shyama Sastha Krishnamoorthy Srinivasan, Harsh Pala, Mohan Kumar +1

IIIT-Delhi New Delhi, India · RIT, NY Rochester, New York, US

Healthcare Clinical Decision Support Human-Centered AI Access Control

Dec 5, 2025cs.CL

Multilingual Medical Reasoning for Question Answering with Large Language Models

Large Language Models (LLMs) with reasoning capabilities have recently demonstrated strong potential in medical Question Answering (QA). Existing approaches are largely English-focused and primarily rely on distillation from general-purpose LLMs, raising concerns about the reliability of their medical knowledge. In this work, we present a method to generate multilingual reasoning traces based on medical knowledge extracted from Wikipedia. We produce 500k traces in English, Italian, and Spanish, using a retrieval-augmented generation approach over medical information from Wikipedia. The traces are generated to solve medical questions drawn from MedQA and MedMCQA, which we extend to Italian and Spanish. We test our pipeline in both in-domain and out-of-domain settings across Medical QA benchmarks, and demonstrate that our reasoning traces improve performance both when utilized via in-context learning (few-shot) and supervised fine-tuning, yielding state-of-the-art results among 8B-parameter LLMs. We believe that these resources can support the development of more transparent clinical decision-support tools in multilingual settings. We release the full suite of resources: reasoning traces, translated QA datasets, Medical-Wikipedia, and fine-tuned models.

Pietro Ferrazzi, Aitor Soroa, Rodrigo Agerri

Fondazione Bruno Kessler, Trento, Italy · HiTZ Center - Ixa, University of the Basque Country EHU, Spain

Knowledge Augmentation for Language Models Healthcare Medical QA Multilingual QA Medical Reasoning Medical LLMs

Dec 3, 2025cs.CV

6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models

Vision-language models (VLMs) are increasingly integrated into clinical workflows. However, existing benchmarks primarily assess performance on common anatomical presentations and fail to capture the challenges posed by rare variants. To address this gap, we introduce AdversarialAnatomyBench, the first benchmark comprising naturally occurring rare anatomical variants across diverse imaging modalities and anatomical regions. We call such variants that violate learned priors about "typical" human anatomy natural adversarial anatomy. Benchmarking 25 state-of-the-art VLMs with AdversarialAnatomyBench yielded three key insights. First, when queried with basic medical perception tasks, mean accuracy dropped from 71% on typical to 28% on atypical anatomy. Even the best-performing models, GPT-5, Gemini 2.5 Pro, and Llama 4 Maverick, showed performance drops of 41-51%. Second, model errors closely mirrored expected anatomical biases. Third, neither model scaling nor interventions, including bias-aware prompting and test-time reasoning, resolved these issues. These findings highlight a critical limitation in current VLMs: their poor generalization to rare anatomical presentations. AdversarialAnatomyBench provides a foundation for systematically measuring and mitigating anatomical bias in multimodal medical artificial intelligence (AI) systems.

Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6

German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany. · Medical Faculty, Heidelberg University, Germany. · Faculty of Mathematics and Computer Science, Heidelberg University, Germany. +9

VLM Evaluation VLM Robustness Healthcare OOD Generalization Medical VLMs Medical Image Benchmarks

Dec 1, 2025cs.CY

First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations

Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized. We present NOHARM (Numerous Options Harm Assessment for Risk in Medicine), a 1,100-task benchmark of primary care-to-specialist consultation cases to measure the frequency and severity of potentially harmful errors from LLM-generated medical consultation recommendations. NOHARM covers 10 specialties, with 12,747 expert annotations for 4,249 clinical management options. Across 20 notable LLMs and 4 widely used retrieval-augmented generation (RAG) clinical AI tools, direct application of recommendations carried potential for severe harm in up to 24.6% of cases, with errors of omission accounting for more than 80% of severe errors. Harm potential was not uniform across systems, with clinical AI tools outperforming generalist LLMs, and multi-agent AI teaming further improving performance in generalist models. In a randomized study of 101 U.S.-licensed generalist physicians, AI assistance improved physician performance compared to conventional resources. However, AI-assisted physicians frequently omitted valuable AI-generated recommendations and still scored lower than many AI systems alone. Had those recommendations been incorporated, combined human-AI responses would have outperformed both the human and AI system as used, suggesting complementary strengths and unrealized potential in human-AI teaming. Collectively, these results show that despite strong performance on medical knowledge benchmarks, widely used AI tools can produce medical consultation advice with the potential for severe harm, and highlight the need for explicit measurement of clinical safety. The benchmark and leaderboard are publicly available to support ongoing evaluation and improvement of AI systems used for clinical care.

David Wu, Fateme Nateghi Haredasht, Saloni Kumar Maharaj +54

Healthcare Clinical Decision Support Human-in-the-Loop AI AI Safety Evaluation Clinical Language Model Evaluation

Nov 30, 2025cs.CV

Generalised Medical Phrase Grounding

Medical phrase grounding (MPG) maps textual descriptions of radiological findings to corresponding image regions. These grounded reports are easier to interpret, especially for non-experts. Existing MPG systems mostly follow the referring expression comprehension (REC) paradigm and return exactly one bounding box per phrase. Real reports often violate this assumption. They contain multi-region findings, non-diagnostic text, and non-groundable phrases, such as negations or descriptions of normal anatomy. Motivated by this, we reformulate the task as generalised medical phrase grounding (GMPG), where each sentence is mapped to zero, one, or multiple scored regions. To realise this formulation, we introduce the first GMPG model: MedGrounder. We adopted a two-stage training regime: pre-training on report sentence--anatomy box alignment datasets and fine-tuning on report sentence--human annotated box datasets. Experiments on PadChest-GR and MS-CXR show that MedGrounder achieves strong zero-shot transfer and outperforms REC-style and grounded report generation baselines on multi-region and non-groundable phrases, while using far fewer human box annotations. Finally, we show that MedGrounder can be composed with existing report generators to produce grounded reports without retraining the generator.

Wenjun Zhang, Shekhar S. Chandra, Aaron Nicolson

The University of Queensland, Brisbane, Australia · Australian e-Health Research Centre, CSIRO Health and Biosecurity, Australia

Radiology VLMs Healthcare Medical Image Grounding

Nov 28, 2025cs.LG

Auditable Context-Aware HFMD Forecasting with Structured LLM Agents

Effective HFMD surveillance requires forecasts capturing both time-series patterns and contextual drivers such as school calendars, weather, and policy or surveillance reports. In clinical settings, forecasts must be trusted and actionable; thus, beyond point accuracy, decision-makers require concise, auditable explanations of why risk is expected to rise or fall. Classical models (e.g., ARIMA and Prophet) and foundation models (e.g., Chronos, Moirai, and TimesFM) treat external covariates as numerical inputs, lacking semantic reasoning to reflect epidemiological mechanisms or resolve conflicting signals. We propose a two-agent neuro-symbolic framework that decouples contextual interpretation from probabilistic forecasting. An LLM-based Event Interpreter ingests heterogeneous signals -- school schedules, weather summaries, government reports, and clinical guidelines -- and outputs a scalar transmission-impact signal. A Forecast Generator combines this signal with historical case counts to produce point forecasts that are mapped to probabilistic predictions through Poisson/negative-binomial moment matching. We focus on one-week-ahead rolling forecasts, aligning with weekly hospital-capacity planning and the rapid, context-driven inflections typical of HFMD. We evaluate on two datasets: Hong Kong surveillance (90 target weeks in 2023--2024) and Lishui hospital visits (33 target weeks in 2024). Against traditional and foundation-model baselines, our approach achieves competitive point accuracy while providing robust 90% intervals (coverage approximately 0.85--1.00) and concise rationales. This demonstrates that integrating domain knowledge through LLM-based agents can match strong numerical forecasters while yielding interpretable, context-aware forecasts aligned with public-health decision-making.

Joongwon Chae, Runming Wang, Chen Xiong +5

Institute of Biomedicine and Health Engineering, Tsinghua Shenzhen International Graduate School (SIGS), Tsinghua University, Shenzhen, China · The First Hospital of Hebei Medical University, Shijiazhuang, China · The Fifth Affiliated Hospital of Wenzhou Medical University, Lishui 323000, China +1

Epidemic Forecasting Explainable Time Series Forecasting Healthcare Time Series Forecasting LLM-Based Time Series Forecasting Multi-Agent LLMs+1

Nov 20, 2025cs.AI

MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support

Clinical language models often assign high confidence to incorrect predictions, particularly in high-severity and out-of-distribution cases. We present MedBayes-Lite, a retraining-free uncertainty governance layer for transformer-based clinical predictors. It combines Monte Carlo dropout, predictive calibration, and confidence-guided abstention to defer low-confidence predictions for human review, adding no trainable parameters. Evaluated on MedMCQA and MedQA-USMLE, MedBayes-Lite reduces expected calibration error by 0.23 to 0.33 and drives harmful overconfident errors (confident, incorrect, high-severity predictions) toward zero. Under domain shift from MedMCQA to MedQA-USMLE, it reduces confident high-severity errors from about 21% to near zero while roughly halving calibration drift. We also introduce the Clinical Uncertainty Score (CUS), which strongly correlates with harmful overconfidence (r approximately 0.88). Although the framework does not improve risk-coverage ranking, and temperature scaling or deep ensembles may provide advantages in calibration cost or risk ranking, MedBayes-Lite offers a practical calibration-and-abstention layer that reduces confident high-severity errors in clinical question-answering benchmarks.

Elias Hossain, Md Mehedi Hasan Nipu, Maleeha Sheikh +5

College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, USA · Department of Computer Science and Engineering, North South University, Dhaka 1229, Bangladesh · Department of Electrical and Computer Engineering, Purdue University Fort Wayne, Fort Wayne, IN 46805, USA +4

Selective Prediction Healthcare Clinical Decision Support LLM Uncertainty Estimation Medical LLMs Uncertainty Calibration

Nov 12, 2025cs.CY

Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

Generative AI chatbots built for mental health could extend access to care, but evidence from real-world use is limited. We report a single-arm, naturalistic pilot of a foundation model trained for mental health, among 299 US adults with at least moderate depressive or anxiety symptoms who were followed for up to 12 months. Depression and anxiety symptoms fell by 10 weeks (Cohen's d 0.93 and 0.79), with loneliness, behavioral activation, and social interaction improving. Clinicians confirmed that automated safeguards were escalated appropriately. Participants fell into non-responding (57.2%), improving (37.1%) and rapidly improving (5.7%) trajectories. Early working alliance and greater engagement were associated with better outcomes. The AI deployed ten identified intervention families whose delivery varied with baseline anxiety and depression, with the overall ratio of clinical to non-clinical content increasing according to severity without survey information access. These findings support the feasibility of purpose-built AI for mental health.

Thomas D. Hull, Lizhe Zhang, Caitlin A. Stamatis +2

Slingshot AI · CREATIV Lab, Department of Psychiatry and Behavioral Sciences, University of Washington · Department of Psychiatry, NYU School of Medicine +1

Healthcare Mental Health Conversational Agents Generative AI Evaluation

Nov 7, 2025cs.CV

Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment

Molecular biomarker testing in pathology is often costly and tissue-consuming, limiting scalable clinical deployment. Artificial intelligence applied to hematoxylin and eosin (HE)-stained histology could enable rapid biomarker screening, but clinical translation requires models that are both accurate and interpretable. Here we introduce Hireca, a biomarker-focused pathology foundation model pretrained on more than 80,000 whole-slide images spanning 38 organ types from three medical centers, together with CytoMap, an interpretability module that localizes cellular-scale evidence underlying predictions. Across 10 biomarker tasks encompassing morphological, molecular, genetic, and spatial-transcriptomic-proxy readouts, Hireca ranked first in five tasks and outperformed comparable models overall. In evaluation by eight pathologists from two countries, CytoMap was consistently preferred over alternative visualization approaches and revealed error patterns in difficult cases. These results position Hireca and CytoMap as a transparent framework for clinically reviewable biomarker assessment directly from routine HE histology.

Jingsong Liu, Han Li, Zhengyang Xu +19

Institute of Pathology, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Munich Data Science Institute (MDSI), Munich, Germany +6

Computational Pathology Healthcare Pathology Foundation Models Explainable Medical Image Analysis Whole-Slide Image Analysis

Nov 4, 2025cs.CL

AyurParam: A State-of-the-Art Bilingual Language Model for Ayurveda

Current large language models excel at broad, general-purpose tasks, but consistently underperform when exposed to highly specialized domains that require deep cultural, linguistic, and subject-matter expertise. In particular, traditional medical systems such as Ayurveda embody centuries of nuanced textual and clinical knowledge that mainstream LLMs fail to accurately interpret or apply. We introduce AyurParam-2.9B, a domain-specialized, bilingual language model fine-tuned from Param-1-2.9B using an extensive, expertly curated Ayurveda dataset spanning classical texts and clinical guidance. AyurParam's dataset incorporates context-aware, reasoning, and objective-style Q&A in both English and Hindi, with rigorous annotation protocols for factual precision and instructional clarity. Benchmarked on BhashaBench-Ayur, AyurParam not only surpasses all open-source instruction-tuned models in its size class (1.5--3B parameters), but also demonstrates competitive or superior performance compared to much larger models. The results from AyurParam highlight the necessity for authentic domain adaptation and high-quality supervision in delivering reliable, culturally congruent AI for specialized medical knowledge.

Sravan Gorugantu, Mohd Nauman, Vijay Devane +7

BharatGen Team

Medical LLMs Healthcare Domain Adaptation for LLMs LLM Fine-Tuning Multilingual Language Models

Nov 3, 2025cs.MA

An Explanation-oriented Inquiry Dialogue Game for Expert Collaborative Recommendations

This work presents a requirement analysis for collaborative dialogues among medical experts and an inquiry dialogue game based on this analysis for incorporating explainability into multiagent system design. The game allows experts with different knowledge bases to collaboratively make recommendations while generating rich traces of the reasoning process through combining explanation-based illocutionary forces in an inquiry dialogue. The dialogue game was implemented as a prototype web-application and evaluated against the specification through a formative user study. The user study confirms that the dialogue game meets the needs for collaboration among medical experts. It also provides insights on the real-life value of dialogue-based communication tools for the medical community.

Qurat-ul-ain Shaheen, Katarzyna Budzynska, Carles Sierra

IIIA, CSIC, Barcelona, Spain · Software Competence Centre Hagenberg, Hagenberg, Austria · Laboratory of The New Ethos, Faculty of Administration and Social Sciences, Warsaw University of Technology, Warsaw, Poland

Explainable Artificial Intelligence Healthcare Multi-Agent Collaboration Clinical Decision-Making Multi-Agent Communication

Nov 1, 2025cs.CL

MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts

Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains under-evaluated, particularly beyond English. We introduce MedRECT, a bilingual benchmark for Japanese and English that formulates medical error handling as three subtasks: error detection, error sentence extraction, and error correction. MedRECT-ja contains 663 samples derived from the Japanese Medical Licensing Examinations, while the separately sourced MedRECT-en contains 458 samples curated from MEDEC. We evaluate 11 LLMs across 17 configurations that cover proprietary and open-weight models, medical-domain specialization, and multiple reasoning settings. Qwen3-32B scores higher in its thinking mode than in its non-thinking mode on error detection F1 and sentence extraction accuracy in both subsets, with sentence extraction accuracy higher by 24.5 percentage points on MedRECT-ja and 10.3 on MedRECT-en. Several leading general-purpose reasoning models outperform all three evaluated medical-domain models on these two subtasks. Most models have lower point estimates on the Japanese subset, although absolute scores are not directly comparable because the subsets differ in source material and error distributions. LoRA fine-tuning yields higher sentence extraction accuracy and higher point estimates on all three reference-based correction similarity metrics in both languages. MedRECT provides an open, reusable evaluation resource for studying medical error correction and reasoning across Japanese and English. Our dataset and code are available at https://github.com/pfnet-research/medrect.

Naoto Iwase, Hiroki Okuyama, Junichiro Iwasawa

Preferred Networks, Inc., Tokyo, Japan · School of Medicine, Nagoya University, Nagoya, Japan

Multilingual Language Model Evaluation Healthcare Clinical Language Model Evaluation Clinical NLP Medical LLMs

Oct 29, 2025eess.IV

MedForj: An open, large-scale foundational generative prior for high-resolution 3D brain MRI

This work introduces MedForj, a suite of 3D foundational generative priors based on diffusion models. The MedForj models were trained on

72{,}659

1~mm isotropic 3D

T_1

-weighted MRI human brain image volumes from

38{,}174

subjects, drawn from a curated corpus of

80{,}675

volumes from

42{,}506

subjects spanning

38

publicly available datasets. These training images were manually inspected to exclude those with poor quality and excessive pathology, and otherwise were minimally processed. The models include six different diffusion training strategies: rectified flow, latent diffusion rectified flow, flow matching, velocity prediction, clean prediction, and noise prediction. Image samples produced by each of these models were compared to each other and against real, ground truth data under downstream segmentation distributions, FID, five inverse problems, and blind human inspection in an observer study. Flow matching was the strongest strategy overall, achieving the best inverse problem solving results at

28.80

~dB PSNR and

0.874

SSIM averaged over the five forward problems, the highest rate of reconstructions judged real by blind human raters at

72.6\%

, and the closest per-structure match to real segmented anatomy in a permutation test. It was not best everywhere: rectified flow produced the most convincing unconditional samples in the observer study and the best FID, and the latent rectified-flow model achieved the smallest joint distributional distance to real anatomy. No other strategy, however, performed consistently well across all four evaluations. We therefore recommend flow matching as the default MedForj prior, while releasing every strategy so that the choice can be revisited per application. All model weights and corresponding code are publicly available at https://github.com/piksl-research/medforj.

Samuel W. Remedios, Aaron Carass, Jerry L. Prince +1

Image Analysis and Communications Laboratory, Johns Hopkins University · Department of Neurology, Johns Hopkins Hospital

Flow Matching Healthcare 3D Medical Imaging Medical Image Synthesis Diffusion-Based Inverse Problems 3D Diffusion Models+1

Oct 20, 2025cs.CL

Zoom In Disparities in Healthcare LLM Q&A

Equitable access to reliable health information is vital when integrating AI into healthcare. Yet, information quality varies across languages, raising concerns about the reliability and consistency of multilingual Large Language Models (LLMs). We systematically examine cross-lingual disparities in pre-training source and factuality alignment in LLM answers for multilingual healthcare Q&A across English, German, Turkish, Chinese (Mandarin), and Italian. We (i) constructed Multilingual Wiki Health Care (MultiWikiHealthCare), a multilingual dataset from Wikipedia; (ii) analyzed cross-lingual healthcare coverage; (iii) assessed LLM response alignment with these references; and (iv) conducted a case study on factual alignment through the use of contextual information and Retrieval-Augmented Generation (RAG). Our findings reveal substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment. Across LLMs, responses align more with English Wikipedia, even when the prompts are non-English. Providing contextual excerpts from non-English Wikipedia at inference time effectively shifts factual alignment toward culturally relevant knowledge. These results highlight practical pathways for building more equitable, multilingual AI systems for healthcare.

Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao +5

Universitat Politècnica de València, Valencia, Spain · University of Trento, Trento, Italy · University of Sheffield, Sheffield, UK +5

Multilingual Language Model Evaluation Knowledge Augmentation for Language Models Healthcare Multilingual QA Factual Knowledge in Language Models Clinical Language Model Evaluation

Oct 5, 2025cs.AI

A global log for medical AI

Modern computer systems rely on syslog, a universal protocol that records critical events across heterogeneous infrastructure. Medicine's rapidly growing AI stack has no equivalent. As medicine deploys AI tools at scale, there is no standard way to record how, when, by whom, and for whom these models are used. Without such records, it is difficult to measure real-world performance and outcomes, detect adverse events, or identify bias and dataset drift. Here we introduce MedLog, a protocol for event-level logging of medical AI. Each time an AI model interacts with a human, another algorithm, or an automated workflow, MedLog creates a record. Each record contains nine core fields: header, model, user, target, inputs, artifacts, outputs, outcomes, and feedback. We apply MedLog across four deployments in the US, Switzerland, and Vietnam: ICU deterioration prediction, tetanus progression monitoring from wearable signals, automated sepsis quality reporting, and patient attendance prediction. MedLog records capture model behavior, workflow interactions, and downstream outcomes, including AI performance degradation during severe weather events in patient attendance prediction and increased laboratory testing after ICU deterioration alerts. MedLog limits the data footprint through risk-based sampling, lifecycle-aware retention policies, and write-behind caching, enabling deployment in low-resource settings. It also supports detailed traces for complex, agentic, or multi-stage workflows, creating a foundation for continuous monitoring, auditing, and improvement of medical AI.

Ayush Noori, Aaron E. Boussina, Hai Ho Bich +48

Algorithmic Auditing Healthcare

Sep 24, 2025cs.LG

Revisiting Performance Claims for Chest X-Ray Models Using Clinical Context

Public datasets of Chest X-Rays (CXRs) have long been a popular benchmark for developing machine learning (ML) computer vision models in healthcare. However, the reported strong average-case performance of these models do not necessarily reflect their actual utility when used in heterogeneous clinical settings, potentially masking weaker performance in medically significant scenarios. In this work we use clinical context to provide a more holistic evaluation of models for CXR diagnosis. In particular, we use discharge summaries, recorded prior to each CXR, to derive a ``pre-CXR'' probability of each CXR label, as a proxy for existing contextual knowledge available to clinicians when interpreting CXRs. We use this measure to probe model performance along two dimensions: First, using a stratified analysis, we show that models tend to have lower performance (as measured by AUROC and other metrics) among individuals with higher pre-CXR probability. Second, by controlling for pre-CXR probability via matching and re-weighting, we demonstrate that performance degrades when the correlation is broken between prior context and the current CXR label, suggesting that model performance is highly sensitive to the underlying distribution of clinical context. Specifically, cases with high pre-test probabilities present a fundamentally more difficult visual classification task, highlighting a gap in clinical utility when models are applied to high-risk cohorts.

Andrew Wang, Jiashuo Zhang, Michael Oberst

Brown University · Johns Hopkins University

Chest X-Ray Classification Distribution Shift Healthcare Medical Image Benchmarks

Sep 13, 2025cs.CL

Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies

In high-stakes domains such as healthcare, privacy concerns severely limit the use of real-world training data. Differentially private (DP) synthetic data offers a promising alternative with formal privacy guarantees, but achieving strong utility remains challenging for clinical note generation due to domain specificity and long-form text complexity. We present Term2Note, a method for synthesising full-length clinical notes under DP constraints. By structurally separating content and form, Term2Note generates section-wise note content conditioned on medical terms, with terms and notes privatised under separate DP constraints, and applies a DP quality maximiser to improve outputs. Experiments demonstrate that Term2Note produces synthetic notes with statistical properties closely aligned with real clinical notes, and that downstream models trained on these notes achieve performance comparable to those trained on real clinical data. Compared to existing DP text generation baselines, Term2Note substantially improves both fidelity and utility, without relying on label distribution assumptions, highlighting its effectiveness as a practical privacy-preserving alternative to real clinical notes.

Yuping Wu, Viktor Schlegel, Warren Del-Pinto +11

Univeristy of Manchester · Imperial College London, Imperial Global Singapore · Khoo Teck Puat Hospital, Singapore +2

Medical Report Generation Healthcare Synthetic Data Generation Privacy-Preserving ML Differential Privacy Conditional Generative Modeling

Sep 9, 2025cs.LG

General Demographic Pre-trained Models for Enhancing Predictive Performance Across Diseases and Population

Foundation models for healthcare require balancing robust generalization across heterogeneous clinical populations and disease settings with the architectural simplicity needed for deployment. We present a pre-trained model focused on demographic attributes that enhances feature utility across medical domains in a plug-and-play fashion. We introduce the General Demographic Pre-trained (GDP) model, designed to extract intrinsic representations of patient status based on age and sex, the two most ubiquitous clinical features. The composition of GDP was optimized by investigating various encoding methods and visit-reordering schemes. The model was pre-trained and transferability was validated by embedding the learned representations into diverse disease and geographic cohorts characterized by distinct demographic profiles. The optimal model configuration was subsequently validated against top-performing tabular foundation models (TabPFN, TabICL, and TabFM). Our findings demonstrate that concatenating GDP-derived embeddings with raw residual features consistently enhances predictive performance across classification tasks while elevating the relative importance of demographic attributes. The embedding transformation provides superior representational separability compared to the original data distribution, yielding competitive discrimination performance across metrics against all three general-purpose foundation models and tree-based algorithm. GDP has successfully served the purpose of a foundation model, which produce enriched representations that amplify the predictive insight of these features beyond their raw form. The generated embeddings can be directly concatenated with residual features, serving as an enhancement layer that maintains full compatibility with standard tabular classifiers.

Li-Chin Chen, Ji-Tian Sheu, Yuh-Jue Chuang

Data Analytics and Digital Transformation Research Center National Taiwan University Taipei, Taiwan · Department of Health Care Management Chang Gung University Taoyuan, Taiwan

Healthcare Representation Learning Clinical Prediction Tabular Foundation Models