Clinical Language Model Evaluation

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 118% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 259

Apr 16, 2026cs.CL

Continued Pretraining of FinBERT on Finnish Histopathological Reports: Train-Time Signals and Proxy Downstream Correlations

In Natural Language Processing (NLP) classification tasks where a lack of labeled data is an issue, continued pretraining (CPT) of transformer models on unlabeled data is an established approach. In this paper, we have two aims. (1) We describe our observations from continued pretraining of the Finnish BERT transformer model (FinBERT) on a Finnish histopathological dataset (below, \emph{the Histopathology data}). (2) Since the Histopathology data has no classification labels, we gather public Finnish datasets as proxy data to analyze whether the signals observed in (1) are associated with downstream classification gains. We observe that CPT train-time loss curves differ strongly by domain, and that, in an exploratory analysis, certain CPT-derived features correlate with proxy classification improvement. In particular, this report contributes to the limited literature on NLP for Finnish healthcare data.

Rami Luisto, Liisa Petäinen, Tommi Grönholm +5

Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland. · Digital Workforce Services, Helsinki, Finland. · Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland. +2

Healthcare Domain-Adaptive Pretraining Continued Pretraining Clinical Language Model Evaluation Clinical NLP

Apr 9, 2026cs.AI

IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models

A strongly safety-trained model will provide a doctor with a benzodiazepine taper schedule, but not a patient who asks for one. The model knows the information, but how much it shares depends on the framing. We introduce IatroBench, a benchmark that evaluates models on two axes of harm (commission and omission) across 60 pre-registered clinical scenarios and 6 models. We use Claude Opus 4.6 to score model responses against a rubric written by a physician, and find that its omission scores are as well-aligned to the physician's scores as another physician's scores are. We find that when the same case is presented as a patient query and a doctor consultation (the variants also differ in register, request and the supervision a treating physician implies), all five models we test share more information with the doctor than the patient. We term this phenomenon "framing-contingent withholding." We find a mean decoupling gap of +0.38 across models (p = 0.003), and of +0.22 under an independent LLM judge (95% CI 0.10-0.36, p = 0.0014). An evaluation that focuses solely on commission harms would consider all of these cases as equally cautious refusals, but closer investigation reveals three different patterns: Claude Opus withholds information from the patient that it demonstrates knowledge of in the doctor framing. Llama 4 does poorly in both framings, so the decoupling gap cannot distinguish information withholding from incompetence. We are forced to exclude GPT-5.2 from this analysis because it returns no text for 33.2% of doctor responses, but 0% of layperson responses. A standard LLM judge rates responses as having zero omission harm in 86.6% of cases where our structured evaluations score them as omission harms. (Because our scenarios are designed to induce tension between safety and helpfulness, these statistics should be taken as only applying to this distribution.)

David Gringras

Harvard T.H. Chan School of Public Health

LLM Safety Benchmarks Healthcare LLM Safety Evaluation Clinical Language Model Evaluation

Mar 31, 2026cs.CL

Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations

Large language models (LLMs) are increasingly used for medical consultation and health information support, where safety depends not only on medical knowledge but also on robust responses to unclear, inconsistent, or misleading patient input. However, most existing medical LLM evaluations assume idealized and well-posed patient questions, limiting their realism. We study challenging patient behaviors that commonly arise in real medical consultations and complicate safe clinical reasoning. We define four clinically grounded categories of such behaviors: information contradiction, factual inaccuracy, self-diagnosis, and care resistance. For each behavior, we specify concrete failure criteria that capture unsafe responses. Building on four existing medical dialogue datasets, we introduce CPB-Bench (Challenging Patient Behaviors Benchmark), a bilingual (English and Chinese) benchmark of multi-turn dialogues annotated for these behaviors. We find that although models perform well overall, they exhibit consistent behavior-specific failures, especially when handling contradictory or medically implausible patient information. We further evaluate four intervention strategies and find inconsistent improvements, with some interventions introducing unnecessary corrections.

Yahan Li, Xinyi Jie, Wanjia Ruan +5

University of Southern California

Healthcare Clinical Reasoning LLM Safety Evaluation Clinical Language Model Evaluation Language Model Robustness Medical LLMs

Jan 25, 2026cs.AI

Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context

Safety alignment in Large Language Models is critical for healthcare; however, reliance on binary refusal boundaries often results in over-refusal of benign queries or unsafe compliance with harmful ones. While existing benchmarks measure these extremes, they fail to evaluate Safe Completion: the model's ability to maximise helpfulness on dual-use or borderline queries by providing safe, high-level guidance without crossing into actionable harm. We introduce Health-ORSC-Bench, the first large-scale benchmark designed to systematically measure Over-Refusal and Safe Completion quality in healthcare. Comprising 31,920 benign boundary prompts across seven health categories (e.g., self-harm, medical misinformation), our framework uses an automated pipeline with human validation to test models at varying levels of intent ambiguity. We evaluate 30 state-of-the-art LLMs, including GPT-5 and Claude-4, revealing a significant tension: safety-optimised models frequently refuse up to 80% of "Hard" benign prompts, while domain-specific models often sacrifice safety for utility. Our findings demonstrate that model family and size significantly influence calibration: larger frontier models (e.g., GPT-5, Llama-4) exhibit "safety-pessimism" and higher over-refusal than smaller or MoE-based counterparts (e.g., Qwen-3-Next), highlighting that current LLMs struggle to balance refusal and compliance. Health-ORSC-Bench provides a rigorous standard for calibrating the next generation of medical AI assistants toward nuanced, safe, and helpful completions. Furthermore, our benchmark facilitates reproducible evaluation, encourages safety calibration, and supports development of clinically reliable, context-aware, human-aligned medical AI systems. Our code and data are available at: https://github.com/ZhihaoZhang97/Health-ORSC-Bench. Warning: Some contents may include toxic or undesired contents.

Zhihao Zhang, Liting Huang, Guanghao Wu +3

Macquarie University1 · UTS2 · MBZUAI3 +1

LLM Safety Benchmarks Language Model Safety Evaluation Healthcare LLM Refusal LLM Refusal Behavior Over-Refusal+2

Jan 23, 2026cs.AI

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

Large language models (LLMs) deployed in clinical decision support may acquiesce to patient requests for care that conflicts with evidence-based guidelines. We developed SycoEval-EM, a multi-agent simulation framework to evaluate LLM robustness to adversarial patient persuasion in emergency medicine. Across 19 contemporary LLMs and 1,425 simulated clinical encounters spanning three Choosing Wisely scenarios, acquiescence rates ranged from 0% to 100%, revealing a bimodal distribution. Seven models maintained near-perfect guideline adherence, while six acquiesced in the majority of encounters. Vulnerability varied substantially across clinical scenarios. Acquiescence was highest for CT imaging requests, intermediate for antibiotic prescriptions for sinusitis, and lowest for opioid prescriptions for acute back pain. Model scale, recency, and performance on static medical benchmarks did not consistently predict robustness. All five persuasion tactics produced similar acquiescence rates, with no statistically significant differences after correction for multiple comparisons, suggesting a generalized susceptibility rather than tactic-specific weaknesses. LLM-as-judge evaluation was validated against two independent physician raters across 95 matched conversations and demonstrated near-perfect agreement for the primary outcome of acquiescence (Cohens kappa = 0.957). These findings indicate that static medical benchmarks are insufficient to predict safety performance under sustained social pressure and support incorporating multi-turn adversarial testing into clinical AI evaluation. Notably, two models achieved perfect guideline adherence across all encounters, demonstrating that robustness to patient pressure is attainable without sacrificing effective clinical communication.

Dongshen Peng, Yi Wang, Austin Schoeffler +5

UNC Chapel Hill · University of Waterloo · Stanford University

LLM Sycophancy Language Model Safety Evaluation Healthcare Adversarial Robustness of Language Models Clinical Language Model Evaluation Medical LLMs

Jan 6, 2026cs.CL

MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence

Trustworthy clinical summarization requires every claim to be traceable to its evidence, yet existing attribution often resolves only to the sentence or document, leaving clinicians to scan surrounding text for the few words that matter. We argue that the unit of attribution should match the unit of verification: the precise phrase the reader's eye must land on. We present MAPLE (Medical Aspect-Based Summarization with Phrase-Level Evidence), a human-annotated benchmark that grounds each summarized claim in both cited sentences and contributory phrases within them. Spanning 152 randomized controlled trial (RCT) abstracts and 16 clinically motivated aspects, MAPLE comprises 1,799 aspect-based summaries with two-level evidence. We further introduce a decoupled evaluation framework that separately scores content, traceability, and locatability, together with a proxy for the amount of source text a clinician must inspect to verify a claim. Benchmarking eleven LLMs shows that sentence-level citation is consistently strong (C-F1 up to 90.9%), while phrase-level grounding remains less stable and the most discriminative axis across models (P-F1 66.1-84.5%). These results suggest that the key challenge is not only producing accurate summaries, but localizing their supporting evidence precisely enough for efficient clinical verification. Data and code are available at https://github.com/chubohao/maple.

Bohao Chu, Hendrik Damm, Tabea M. G. Pakull +5

University of Duisburg-Essen · University of Applied Sciences and Arts Dortmund · University Hospital Essen +1

Healthcare Text Summarization Summarization Evaluation Evidence-Grounded Generation Clinical Language Model Evaluation

Dec 25, 2025cs.CL

Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data

Online medical consultations contain sensitive health information whose privacy implications depend not only on the entities mentioned but also on how those entities are described in context. Existing classification and grading approaches often map health-information entities directly to predefined sensitivity levels, potentially overlooking whether a condition is confirmed, suspected, negated, hypothetical, or merely planned for investigation. In this study, we formulate sensitive-information grading in online medical dialogues as a context-aware evaluation task. We develop a standard-informed operational framework that incorporates assertion status, experiencer, test-result status, and information granularity. We further design a naturalistic evaluation setting together with contrastive cases that minimally alter negation, uncertainty, experiencer, or granularity, and compare large language models under mention-only and full-context conditions. The study aims to quantify the contribution of contextual information to sensitivity grading and to characterize safety-critical over- and under-grading errors. Our framework provides a reproducible basis for evaluating whether LLMs can distinguish sensitive entity mentions from contextually established sensitive disclosures.

Yiwei Yan, Guanfeng Liu

School of Computing, Macquarie University, Australia

Healthcare Text Classification LLM Safety Evaluation Clinical Language Model Evaluation

Dec 1, 2025cs.CY

First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations

Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety profiles remain poorly characterized. We present NOHARM (Numerous Options Harm Assessment for Risk in Medicine), a 1,100-task benchmark of primary care-to-specialist consultation cases to measure the frequency and severity of potentially harmful errors from LLM-generated medical consultation recommendations. NOHARM covers 10 specialties, with 12,747 expert annotations for 4,249 clinical management options. Across 20 notable LLMs and 4 widely used retrieval-augmented generation (RAG) clinical AI tools, direct application of recommendations carried potential for severe harm in up to 24.6% of cases, with errors of omission accounting for more than 80% of severe errors. Harm potential was not uniform across systems, with clinical AI tools outperforming generalist LLMs, and multi-agent AI teaming further improving performance in generalist models. In a randomized study of 101 U.S.-licensed generalist physicians, AI assistance improved physician performance compared to conventional resources. However, AI-assisted physicians frequently omitted valuable AI-generated recommendations and still scored lower than many AI systems alone. Had those recommendations been incorporated, combined human-AI responses would have outperformed both the human and AI system as used, suggesting complementary strengths and unrealized potential in human-AI teaming. Collectively, these results show that despite strong performance on medical knowledge benchmarks, widely used AI tools can produce medical consultation advice with the potential for severe harm, and highlight the need for explicit measurement of clinical safety. The benchmark and leaderboard are publicly available to support ongoing evaluation and improvement of AI systems used for clinical care.

David Wu, Fateme Nateghi Haredasht, Saloni Kumar Maharaj +54

Healthcare Clinical Decision Support Human-in-the-Loop AI AI Safety Evaluation Clinical Language Model Evaluation

Nov 1, 2025cs.CL

MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts

Large language models (LLMs) show promise in medical applications, but their ability to detect and correct errors in clinical texts remains under-evaluated, particularly beyond English. We introduce MedRECT, a bilingual benchmark for Japanese and English that formulates medical error handling as three subtasks: error detection, error sentence extraction, and error correction. MedRECT-ja contains 663 samples derived from the Japanese Medical Licensing Examinations, while the separately sourced MedRECT-en contains 458 samples curated from MEDEC. We evaluate 11 LLMs across 17 configurations that cover proprietary and open-weight models, medical-domain specialization, and multiple reasoning settings. Qwen3-32B scores higher in its thinking mode than in its non-thinking mode on error detection F1 and sentence extraction accuracy in both subsets, with sentence extraction accuracy higher by 24.5 percentage points on MedRECT-ja and 10.3 on MedRECT-en. Several leading general-purpose reasoning models outperform all three evaluated medical-domain models on these two subtasks. Most models have lower point estimates on the Japanese subset, although absolute scores are not directly comparable because the subsets differ in source material and error distributions. LoRA fine-tuning yields higher sentence extraction accuracy and higher point estimates on all three reference-based correction similarity metrics in both languages. MedRECT provides an open, reusable evaluation resource for studying medical error correction and reasoning across Japanese and English. Our dataset and code are available at https://github.com/pfnet-research/medrect.

Naoto Iwase, Hiroki Okuyama, Junichiro Iwasawa

Preferred Networks, Inc., Tokyo, Japan · School of Medicine, Nagoya University, Nagoya, Japan

Multilingual Language Model Evaluation Healthcare Clinical Language Model Evaluation Clinical NLP Medical LLMs

Oct 20, 2025cs.CL

Zoom In Disparities in Healthcare LLM Q&A

Equitable access to reliable health information is vital when integrating AI into healthcare. Yet, information quality varies across languages, raising concerns about the reliability and consistency of multilingual Large Language Models (LLMs). We systematically examine cross-lingual disparities in pre-training source and factuality alignment in LLM answers for multilingual healthcare Q&A across English, German, Turkish, Chinese (Mandarin), and Italian. We (i) constructed Multilingual Wiki Health Care (MultiWikiHealthCare), a multilingual dataset from Wikipedia; (ii) analyzed cross-lingual healthcare coverage; (iii) assessed LLM response alignment with these references; and (iv) conducted a case study on factual alignment through the use of contextual information and Retrieval-Augmented Generation (RAG). Our findings reveal substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment. Across LLMs, responses align more with English Wikipedia, even when the prompts are non-English. Providing contextual excerpts from non-English Wikipedia at inference time effectively shifts factual alignment toward culturally relevant knowledge. These results highlight practical pathways for building more equitable, multilingual AI systems for healthcare.

Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao +5

Universitat Politècnica de València, Valencia, Spain · University of Trento, Trento, Italy · University of Sheffield, Sheffield, UK +5

Multilingual Language Model Evaluation Knowledge Augmentation for Language Models Healthcare Multilingual QA Factual Knowledge in Language Models Clinical Language Model Evaluation

Aug 5, 2025cs.CL

Somatic in the East, Psychological in the West?: Investigating Clinically-Grounded Cross-Cultural Depression Symptom Expression in LLMs

Prior clinical psychology research shows that Western individuals with depression tend to report psychological symptoms, while Eastern individuals report somatic ones. We test whether Large Language Models (LLMs), which are increasingly used in mental health, reproduce these cultural patterns by prompting them with Western or Eastern personas. Results show that LLMs largely fail to replicate the patterns when prompted in English, though prompting in major Eastern languages (i.e., Chinese, Japanese, and Hindi) improves alignment in several configurations. Our analysis pinpoints two key reasons for this failure: the models' low sensitivity to cultural personas and a strong, culturally invariant symptom hierarchy that overrides cultural cues. These findings reveal that while prompt language is important, current general-purpose LLMs lack the robust, culture-aware capabilities essential for safe and effective mental health applications.

Shintaro Sakai, Jisun An, Migyeong Kang +1

Indiana University Bloomington, USA · Sungkyunkwan University, Republic of Korea

Mental Health Cultural Bias in Language Models Clinical Language Model Evaluation

Aug 1, 2025cs.CL

Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications

The proliferation of Large Language Models (LLMs) in medicine has enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning, a cornerstone of clinical practice. This has catalyzed a shift from single-step answer generation to the development of LLMs explicitly designed for medical reasoning. This paper provides the first systematic review of this emerging field. We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies (e.g., supervised fine-tuning, reinforcement learning) and test-time mechanisms (e.g., prompt engineering, multi-agent systems). We analyze how these techniques are applied across different data modalities (text, image, code) and in key clinical applications such as diagnosis, education, and treatment planning. Furthermore, we survey the evolution of evaluation benchmarks from simple accuracy metrics to sophisticated assessments of reasoning quality and visual interpretability. Based on an analysis of 60 seminal studies from 2022-2025, we conclude by identifying critical challenges, including the faithfulness-plausibility gap and the need for native multimodal reasoning, and outlining future directions toward building efficient, robust, and sociotechnically responsible medical AI.

Zizhan Ma, Wenxuan Wang, Meidan Ding +7

The Chinese University of Hong Kong · Renmin University of China · Shenzhen University +3

LLM Prompting RL for Language Model Reasoning Multimodal Reasoning Clinical Language Model Evaluation Medical LLMs

Jul 29, 2025cs.CL

HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs

Millions of individuals' well-being are challenged by the harms of substance use. Harm reduction as a public health strategy provides non-judgemental, evidence-based information intended to improve health outcomes and reduce associated safety risks. Some large language models (LLMs) have demonstrated a high level of medical reasoning, promising to address the information needs of people who use drugs (PWUD). However, their performance in relevant tasks remains largely unexplored. We introduce HarmReduction, a benchmark designed to evaluate LLMs' accuracy and safety risks in harm reduction information provision. The benchmark dataset (HR-Basic) has 2,160 question-answer-evidence pairs. The scope covers three tasks: checking safety boundaries, providing quantitative values, and inferring polysubstance use risks. We build the Instruction and RAG schemes to evaluate model behaviours based on their inherent knowledge and the integration of domain knowledge. Our results indicate that state-of-the-art LLMs still struggle to provide accurate harm reduction information, and sometimes, present severe safety risks to PWUD. This work contributes an evaluation framework for LLMs to deliver harm reduction information to avoid introducing negative health outcomes through the use of LLMs.

Kaixuan Wang, Chenxin Diao, Jason T. Jacques +2

University of Oxford, United Kingdom · University of Edinburgh, United Kingdom · University of St Andrews, United Kingdom +1

Healthcare Question Answering LLM Safety Evaluation Clinical Language Model Evaluation

Jun 28, 2025cs.CL

Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues

Large language models (LLMs) may support counseling training, yet evidence from Japanese-language interactions and automated quality ratings remains limited. We examined 18 fixed Japanese-language counseling transcripts generated through artificial intelligence (AI)-to-AI interactions under three counselor conditions: GPT-minimal (GPT-4-turbo with a minimal role instruction), GPT-SMDP (GPT-4-turbo with the Structured Multi-step Dialogue Prompt [SMDP]), and Claude-SMDP (Claude-3-Opus with SMDP). Fifteen counseling experts rated transcripts on four adapted global scales from the Motivational Interviewing Treatment Integrity coding manual and an overall-quality item; three newer LLMs independently rated the same transcripts in three iterations. In this fixed stimulus set, SMDP-condition dialogues received higher expert ratings for cultivating change talk, partnership, empathy, and overall quality than GPT-minimal dialogues; the two SMDP counselor models did not differ. LLM ratings were reproducible but generally more lenient than expert-reference ratings, particularly for softening sustain talk and overall quality. Simulated-client naturalness was below the scale midpoint. These findings provide an expert-referenced benchmark for Japanese-language AI counseling simulations and show that reproducible LLM ratings should not be treated as calibrated counseling-quality evidence without expert validation. This study does not test clinical effectiveness or human-client outcomes.

Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Goto +5

Japan National Institute of Occupational Safety and Health, Kanagawa, Japan · Kaze To Taiyo, Tokyo, Japan · Saga Occupational Health Association, Saga, Japan +5

Mental Health Counseling LLM-as-a-Judge Emotional Support Conversation LLM Prompting Clinical Language Model Evaluation

Sep 3, 2024cs.HC

Therapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions

Large language models (LLMs) are increasingly being used as ad hoc therapists. While prior research has found that LLMs outperform human counselors in generating single-turn empathetic responses, fewer studies have compared their behaviors across multi-turn sessions. In this study, we compare the session-level behaviors of human peer counselors with those of an LLM, both trained on the same manual to deliver multi-turn, single-session Cognitive Behavioral Therapy (CBT). Our three-phase, mixed-methods study involved: (a) an 18-month ethnography of a peer support platform, where seven counselors iteratively refined CBT prompts through 110 self-counseling sessions and 60 weekly focus groups; (b) a novel session generation method that allows direct, controlled comparison of human and LLM counselors under matched conditions---client responses were drawn from publicly available human-led CBT sessions while counselor responses were generated by a CBT-prompted LLM; and (c) expert evaluations conducted by three licensed clinical psychologists. Through data triangulation, our results show a trade-off. Human peer counselors use relational techniques to interpret subtle cues, adapt CBT to users' values and cultural contexts, and use strategies such as small talk and contextually relevant self-disclosure to build rapport and guide the session, but often at the expense of session structure and therapeutic focus. LLM counselors, on the other hand, demonstrate greater methodological adherence to CBT techniques, but struggle to sustain turn-taking, frequently fail to distinguish between clinically important and trivial content, and are more prone to lecturing and imposing solutions. LLM counselors also tend to produce ``deceptive empathy'', excessively anthropomorphic responses that can inflate user expectations of genuine human care.

Zainab Iftikhar, Sean Ransom, Amy Xiao +2

Brown University, USA · Department of Psychiatry, Louisiana State University Health Sciences, USA · Alpert Medical School of Brown University, Brown University, USA

Mental Health Counseling Conversational Agents Clinical Language Model Evaluation Cognitive Behavioral Therapy

Date pendingcs.CL

"Mirror" Large Language Model Evaluations of Depression are Criterion Contaminated

Large Language Model (LLM) studies that use language responses elicited from depression assessments to predict scores on those same assessments often report near-perfect prediction of depression. We refer to these as "Mirror" evaluations and demonstrate an applied case of criterion contamination. N = 110 participants completed both structured diagnostic depression interviews (Mirror condition) and life history interviews ("Non-Mirror" condition). LLMs were prompted to predict depression scores in each condition. As expected, Mirror evaluations were near-perfect. However, Non-Mirror evaluations also displayed prediction sizes considered outstanding in psychology. Further, both Mirror and Non-Mirror predictions correlated with Patient Health Questionnaire-9 scores at similar sizes, suggesting the Mirror condition's advantage collapses when predicting an independent depression measurement. Topic modeling revealed differing depression-related themes across interview types. Mirror evaluations are better considered as reliability evaluations than as validity evaluations. Incorporating Non-Mirror approaches in LLM depression assessment may support more valid and clinically-relevant applications. Keywords: large language models, psychological assessment, psychopathology, depression, reliability, validity, criterion contamination

Tong Li, Rasiq Hussain, Mehak Gupta +1

Division of Computational & Data Sciences (DCDS), Washington University in St. Louis, St. Louis, MO · Department of Computer Science, SMU · Department of Psychological & Brain Sciences, Washington University in St. Louis

Depression Detection Mental Health Benchmark Contamination Psychometric Validation Depression Severity Estimation Clinical Language Model Evaluation

Date pendingcs.CL

Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation

Large-scale language models (LLMs) often offer clinical judgments based on incomplete information, increasing the risk of misdiagnosis. Existing studies have primarily evaluated confidence in single-turn, static settings, overlooking the coupling between confidence and correctness as clinical evidence accumulates during real consultations, which limits their support for reliable decision-making. We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations. Our benchmark unifies three types of medical data for open-ended diagnostic generation and introduces an information sufficiency gradient to characterize the confidence-correctness dynamics as evidence increases. We implement and compare 27 representative methods on this benchmark; two key insights emerge: (1) medical data amplifies the inherent limitations of token-level and consistency-level confidence methods, and (2) medical reasoning must be evaluated for both diagnostic accuracy and information completeness. Based on these insights, we present MedConf, an evidence-grounded linguistic self-assessment framework that constructs symptom profiles via retrieval-augmented generation, aligns patient information with supporting, missing, and contradictory relations, and aggregates them into an interpretable confidence estimate through weighted integration. Across two LLMs and three medical datasets, MedConf consistently outperforms state-of-the-art methods on both AUROC and Pearson correlation coefficient metrics, maintaining stable performance under conditions of information insufficiency and multimorbidity. These results demonstrate that information adequacy is a key determinant of credible medical confidence modeling, providing a new pathway toward building more reliable and interpretable large medical models.

Zhiyao Ren, Yibing Zhan, Siyuan Liang +3

Nanyang Technological University, Singapore, Singapore. · Wuhan University, Wuhan, China.

Healthcare Clinical Reasoning LLM Uncertainty Estimation Clinical Language Model Evaluation Medical LLMs

Date pendingcs.AI

VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise

Medical large language models are typically evaluated on idealized patient cases that do not reflect how real patients communicate. We introduce VeriSim, a patient simulation framework that injects controllable noise along six clinically grounded communication dimensions while substantially preserving each patient's medical record. Truth adherence is supported by a verifier that extracts atomic claims from each candidate utterance and judges them against a UMLS-grounded vector index built with BioLORD embeddings, using the retrieved atoms' structured clinical metadata (e.g., drug class, anatomical site, treats-condition relations) rather than surface-text similarity alone. Across seven open-weight LLMs, realistic noise reduces diagnostic accuracy by 15-25 percentage points and increases conversation length by 34-55%; the 7-8B models degrade 1.4x more than 70B+ models. A board-certified physician and a licensed nurse rate VeriSim's conversations highly on truth, realism, clinical utility, and noise fidelity (inter-annotator agreement >= 0.80 across all dimensions), and an LLM-as-judge closely tracks their ratings, enabling scalable evaluation. We release VeriSim as an open-source framework for rigorous stress-testing of clinical AI: https://github.com/mohitmarvania/VeriSim

Sina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5

Department of Computer Science, George Mason University · Department of Health Administration and Policy, College of Public Health, George Mason University · Department of Science, George Mason University

Healthcare LLM Reliability Clinical Language Model Evaluation Language Model Robustness Medical LLMs

Date pendingcs.CL

Expert-Level Crisis Detection in Mental Health Conversations

Real-world crisis intervention is inherently conversational, yet existing research largely focuses on static texts. When applied to multi-turn dialogues, current models exhibit significant performance degradation, struggling to track risk signals that emerge as context evolves. To address this gap, we introduce CRADLE-Dialogue, a clinician-annotated benchmark for turn-level crisis detection in conversational settings. The dataset features 600 dialogues with multi-label annotations across clinically grounded risks, including suicide ideation, self-harm, and child abuse, distinguishing past from ongoing risk. We further propose an Alert-Confirm evaluation protocol that distinguishes early warning signals (Alert) from turns where a specific crisis becomes explicitly identifiable (Confirm), reflecting the clinical need to intervene before risk becomes explicit. Experiments show that identifying when risk emerges is much harder than recognizing that it exists: models achieve only mid-40% to high-60% Micro F1. Additionally, we release a synthetic training corpus and a 32B-parameter model that substantially outperforms existing open-source models and achieves competitive or superior results against proprietary models across turn-level, dialogue-level, and confirm-only evaluation settings.

Grace Byun, Abigail Lott, Rebecca Lipschutz +3

Department of Computer Science, Emory University · Department of Psychiatry and Behavioral Sciences, Emory University

Mental Health Suicide Risk Modeling Clinical Language Model Evaluation Clinical Risk Prediction