Clinical Language Model Evaluation

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 118% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 259

All topics

Continued Pretraining of FinBERT on Finnish Histopathological Reports: Train-Time Signals and Proxy Downstream Correlations
Apr 16, 2026Rami Luisto, Liisa Petäinen, Tommi Grönholm +5Healthcare Domain-Adaptive Pretraining
Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland. · Digital Workforce Services, Helsinki, Finland. · Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland. +2
IatroBench: A Pre-Registered Benchmark of Clinical Omission in Language Models
Apr 9, 2026David GringrasLLM Safety Benchmarks Healthcare
Harvard T.H. Chan School of Public Health
Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations
Mar 31, 2026Yahan Li, Xinyi Jie, Wanjia Ruan +5Healthcare Clinical Reasoning
University of Southern California
Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context
Jan 25, 2026Zhihao Zhang, Liting Huang, Guanghao Wu +3LLM Safety Benchmarks Language Model Safety Evaluation
Macquarie University1 · UTS2 · MBZUAI3 +1
SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care
Jan 23, 2026Dongshen Peng, Yi Wang, Austin Schoeffler +5LLM Sycophancy Language Model Safety Evaluation
UNC Chapel Hill · University of Waterloo · Stanford University
MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence
Jan 6, 2026Bohao Chu, Hendrik Damm, Tabea M. G. Pakull +5Healthcare Text Summarization
University of Duisburg-Essen · University of Applied Sciences and Arts Dortmund · University Hospital Essen +1
Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data
Dec 25, 2025Yiwei Yan, Guanfeng LiuHealthcare Text Classification
School of Computing, Macquarie University, Australia
First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations
Dec 1, 2025David Wu, Fateme Nateghi Haredasht, Saloni Kumar Maharaj +54Healthcare Clinical Decision Support
MedRECT: A Bilingual Medical Reasoning Benchmark for Error Correction in Clinical Texts
Nov 1, 2025Naoto Iwase, Hiroki Okuyama, Junichiro IwasawaMultilingual Language Model Evaluation Healthcare
Preferred Networks, Inc., Tokyo, Japan · School of Medicine, Nagoya University, Nagoya, Japan
Zoom In Disparities in Healthcare LLM Q&A
Oct 20, 2025Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao +5Multilingual Language Model Evaluation Knowledge Augmentation for Language Models
Universitat Politècnica de València, Valencia, Spain · University of Trento, Trento, Italy · University of Sheffield, Sheffield, UK +5
Somatic in the East, Psychological in the West?: Investigating Clinically-Grounded Cross-Cultural Depression Symptom Expression in LLMs
Aug 5, 2025Shintaro Sakai, Jisun An, Migyeong Kang +1Mental Health Cultural Bias in Language Models
Indiana University Bloomington, USA · Sungkyunkwan University, Republic of Korea
Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
Aug 1, 2025Zizhan Ma, Wenxuan Wang, Meidan Ding +7LLM Prompting RL for Language Model Reasoning
The Chinese University of Hong Kong · Renmin University of China · Shenzhen University +3
HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs
Jul 29, 2025Kaixuan Wang, Chenxin Diao, Jason T. Jacques +2Healthcare Question Answering
University of Oxford, United Kingdom · University of Edinburgh, United Kingdom · University of St Andrews, United Kingdom +1
Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues
Jun 28, 2025Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Goto +5Mental Health Counseling LLM-as-a-Judge
Japan National Institute of Occupational Safety and Health, Kanagawa, Japan · Kaze To Taiyo, Tokyo, Japan · Saga Occupational Health Association, Saga, Japan +5
Therapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions
Sep 3, 2024Zainab Iftikhar, Sean Ransom, Amy Xiao +2Mental Health Counseling Conversational Agents
Brown University, USA · Department of Psychiatry, Louisiana State University Health Sciences, USA · Alpert Medical School of Brown University, Brown University, USA
"Mirror" Large Language Model Evaluations of Depression are Criterion Contaminated
Date pendingTong Li, Rasiq Hussain, Mehak Gupta +1Depression Detection Mental Health
Division of Computational & Data Sciences (DCDS), Washington University in St. Louis, St. Louis, MO · Department of Computer Science, SMU · Department of Psychological & Brain Sciences, Washington University in St. Louis
Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation
Date pendingZhiyao Ren, Yibing Zhan, Siyuan Liang +3Healthcare Clinical Reasoning
Nanyang Technological University, Singapore, Singapore. · Wuhan University, Wuhan, China.
VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise
Date pendingSina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5Healthcare LLM Reliability
Department of Computer Science, George Mason University · Department of Health Administration and Policy, College of Public Health, George Mason University · Department of Science, George Mason University
Expert-Level Crisis Detection in Mental Health Conversations
Date pendingGrace Byun, Abigail Lott, Rebecca Lipschutz +3Mental Health Suicide Risk Modeling
Department of Computer Science, Emory University · Department of Psychiatry and Behavioral Sciences, Emory University

Apr 16, 2026, cs.CLJ/K move · Enter open · S save

Continued Pretraining of FinBERT on Finnish Histopathological Reports: Train-Time Signals and Proxy Downstream Correlations

Rami Luisto, Liisa Petäinen, Tommi Grönholm, Jan Böhm+4

Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland. · Digital Workforce Services, Helsinki, Finland. · Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland.+2

Healthcare Domain-Adaptive Pretraining Continued Pretraining Clinical Language Model Evaluation Clinical NLP

Open paper AskPDF arXiv

In Natural Language Processing (NLP) classification tasks where a lack of labeled data is an issue, continued pretraining (CPT) of transformer models on unlabeled data is an established approach. In this paper, we have two aims. (1) We describe our observations from continued pretraining of the Finnish BERT transformer model (FinBERT) on a Finnish histopathological dataset (below, \emph{the Histopathology data}). (2) Since the Histopathology data has no classification labels, we gather public Finnish datasets as proxy data to analyze whether the signals observed in (1) are associated with downstream classification gains. We observe that CPT train-time loss curves differ strongly by domain, and that, in an exploratory analysis, certain CPT-derived features correlate with proxy classification improvement. In particular, this report contributes to the limited literature on NLP for Finnish healthcare data.

Momentum

Related topics

Latest papers 259