Fine-Tune, Don't Prompt, Your Language Model to Identify Biased Language in Clinical Notes
Authors: Isotta Landi, Eugenia Alleva, Nicole Bussola, Rebecca M. Cohen, Sarah Nowlin, Leslee J. Shaw, Alexander W. Charney, Kimberly B. Glazer
Organizations: Department of AI and Human Health, Icahn School of Medicine at Mount Sinai, NY. · Icahn School of Medicine at Mount Sinai, NY. · Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, NY. · Obstetrics, Gynecology and Reproductive Science, Blavatnik Family Women’s Health Research Institute, Icahn School of Medicine at Mount Sinai, NY. · Obstetrics and Gynecology, University of Pennsylvania, PA.
Clinical documentation can contain emotionally charged language with stigmatizing or privileging valences. We present a framework for detecting and classifying such language as stigmatizing, privileging, or neutral. We constructed a curated lexicon of biased terms scored for emotional valence. We then used lexicon-based matching to extract text chunks from OB-GYN delivery notes (Mount Sinai Hospital, NY) and MIMIC-IV discharge summaries across multiple specialties. Three clinicians annotated all chunks, enabling characterization of valence patterns across specialties and healthcare systems. We benchmarked multiple classification strategies (zero-shot prompting, in-context learning, and supervised fine-tuning) across encoder-only models (GatorTron) and generative large language models (Llama). Fine-tuning with lexically primed inputs consistently outperformed prompting approaches. GatorTron achieved an F1 score of 0.96 on the OB-GYN test set, outperforming larger generative models while requiring minimal prompt engineering and fewer computational resources. External validation on MIMIC-IV revealed limited cross-domain generalizability (F1 < 0.70, 44% drop). Training on the broader MIMIC-IV dataset improved generalizability when testing on OB-GYN (F1 = 0.71, 11% drop), but at the cost of reduced precision. Our findings demonstrate that fine-tuning outperforms prompting for emotional valence classification and that models must be adapted to specific medical specialties to achieve clinically appropriate performance. The same terms can carry different emotional valences across specialties: words with clinical meaning in one context may be stigmatizing in another. For bias detection, where misclassification risks undermining clinician trust or perpetuating patient harm, specialty-specific fine-tuning is essential to capture these semantic shifts. * Equal contribution.
Figures & tables
Figure 1: Multi-stage methodology outline.
Figure 2: Mean lexicon scores for stigmatizing and privileging valence are shown alongside emotional valence annotation labels for OB-GYN (panel a) and MIMIC-IV (panel b) text chunks. Solid dots represent annotated valence labels, while vertical lines indicate the range of emotional valences observed in clinical note chunks relative to the original lexicon scores. Blue dots denote instances labeled as neutral during real-world annotation. Solid black dots represent words that were not assigned a stigmatizing or privileging score in the lexicon but were labeled as such during annotation of real-world text chunks.
Model
Input
Training Strategy
OB-GYN
MIMIC-IV
Dev
Test
GatorTron
Anchored
Standard FT
0.9531
0.9456(0.9069,0.9755)
0.6765(0.6133,0.7323)
Prompt FT
0.9635
0.9487(0.9117,0.9794)
0.5918(0.5281,0.6560)
LP
Prompt FT
0.9783
0.9572(0.9250,0.9839)
0.6598(0.5954,0.7210)
BERT
Anchored
Standard FT
0.9510
0.9154(0.8695,0.9540)
0.5910(0.5233,0.6572)
Prompt FT
0.9581
0.9311(0.8883,0.9665)
0.5655(0.4967,0.6341)
Table 1: F1 scores across different input types and tuning strategies. Numbers in bold represent the highest scores per group of models (encoder-only vs decoder-only) and datasets. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Valence
Word(s)
Example
Privileging
engaged, engagement, engages
Patient is highly engaged in their care.
pleasant
Patient was pleasant throughout encounter.
compliant
Patient is compliant with all recommendations.
Stigmatizing
noncompliant, noncompliance,
…has been noncompliant with follow up.
poor compliance
Noncompliant with treatment recommendations.
lazy
Patient seems too lazy to make lifestyle changes.
Appendix
Table 2: Examples of lexicon and sentences for stigmatizing and privileging valence scoring.
Dataset
Annotation
Text Chunk
OB-GYN
Neutral
…Mrs. ___ is unwilling to divulge the patient’s psychiatric history…
Privileging
…PO intake gradually increasing; more willing to take PO psychiatric medications…
Stigmatizing
…Pt is unwilling to continue her IOL She feels that she has been in pain since hours prior and wants to be delivered…
MIMIC-IV
Neutral
…You are aware that there are food services such as Wheels on Meals, if your father is willing …
Privileging
…Upon arrival to the floor, the patient is pleasant and in no distress…
Stigmatizing
…He has been unwilling to eat or get out of bed lately as per the daughter…
Appendix
Table 3: Clinical note chunks labeled as neutral, privileging, or stigmatizing from the OB-GYN and MIMIC-IV datasets.
Method
Word(s)
Target Class
Single-Word
negative
stigmatizing
positive
privileging
neutral
neutral
Multi-Word
negative, stigma, bad
stigmatizing
positive, privilege, good
privileging
neutral, instruction, clinical
neutral
Appendix
Table 4: Single- and multi-word verbalizer mappings.
Input
Tuning Strategy
Writer
Instruction
Anchored chunks
Zero-shot
Human
Classify whether the chunk extracted from a clinical note carries a privileging, stigmatizing, or neutral valence.
LP chunks
Zero-shot
Human
Classify whether the word indicated at the beginning of the chunk extracted from a clinical note carries a privileging, stigmatizing, or neutral valence in the context provided.
LP chunks
MIPROv2
Llama 3.3-70B
1. Analyze the given clinical note chunk, focusing on the word indicated at the beginning, and classify its valence as privileging, stigmatizing, or neutral based on the context provided. Consider the emotional tone conveyed by the language used in the chunk, taking into account the medical terminology, patient description, and overall sentiment expressed. Assign a valence that accurately reflects the emotional connotation of the word within the clinical note, ensuring that your classification is informed by the nuances of medical communication and the specific details presented in the chunk.
2. Analyze the given chunk of text from a clinical note, focusing on the word indicated at the beginning. Determine whether this word conveys a privileging, stigmatizing, or neutral valence within the provided context, and classify it accordingly.
LP chunks
MIPROv2
Med42-70B
1. For each identified keyword in a clinical note chunk (e.g., ”pleasant,” ”adherence”), determine its valence as privileging, stigmatizing, or neutral by considering both medical context and linguistic cues. Implement domain-specific sentiment analysis rules or train a machine learning model on annotated datasets to assign accurate valences. Ensure the instruction emphasizes capturing nuanced expressions typical in clinical documentation while prioritizing factual representations over emotional bias.
2. In a critical care setting, accurately classify sentiment valence (privileging, stigmatizing, or neutral) in clinical text chunks to inform timely interventions and patient summaries. Given phrases from psychiatric and internal medicine notes, such as ”difficulty inspiring” for respiratory symptoms or ”appropriately dressed” for mental status evaluation, the Language Model must assign correct valence labels considering domain-specific context.
Appendix
Table 5: Instructions for generative tasks with large language models. Optimized instructions are derived from tuning the prompts on (1) OB-GYN; and (2) MIMIC-IV datasets.
Model
Input
Training Strategy
OB-GYN
MIMIC-IV
Dev
Test
GatorTron
Anchored
Standard FT
0.9481
0.9361(0.8892,0.9720)
0.6704(0.5986,0.7348)
Prompt FT
0.9577
0.9388(0.8944,0.9762)
0.5998(0.5354,0.6671)
LP
Prompt FT
0.9745
0.9503(0.9111,0.9818)
0.6480(0.5824,0.7155)
BERT
Anchored
Standard FT
0.9520
0.9193(0.8714,0.9599)
0.5982(0.5256,0.6700)
Prompt FT
0.9481
0.9172(0.8683,0.9603)
0.5618(0.4935,0.6356)
Appendix
Table 6: Precision scores across different input types and tuning strategies. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Model
Input
Training Strategy
OB-GYN
MIMIC-IV
Dev
Test
GatorTron
Anchored
Standard FT
0.9585
0.9576(0.9245,0.9817)
0.7115(0.6547,0.7654)
Prompt FT
0.9699
0.9637(0.9365,0.9855)
0.6042(0.5371,0.6727)
LP
Prompt FT
0.9823
0.9662(0.9384,0.9885)
0.6808(0.6138,0.7431)
BERT
Anchored
Standard FT
0.9502
0.9132(0.8624,0.9526)
0.6016(0.5319,0.6673)
Prompt FT
0.9699
0.9504(0.9156,0.9777)
0.5852(0.5164,0.6529)
Appendix
Table 7: Recall scores across different input types and tuning strategies. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Model
Input
Training Strategy
MIMIC-IV
OB-GYN
Dev
Test
GatorTron
Anchored
Standard FT
0.6168
0.5949(0.4187,0.7640)
0.5901(0.5554,0.6247)
Prompt FT
0.7532
0.8034(0.6482,0.9136)
0.7144(0.6843,0.7455)
LP
Prompt FT
0.9381
0.7189(0.5477,0.8619)
0.6547(0.6225,0.6853)
BERT
Anchored
Standard FT
0.5166
0.5190(0.3552,0.6863)
0.4773(0.4452,0.5102)
Prompt FT
0.7315
0.7786(0.6249,0.9010)
0.6201(0.5859,0.6521)
Appendix
Table 8: F1 scores across different input types and tuning strategies when training on MIMIC-IV and testing on OB-GYN datasets. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Model
Input
Training Strategy
MIMIC-IV
OB-GYN
Dev
Test
GatorTron
Anchored
Standard FT
0.6741
0.6147(0.4324,0.8046)
0.6839(0.6452,0.7230)
Prompt FT
0.7468
0.7755(0.6258,0.9020)
0.7748(0.7461,0.8022)
LP
Prompt FT
0.9484
0.7185(0.5555,0.8693)
0.7680(0.7399,0.7957)
BERT
Anchored
Standard FT
0.5748
0.5306(0.3659,0.7089)
0.5660(0.5299,0.6029)
Prompt FT
0.7349
0.7998(0.6227,0.9350)
0.7224(0.6924,0.7516)
Appendix
Table 9: Precision scores across different input types and tuning strategies when training on MIMIC-IV and testing on OB-GYN datasets. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Model
Input
Training Strategy
MIMIC-IV
OB-GYN
Dev
Test
GatorTron
Anchored
Standard FT
0.6278
0.6027(0.4271,0.7927)
0.5662(0.5353,0.5961)
Prompt FT
0.7722
0.8699(0.7756,0.9482)
0.6931(0.6617,0.7257)
LP
Prompt FT
0.9308
0.7428(0.5679,0.8926)
0.6328(0.6017,0.6627)
BERT
Anchored
Standard FT
0.4884
0.5309(0.3629,0.7100)
0.4790(0.4502,0.5076)
Prompt FT
0.7350
0.7784(0.6049,0.9078)
0.6075(0.5759,0.6384)
Appendix
Table 10: Recall scores across different input types and tuning strategies when training on MIMIC-IV and testing on OB-GYN datasets. Mean and 95% confidence intervals obtained via bootstrapping on the test and external validation datasets are reported.
Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as clinical decision support and medical documentation. However, the robustness of these models against subtle linguistic variations, specifically stigmatizing language (SL) commonly found in human-authored clinical notes, remains critically under-explored. In this work, we investigate whether frontier LLMs inherit and propagate this human bias when processing clinical text. We systematically evaluate nine frontier LLMs across four stigmatized medical conditions, utilizing clinical vignettes injected with varying intensities and phenotypes of SL (doubt, blame, and maligning). Our results demonstrate that all evaluated models exhibit substantial bias, with clinical decision-making significantly skewed towards less aggressive patient management. Notably, we observe a high sensitivity to linguistic framing, where a single SL sentence is sufficient to alter model outputs, revealing a clear dose-response relationship. Furthermore, we evaluate standard prompt-based mitigation strategies, including Chain-of-Thought (CoT) reasoning and model self-debiasing. These approaches show limited efficacy; models struggle to explicitly identify SL while remaining implicitly influenced by it. Our findings expose a critical vulnerability in current LLMs regarding fairness and robustness in clinical NLP, underscoring the need for rigorous algorithmic guardrails to prevent the automation of health disparities.
Ambient artificial intelligence (AI) documentation tools are increasingly deployed to reduce clinician documentation burden, but their implications for biased language in clinical notes remain unclear. We conducted a large-scale comparison analysis of AI drafts and corresponding clinician finalized notes to quantify stigmatizing language changes pre- and post-editing. Using a lexicon-based natural language processing (NLP) pipeline, we measured (1) the prevalence of stigmatizing language in AI drafts, (2) the prevalence and term composition in final notes, and (3) the frequency of removal or introduction of stigmatizing terms. Across 66,297 paired note sections, 21.4% of AI draft sections contained at least one stigmatizing language mention, rising to 24.0% in clinician finalized versions. Introductions occurred more often than removals, suggesting clinician editing can be a net source of stigmatizing language entering the EHR with using Ambient AI.
Yiliang Zhou, Yawen Guo, Sairam Sutari +9
Department of Informatics, University of California, Irvine, Irvine, CA, USA · Institute for Clinical and Translational Science, University of California, Irvine, Irvine, CA, USA · Department of Radiological Sciences, University of California, Irvine, Irvine, CA, USA +3
Clinical language models are evaluated on vignettes written in a clinician's voice, but patients describe the same illness in their own. We pair 1,000 MedQA cases with three retellings that carry the same audited original facts, in a clinical, a socioeconomic and a cultural voice, and evaluate 12 open models (3B to 24B, including two medical fine-tunes). Every model is less accurate when the patient speaks in a marked voice: 26% more errors, a pooled penalty of −9.6 accuracy points (95% CI [−11.0,−8.2]), with no detectable reduction at larger scale. The penalty is not explained by any retelling property we measured (length, omitted age, MedQA split, added clinical content) and persists when the marked narrative contains the original vignette verbatim; because generation settings differ between voices, we read it as sensitivity to these marked retellings, not proof that register alone is the cause. Chain-of-thought and explicit debiasing instructions together remove only about a fifth of the gap. Having the same model first rewrite the patient's account as a clinical vignette, then answer from the rewrite, halves the gap while raising marked-voice accuracy by 3.9 points (means over the 11 models that can run the pipeline). A matched pipeline using a structured schema instead of prose closes the same gap but is 7.1 points less accurate in all 11 models, and adding reasoning agents or tools costs more still. Rewriting is not free: it recovers more facts than extraction but also invents plausible details, which we quantify. In our experiments, the representation a model reasons over matters more than how much it reasons.
Prabhjot Singh, Pritam Deka, Vijay Chennareddy
University of Texas at Austin Austin, Texas, USA · Queen’s University Belfast Belfast, United Kingdom · Middlesex University London, United Kingdom