cs.CLApr 14, 2026

L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification

Authors: Rishik Kondadadi, John E. Ortega

Organizations: University of Minnesota, Northeastern University, Minneapolis, Boston, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Clinical text classification requires choosing between specialized fine-tuned models (BERT variants) and general-purpose large language models (LLMs), yet neither dominates across all instances. We introduce Learning to Defer for clinical text (L2D-Clinical), a framework that learns when a BERT classifier should defer to an LLM based on uncertainty signals and text characteristics. Unlike prior L2D work that defers to human experts assumed universally superior, our approach enables adaptive deferral-improving accuracy when the LLM complements BERT. We evaluate on two English clinical tasks: (1) ADE detection (ADE Corpus V2), where BioBERT (F1=0.911) outperforms the LLM (F1=0.765), and (2) treatment outcome classification (MIMIC-IV with multi-LLM consensus ground truth), where GPT-5-nano (F1=0.967) outperforms ClinicalBERT (F1=0.887). On ADE, L2D-Clinical achieves F1=0.928 (+1.7 points over BERT) by selectively deferring 7% of instances where the LLM's high recall compensates for BERT's misses. On MIMIC, L2D-Clinical achieves F1=0.980 (+9.3 points over BERT) by deferring only 16.8% of cases to the LLM. The key insight is that L2D-Clinical learns to selectively leverage LLM strengths while minimizing API costs.

Figures & tables

MethodF1AccLLM%
BioBERT only0.9110.9480%
LLM only0.7650.834100%
Random (7%)0.9030.9387.0%
Fixed θ=0.95\theta=0.950.9210.9542.2%
L2D (Ours)0.9280.9587.0%
Oracle (upper bound)0.9450.9682.0%
Table 1: ADE Corpus V2: BioBERT outperforms LLM, but L2D-Clinical improves further by selectively using LLM for uncertain cases. Random deferral at the same rate (7%) yields F1=0.903, confirming L2D-Clinical’s gains come from intelligent routing.
MethodF1AccLLM%
ClinicalBERT only0.8870.9460%
GPT-5-nano only0.9670.968100%
Fixed θ=0.95\theta=0.950.971—26.2%
L2D-Clinical (Ours)0.9800.98616.8%
Table 2: MIMIC-IV Treatment Outcomes with consensus ground truth: GPT-5-nano outperforms ClinicalBERT on this task. L2D-Clinical achieves the best F1 (0.980) by deferring only 16.8% of cases, outperforming the fixed threshold baseline while using fewer LLM calls.
TaskBERTL2DΔ\DeltaLLM%
ADE0.9110.928+1.77.0%
MIMIC0.8870.980+9.316.8%
Table 3: L2D-Clinical adapts to different model dynamics, improving F1 in both scenarios. On ADE (BERT >> LLM), L2D achieves gains with minimal deferral. On MIMIC (LLM >> BERT), L2D defers more frequently.
FeatureADEMIMIC
Entropy+0.82+0.91
Margin (top-2 diff)−- 0.71−- 0.68
Confidence−- 0.54−- 0.47
Log(text length)+0.23+0.31
Has “discontinued”—−- 0.44
Has causal language−- 0.38—
Table 4: Deferral model coefficients (positive = defer to LLM). Uncertainty features dominate; task-specific keywords add interpretable signal.
 Prior L2DL2D-Clinical
Expert typeHumanAI (LLM)
Expert reliabilityAlways betterInstance-dep.
Deferral signalUncertaintyUncert. + feat.
Table 5: L2D-Clinical vs. prior Learning to Defer approaches.

Explore similar work

CardsList
Sep 29, 2026

Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts

As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.

Abinitha Gourabathina, Haoran Zhang, Yuexing Hao +2

Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science · Worcester Polytechnic Institute, Department of Computer Science

Clinical TriageClinical Decision-MakingLanguage Model Bias EvaluationClinical Language Model EvaluationLanguage Model Robustness
AskPDF
Jun 16, 2026

Possible or Definite? A Benchmark for Evaluating Diagnostic Uncertainty Preservation in Clinical Text

Large language models (LLMs) are increasingly used for clinical text tasks such as summarization and revision. While most studies evaluate the fluency and coherence of LLM-generated text, whether LLMs correctly preserve diagnostic uncertainty remains underexplored. In clinical practice, phrases such as ``possible pneumonia'' communicate the strength of available evidence and directly guide decisions about follow-up testing and treatment. Altering these uncertainty expressions can change the clinical meaning entirely. In this paper, we systematically evaluated this problem in two steps. First, we constructed a benchmark of 1,200 clinical documents with 9,184 uncertainty annotations across five levels. Second, we evaluated three LLMs on this benchmark. Our results show that (1) LLMs preserve the original uncertainty cues poorly, often less than half the time; (2) LLMs struggle with nuanced distinctions between adjacent levels. This work reveals a failure mode not captured by standard evaluation metrics and provides implications for the safe deployment of LLMs in clinical workflows.

Hongbo Du, Zixin Lu, Jiaming Qu

Trine University · University of Michigan · Amazon

LLM ReliabilityClinical Language Model EvaluationMedical LLMs
AskPDF
Mar 27, 2026

Clinical named entity recognition in the Portuguese language: a benchmark of modern BERT models and LLMs

Clinical notes contain valuable unstructured information. Named entity recognition (NER) enables the automatic extraction of medical concepts; however, benchmarks for Portuguese remain scarce. In this study, we aimed to evaluate BERT-based models and large language models (LLMs) for clinical NER in Portuguese and to test strategies for addressing multilabel imbalance. We compared BioBERTpt, BERTimbau, ModernBERT, and mmBERT with LLMs such as GPT-5 and Gemini-2.5, using the public SemClinBr corpus and a private breast cancer dataset. Models were trained under identical conditions and evaluated using precision, recall, and F1-score. Iterative stratification, weighted loss, and oversampling were explored to mitigate class imbalance. The mmBERT-base model achieved the best performance (micro F1 = 0.76), outperforming all other models. Iterative stratification improved class balance and overall performance. Multilingual BERT models, particularly mmBERT, perform strongly for Portuguese clinical NER and can run locally with limited computational resources. Balanced data-splitting strategies further enhance performance.

Vinicius Anjos de Almeida, Sandro Saorin da Silva, Josimar Chire +8

Spesia, Curitiba - PR, Brazil · Faculdade de Medicina, Universidade de São Paulo, São Paulo - SP, Brazil · Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba - PR, Brazil +2

Named Entity RecognitionClinical NLPClinical Language Model EvaluationMedical LLMs
AskPDF