Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains

Manil Shrestha; Edward Kim

cs.CLMar 1, 2026

Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains

Authors: Manil Shrestha, Edward Kim

Organizations: Department of Computer Science, Drexel University, Philadelphia, USA

Abstract

Large Language Models (LLMs) are increasingly used for medical entity extraction, yet their confidence scores are often miscalibrated, limiting safe deployment in clinical settings. We present a conformal prediction framework that provides finite-sample coverage guarantees for LLM-based extraction across two clinical domains. First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1, verified via FactScore-based atomic statement evaluation (97.7% accuracy over 128,906 entities). Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick, evaluated against physician annotations (entity F1: 0.81 to 0.84). Our central finding is that miscalibration direction reverses across domains: on well-structured FDA labels, models are underconfident, requiring modest conformal thresholds ( $τ\approx 0.06$ ), while on free-text radiology reports, models are overconfident, demanding strict thresholds ( $τ$ up to 0.99). Despite this heterogeneity, conformal prediction achieves target coverage ( $\geq 90\%$ ) in both settings with manageable rejection rates (9--13%). These results demonstrate that calibration is not a global model property but depends on document structure, extraction category, and model architecture, motivating domain-specific conformal calibration for safe clinical deployment.

Figures & tables

$\alpha$	$\tau_{c}$	$\hat{p}_{\min}$	Rej.%	Cov.	Prec.
0.01	$\infty$	–	100.0	0.000	–
0.03	$-0.49$	.380	0.0	1.000	0.977
0.05	$-0.49$	.380	0.0	1.000	0.977
0.10	$-0.49$	.380	0.0	1.000	0.977
0.15	$-0.49$	.380	0.0	1.000	0.977
0.20	$-0.49$	.380	0.0	1.000	0.977

Table 1: FDR-controlled conformal prediction sweep across

\alpha

for FDA drug label extraction (GPT-4.1,

n{=}110{,}664

entities, global threshold). At

\alpha\geq 0.03

the global baseline FDR (2.3%) is already below target, so nearly all entities are accepted; per-section thresholds (Table 2 ) reveal the heterogeneity masked by global pooling.

Section	# Entities	Brier $\downarrow$	ECE $\downarrow$	FactS.	$\tau_{c}$	$\hat{p}_{\min}$	Rej.%	Cov.	Prec.
Adverse Reactions	63,442	0.013	0.012	2.98	$-0.09$	.479	0.0	1.000	0.987
Indications & Usage	34,895	0.022	0.018	2.94	$0.85$	.701	0.0	1.000	0.979
Drug Interactions	7,251	0.051	0.055	2.92	$10.52$	${>}.999$	59.8	0.400	0.948
Contraindications	2,109	0.048	0.041	2.90	$7.65$	${>}.999$	41.5	0.590	0.966
Pregnancy	545	0.005	0.004	2.99	$2.71$	.938	0.4	0.996	0.996
Pediatric Use	1,954	0.236	0.214	2.52	$\infty$	–	100.0	0.000	–

Table 2: Calibration metrics and FDR-controlled conformal prediction for FDA drug label extraction with GPT-4.1 (

\alpha=0.05

n{=}1{,}000

drugs).

\tau_{c}

is the per-section FDR threshold in log-odds;

\hat{p}_{\min}=\sigma(\tau_{c})

is the equivalent confidence cutoff. FDR control guarantees precision

\geq 1{-}\alpha

among accepted entities. Warnings section omitted (no valid entities).

Figure 1: Calibration curves for four sections from FDA label extraction with GPT-4.1 ( $n{=}1{,}000$ drugs). Predicted span confidence $\hat{p}$ (geometric mean of token probabilities) versus empirical accuracy. The model is systematically underconfident across most sections: curves lie above the diagonal. Pediatric Use is an exception, exhibiting overconfidence. ECE ranges from 0.012 (Adverse Reactions) to 0.214 (Pediatric Use).

		Entity			Relation
Model	Setting	Prec.	Rec.	F1	Prec.	Rec.	F1
GPT-4.1	5-shot	.798	.856	.826	.484	.521	.502
GPT-4.1	0-shot	.787	.710	.746	.301	.252	.274
Llama-4-Maverick	5-shot	.865	.818	.841	.526	.458	.490
Llama-4-Maverick	0-shot	.783	.674	.724	.228	.227	.228

Table 3: RadGraph entity and relation extraction on 100 test reports. Few-shot prompting (5 examples from dev set) substantially improves both entity and relation extraction compared to zero-shot.

Figure 2: Per-label calibration curves for GPT-4.1 and Llama-4-Maverick (5-shot, 100 CXR reports [ 11 ] ). Predicted span confidence $\hat{p}$ versus empirical accuracy across four entity categories. Both models are overconfident (curves below diagonal). Llama-4-Maverick achieves lower ECE overall (0.085 vs. 0.147), while OBS-U remains poorly calibrated for both models.

	GPT-4.1			Llama-4-Maverick
$\alpha$	Rej.%	Cov.	Prec.	Rej.%	Cov.	Prec.
0.05	100.0	0.000	–	100.0	0.000	–
0.10	59.3	0.445	0.876	19.6	0.824	0.870
0.15	41.7	0.617	0.849	0.2	0.998	0.849
0.20	6.8	0.936	0.805	0.2	0.998	0.849
0.25	0.1	0.999	0.802	0.2	0.998	0.849

Table 4: FDR-controlled conformal prediction sweep across

\alpha

for RadGraph entity extraction (global threshold, 5-shot). At

\alpha<0.10

, neither model can guarantee the target precision, resulting in full rejection. The sharp transition at

\alpha=0.10

reflects the baseline error rate (

{\sim}15{-}20\%

); Llama-4-Maverick’s better calibration yields a lower threshold and less rejection at the same

\alpha

Category	Prec.	Rec.	F1	ECE $\downarrow$	$\tau_{c}$	$\hat{p}_{\min}$	Rej.%	Prec.	Rec.	F1	ECE $\downarrow$	$\tau_{c}$	$\hat{p}_{\min}$	Rej.%
	GPT-4.1							Llama-4-Maverick
anat-dp	.834	.933	.880	.128	$9.92$	${>}.999$	57.3	.865	.893	.879	.085	$11.11$	${>}.999$	41.4
obs-dp	.827	.769	.797	.140	$6.45$	.998	52.1	.905	.724	.804	.064	$0.00$	.500	0.2
obs-da	.833	.914	.871	.102	$13.12$	${>}.999$	62.6	.902	.900	.901	.051	$0.00$	.500	0.0
obs-u	.305	.648	.415	.525	$\infty$	–	100.0	.444	.580	.502	.396	$\infty$	–	100.0

Table 5: Per-category entity extraction and FDR-controlled conformal prediction on 100 RadGraph test reports (

\alpha=0.10

, 5-shot).

\tau_{c}

is the per-category FDR threshold in log-odds;

\hat{p}_{\min}=\sigma(\tau_{c})

is the equivalent confidence cutoff. FDR control guarantees precision

\geq 1{-}\alpha

among accepted entities.

Explore similar work

CARE: A Conformal Safety Layer for Medical Summarization
Jun 8, 2026Suhana Bedi, Bridget Lin, Anson Y. Zhou +5Text Summarization Conformal Risk Control
Department of Biomedical Data Science, Stanford University · Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University · Department of Computer Science, Stanford University +2
Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models
Apr 15, 2026Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov +1Conformal Prediction Language Model Calibration
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE
Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
Apr 17, 2026Yanli Wang, Peng Kuang, Xiaoyu Han +2Confidence Estimation in Language Models Conformal Prediction
Imperial College London · Zhejiang University · University of Illinois Urbana-Champaign +1

Jun 8, 2026J/K move · Enter open · S save

CARE: A Conformal Safety Layer for Medical Summarization

Suhana Bedi, Bridget Lin, Anson Y. Zhou, Chloe O. Stanwyck+4

Department of Biomedical Data Science, Stanford University · Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University · Department of Computer Science, Stanford University+2

Text Summarization Conformal Risk Control LLM Hallucination Detection Medical LLMs

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used for medical summarization, but their outputs can omit medically important information and introduce unsupported claims. Existing error-detection methods produce heuristic or uncalibrated scores, providing no formal control over missed errors and no principled way to trade off safety against clinician review burden. We introduce Conformal Assessment for Risk Evaluation (CARE), a post-hoc, model-agnostic safety layer that uses conformal risk control to overlay calibrated omission and hallucination flags onto summaries from any LLM without retraining. CARE provides finite-sample, distribution-free guarantees through two controllers: a hallucination controller that bounds the probability of a document containing any unflagged hallucinated sentence, and an omission controller that bounds the expected fraction of important omissions not surfaced for review. Unlike hallucination detection, omissions depend jointly on whether a source sentence is important and whether it is covered by the summary. We show that calibrating only one dimension can violate the target risk bound, while marginal decompositions remain valid but overly conservative. By jointly calibrating over the full

(τ,γ)

threshold space, CARE preserves formal guarantees while surfacing up to 5

\times

fewer sentences than alternative calibrated baselines. Across five medical summarization tasks, CARE satisfies the target risk bound at

α= 0.15

with 95% confidence across 100 calibration/test resplits, using only ~100 labeled documents per domain. In a preliminary clinician study (75 document reviews), calibrated flags improved omission detection by 28.6 percentage points on average. These results show that sentence-level safety guarantees are feasible for LLM-assisted medical summarization and offer a tunable mechanism for balancing residual risk and review effort.

Abstract

Explore similar work

CARE: A Conformal Safety Layer for Medical Summarization

Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models

Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations