Evidence-Anchored Overdose Death Prediction from Death Investigation Reports

Raavi Gupta; Ziqing Yang; Karli R. Hochstatter; Talia Nadel; Nicole D'Anna; Jason K. Graham; Hannah Johnson; Kelsey Opozda

cs.CLOct 14, 2026

Evidence-Anchored Overdose Death Prediction from Death Investigation Reports

Authors: Raavi Gupta, Ziqing Yang, Karli R. Hochstatter, Talia Nadel, Nicole D'Anna, Jason K. Graham, Hannah Johnson, Kelsey Opozda,

Organizations: Columbia University, USA · Friends Research Institute, USA · The New York City Office of Chief Medical Examiner, USA · Independent Researcher, USA ·

Abstract

Confirmation of overdose deaths in the United States is often delayed due to the complexities of certifying drug-involved deaths. Death investigation reports represent a rich but underutilized data source that could be used to predict whether a death is a suspected overdose (i.e., a death preliminarily identified as an overdose prior to the final cause and manner of death determination). We present an automated framework that classifies suspected overdoses from death investigation reports, while producing supporting explanations guided by expert knowledge. Through an expert-guided design process, we investigate 1) what type of explanations are best for model prediction performance, and 2) which architecture is better suited for this task: pipeline models that rationalize first and then predict, or self-rationalizing models that jointly predict and rationalize? Our experiments show that a pipeline model with explanations grounded in the input death investigation reports outperforms the other settings. Our best model also outperforms a rule-based system currently in deployment by 15%, while allowing expert interventions for final decisions.

Figures & tables

Figure 1: The two explanation types on an example death investigation report of mock data (completely deidentified). Feature-verbalizations

R

(top) are predefined statements per expert-identified feature; feature-guided rationales

R_{G}

(highlighted) are verbatim sentences extracted from the report corresponding to the given expert-identified feature.

Figure 2: Overview of the proposed method. We compare two types of explanations: feature-guided rationales

R_{G}

(sentences extracted from the report) and feature-verbalizations

R

(predefined statements for each present feature). A classifier model then predicts the overdose label

O

from the explanation, together with the report. We show here the decoupled regime, in which explanation generation and prediction are two separate steps rather than a single joint generation.

Partition	Set	No	Yes	Total
Feature-annotated	Train	786	3,697	4,483
	Val	95	457	552
	Test	96	449	545
Weakly-annotated	Train	1,828	1,099	2,927
	Val	229	145	374
	Test	233	149	382

Table 1: Overdose label distribution across the Feature-annotated and Weakly-annotated partitions. Yes and No denote whether the death was certified as an overdose.

		$I\rightarrow O$		$\ R_{G}\rightarrow O$		$I,R_{G}\rightarrow O$		$I\rightarrow R,O$		$I,R\rightarrow O$
Split	Model	Acc.	Macro-F1	Acc.	Macro-F1	Acc.	Macro-F1	Acc.	Macro-F1	Acc.	Macro-F1
Overall	Llama-3.1	0.832	0.811	0.798	0.778	0.847	0.829	0.809	0.779	0.832	0.812
Overall	Qwen3	0.789	0.766	0.786	0.765	0.849	0.831	0.799	0.768	0.794	0.776
Feature-annotated	Llama-3.1	0.851	0.674	0.828	0.656	0.866	0.728	0.837	0.619	0.842	0.639
Feature-annotated	Qwen3	0.820	0.637	0.818	0.632	0.873	0.738	0.829	0.597	0.828	0.605
Weakly-annotated	Llama-3.1	0.804	0.791	0.757	0.736	0.819	0.809	0.770	0.762	0.817	0.801

Table 2: Comparison of all variants across test splits.

I\rightarrow O

: direct prediction from report.

\ R_{G}\rightarrow O

: pipeline model with just feature-guided rationale as input for prediction.

I,R_{G}\rightarrow O

: pipeline model with feature-guided rationales and report as input.

I\rightarrow R,O

: self-rationalized model with feature verbalization as explanations.

I,R\rightarrow O

: pipeline model with report and feature verbalization as explanations as input for prediction. Bold = best configuration per row.

Variable	w/ Feedback	w/o Feedback	% Improv.
History of Prior Overdose(s)	0.5882	0.5195	+13.2
S.U.H.: Benzodiazepine(s)	0.5185	0.2632	+97.0
S.U.H.: Cocaine (including Crack)	0.8673	0.8689	$-$ 0.2
S.U.H.: Injection Drug Use	0.5116	0.4835	+5.8
S.U.H.: Ketamine	1.0000	1.0000	0.0
S.U.H.: Opioid(s)	0.7820	0.7070	+10.6

Table 3: F1 scores for sentence-level classification without and with expert feedback integration. % Improvement is computed as

\frac{\text{w/ Feedback}-\text{w/o Feedback}}{\text{w/o Feedback}}\times 100

. "S.U.H." denotes "Substance Use History" and "D/P" denotes "Drugs/Paraphernalia".

	(a) Correct rejection	(b) Missed suspected overdose	(c) Extraction error
	Gold: No Pred: No ✓	Gold: Yes Pred: No $\times$	Gold: Yes Pred: No $\times$
Evidence	$\bullet$ person with PMHx of CAD, HTN, DM, COPD, remote h/o drug use , depression and schizophrenia; found down in a shelter. $\bullet$ The only drug use she is aware of is marijuana use in their teens–20s .	$\bullet$ Drank socially; family denied any known illicit drug use. $\bullet$ A cut red straw and a blue pill (imprint B706) found on a nearby table. $\bullet$ Powder, appearing to be crushed pills , in the clear plastic wrapper of a cigarette box.	$\bullet$ No illicit drugs were found. $\bullet$ Family stated the decedent was not known to use drugs or alcohol.
$\hat{v}_{k}$	History of substance use: 1 Drug paraphernalia: 0	History of substance use: 0 Paraphernalia at scene: 1 Loose/non-prescribed pills: 1	History of substance use: 0 Paraphernalia at scene: 0 Loose/non-prescribed pills: 0
Analysis	The model correctly classifies this case as a non-overdose. A history-of-substance-use flag is present, but the extracted spans reveal that the “remote h/o drug use” refers specifically to marijuana. Since marijuana is excluded from the substance use history in the overdose prediction problem due to low risk, this is a false annotation that is correctly rejected using the additional context from the extracted evidence.	The model fails to classify this case as a suspected overdose despite extracting substantial scene evidence: drug packaging (a clear plastic wrapper), a loose pill not prescribed to the decedent, a cut straw, and powder/residue. The rationale is accurate, but the classifier does not weigh the accumulated scene indicators strongly enough against the family’s denial of drug use.	Here the error originates in the extraction stage: the model extracts misleading spans, which propagate to an incorrect classification. This failure mode is specific to the decoupled paradigm and marks the point where expert review of the rationale prior to prediction would intervene and help in downstream classification.

Table 4: Qualitative analysis of three representative cases under the decoupled paradigm. Evidence spans are extracted verbatim and abridged here. Cases (a) and (b) show correct extraction of evidence; case (c) shows failure propagating from the extraction stage.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Variable	Pos H	Pos AI	TP	FP	FN	TN	Recall	$\boldsymbol{\kappa}$
Substance use history
History of prior overdose(s)	45	41	41	0	4	500	0.911	0.950
Benzodiazepine(s)	25	22	22	0	3	520	0.880	0.933
Cocaine (including Crack)	173	173	173	0	0	372	1.000	1.000
Injection drug use	48	47	47	0	1	497	0.979	0.988
Ketamine	2	2	2	0	0	543	1.000	1.000

Appendix

Table 5: Feature extraction accuracy for on Feature-annotated test split. H = human; AI = our system

Panel of Figure 3 — Figure 3: Prompt used for five-shot feature indicator extraction with Llama-3.1-8B-Instruct.

Variable	Question ( $q_{k}$ )
History of Prior Overdose(s)	Is there any mention of a prior overdose?
Benzodiazepine(s)	Is there any reported substance use history of benzodiazepines?
Cocaine (including crack)	Is there any reported substance use history of cocaine (including crack)?
Injection Drug Use	Is there any reported substance use history of injection drug use (answer No if only the word paraphernalia is present without any further evidence)?
Ketamine	Is there any reported substance use history of ketamine?
Opioid(s)	Is there any reported substance use history of opioid(s)?

Appendix

Table 6: Questions used to extract each feature variable from the MLI report. Each question is substituted into the {feature_question} slot of the prompt template.

Feature	Statement ( $s_{k}$ )
History of Prior Overdose(s)	There is mention of a prior overdose.
Benzodiazepine(s)	There is reported substance use history of benzodiazepines.
Cocaine (including Crack)	There is reported substance use history of cocaine.
Injection Drug Use	There is reported substance use history of IV injection drugs.
Ketamine	There is reported substance use history of ketamine.
Opioid(s)	There is reported substance use history of opioid(s).

Appendix

Table 7: Predefined natural-language statements mapped to each of the feature variables, used to construct the feature-verbalized explanations

R

Panel of Figure 4 — Figure 4: Prompt used for extracting feature-guided rationales via question answering in teacher models.

Variable	MedGemma and Qwen3	MedGemma and Llama	Qwen3 and Llama	All Annotators
History of Prior Overdose(s)	0.406112	0.656521	0.391848	0.484827
Substance Use History: Benzodiazepine(s)	0.251875	0.423022	0.265501	0.313466
Substance Use History: Cocaine (including Crack)	0.326150	0.562726	0.347551	0.412142
Substance Use History: Injection Drug Use	0.131958	0.404878	0.104612	0.213816
Substance Use History: Ketamine	0.297646	0.620090	0.324882	0.414206
Substance Use History: Opioid(s)	0.285187	0.504151	0.319806	0.369715

Appendix

Table 8: Cohen’s

\kappa

agreement between teacher models for sentence-level evidence extraction across 18 forensic variables. Values represent pairwise and three-way agreement in the silver dataset construction.

		Llama-70B			MedGemma-27B			Qwen3-30B
Variable	Pos H	Pos AI	Recall	$\kappa$	Pos AI	Recall	$\kappa$	Pos AI	Recall	$\kappa$
Substance use history
Benzodiazepine(s)	211	182	0.863	0.924	176	0.834	0.906	120	0.569	0.717
Cocaine (including Crack)	1609	1595	0.991	0.994	1564	0.972	0.980	1295	0.805	0.854
Injection drug use	523	496	0.948	0.971	396	0.757	0.850	140	0.268	0.399
Ketamine	27	27	1.000	1.000	26	0.963	0.981	20	0.741	0.850

Appendix

Table 9: Per-variable extraction accuracy of three teacher models against human-annotated labels across all Feature-annotated cases (

n=5{,}580

; train / val / test = 4,483 / 552 / 545). A variable is predicted positive if the model’s extracted span list is non-empty. PPV = 1.000 for all variables and all models (zero false positives) and is omitted for brevity. Cohen’s

\kappa

is computed against human labels as ground truth.

Type	Error type	Description
Over-extraction (111)	On-scene evidence as history	Scene items cited as a history of use
	Prescription as misuse	Prescribed medications or medication lists cited as misuse
	Generic mention as a specific drug	Vague mentions of drug use mapped to a specific substance
	Neighboring scene category	Scene item assigned to an adjacent scene feature
	Indirect clinical sign	Clinical signs or hospital visits cited as direct evidence
	Negation or out-of-scope substance	Denials, abstinence, tobacco, or marijuana cited as evidence

Appendix

Table 10: Types of expert feedback on feature-guided rationales

R_{G}

extracted by the student model. Numbers in parentheses give the count of edited samples per type.

Input	ClinicalModernBERT	DeepSeek
$I\rightarrow O$	0.78	0.82
$R_{G}\rightarrow O$	0.71	0.77
$I,R_{G}\rightarrow O$	0.81	0.83

Appendix

Table 11: Macro-F1 of additional prediction models.

Test subset	Macro-F1
Test subset	Exp. 1	Exp. 2	Exp. 3
Feature-annotated	0.552	0.561	0.523
Weakly-annotated	—	0.485	0.478
All	—	0.545	0.621

Appendix

Table 12: Macro-F1 for logistic regression baselines across evidence variable source and training distribution. Exp. 1: Feature-annotated train samples trained with human evidence variables. Exp. 2: Feature-annotated train samples trained with model-extracted variables. Exp. 3: Entire train split trained with model-extracted variables. Dashes indicate evaluations unavailable by construction.

Healthcare

Healthcare AI develops and evaluates models and systems for clinical care, medical research, public health, and healthcare operations.

See all 616 papers

OverdoseMoE: A Multi-Expert Framework for Opioid Overdose Risk Prediction
Sep 30, 2026Mingchen Li, Rohan Pandey, Junhui Qian +3Mixture of Experts Longitudinal EHR Modeling
Manning College of Information and Computer Sciences, UMass Amherst, Amherst, MA, USA · Center for Healthcare Organization and Implementation Research, VA Bedford Health Care, Bedford, MA, USA · Miner School of Computer and Information Sciences, UMass Lowell, Lowell, MA, USA
Large language models can disambiguate opioid slang on social media
Mar 11, 2026Kristy A. Carpenter, Issah A. Samori, Mathew V. Kiang +4Word Sense Disambiguation Social Media Analysis
Departments of 1Biomedical Data Science, Stanford University, Stanford, CA 94305, USA · Departments of 2Bioengineering, Stanford University, Stanford, CA 94305, USA · Departments of 3Epidemiology and Population Health, Stanford University, Stanford, CA 94305, USA +5
Social Determinants of Health and Fentanyl Overdose Mortality Across US Counties: An XGBoost and SHAP Analysis Identifying Silent Risk Counties and Treatment Deserts
May 6, 2026Kabi Raj Tiruwa, Abhisan Ghimire, Anuj Kumar ShahMortality Prediction
Clark University · Clark University, 910 Main St, Worcester, 01610, MA, USA. · Yeshiva University +1

Sep 30, 2026J/K move · Enter open · S save

OverdoseMoE: A Multi-Expert Framework for Opioid Overdose Risk Prediction

Mingchen Li, Rohan Pandey, Junhui Qian, Feiyun Ouyang+2

Manning College of Information and Computer Sciences, UMass Amherst, Amherst, MA, USA · Center for Healthcare Organization and Implementation Research, VA Bedford Health Care, Bedford, MA, USA · Miner School of Computer and Information Sciences, UMass Lowell, Lowell, MA, USA

Mixture of Experts Longitudinal EHR Modeling Medical LLMs Clinical Risk Prediction

Open paper AskPDF arXiv

Opioid overdose remains a major clinical and public health burden, highlighting the need for scalable approaches to identify patients at high risk. Here, we investigate diagnosis-specific adaptation for 180-day opioid overdose risk prediction from patients' preceding one-year longitudinal ICD histories. We develop OODMAMBA and OODQWEN through continued pretraining on longitudinal diagnostic sequences followed by task-specific fine-tuning. Building on the stronger Qwen-based predictors, we further propose OVERDOSEMOE, a multi-expert framework that integrates models of different scales using complementary expert-weighting strategies. Diagnosis-specific adaptation consistently improved predictive performance over general-purpose language-model baselines, with OODQWEN achieving an AUPRC of 24.47 and an AUROC of 68.56. OVERDOSEMOE further improved discrimination and precision, achieving an AUPRC of 25.17 and an AUROC of 69.49 while outperforming the strongest single-model baselines. Among patients ranked in the top 5% of predicted risk, OVERDOSEMOE identified substantially enriched overdose risk, achieving a PPV of 25.38% while retaining meaningful recall. Evaluation on an independent MIMIC-IV cohort further demonstrated cross-cohort robustness, with complementary weighting strategies showing advantages across different performance measures. These findings demonstrate that diagnosis-specific language-model adaptation combined with multi-expert integration can improve opioid overdose risk stratification and support more robust prediction across heterogeneous electronic health record populations.

Abstract

Healthcare

Explore similar work

OverdoseMoE: A Multi-Expert Framework for Opioid Overdose Risk Prediction

Large language models can disambiguate opioid slang on social media

Social Determinants of Health and Fentanyl Overdose Mortality Across US Counties: An XGBoost and SHAP Analysis Identifying Silent Risk Counties and Treatment Deserts