Curation and Extraction of Drug-Related Entities from Reddit Platform

Zewei Wang; Zihan Xu; Yishu Wei; Michael Chary; Yifan Peng

cs.CLApr 8, 2026

Curation and Extraction of Drug-Related Entities from Reddit Platform

Authors: Zewei Wang, Zihan Xu, Yishu Wei, Michael Chary, Yifan Peng

Organizations: Population Health Sciences, Weill Cornell Medicine, New York City, USA · School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · Emergency Medicine, Weill Cornell Medicine, New York City, USA

Abstract

Physicians learn primarily about illicit drugs from clinical overdose cases, limiting their understanding of real-world usage. Meanwhile, drug users share first-hand experiences online, offering insights into dosage and effects of drugs. To bridge this gap, we introduce ReDose (REddit Drug DOSe and Effect), a dataset of 6,435 Reddit posts on substance use. A board-certified toxicologist primarily annotated both the training and test sets, while two medical science students contributed to the test set, labeling DRUG, DOSE, and EFFECT entities. We benchmarked 6,267 annotations using BERT-based, large language model (LLM)-based, and Retrieval-Augmented Generation (RAG) models. BiomedBERT achieved an F1-score of 0.843 for DRUG, while Llama-3 70B outperformed GPT-4 (F1 = 0.79 vs. 0.72). EFFECT extraction remains challenging, with GPT-4 achieving a recall of 0.41. ReDose captures patient-curated narratives to advance medical data extraction from social media.

Figures & tables

Dataset Name	Source	Annotated Drugs	Open source	Social media
Graves et al. [ 3 ]	Public posts from the /r/suboxone subreddit of Reddit	Buprenorphine–naloxone	$\times$	$\checkmark$
Leas et al. [ 13 ]	r/CBD (2014–2019)	Cannabidiol	$\times$	$\checkmark$
Spadaro et al. [ 10 ]	r/fentanyl, r/heroin, r/microdosing, r/opiates, r/OpiatesRecovery, r/OurOverUsedVeins, r/suboxone	Buprenorphine and fentanyl	$\times$	$\checkmark$
BC5CDR [ 12 ]	PubMed Articles	Chemicals	$\checkmark$	$\times$
ADE [ 11 ]	MIMIC–III clinical care database	Prescription Medication	$\checkmark$	$\times$
Reddit–Impacts [ 14 ]	Opioid–related subreddits on Reddit	None (impact only)	$\times$	$\checkmark$

Table 1: Summary of Datasets in Related Studies

Subreddit	Description	#Posts
r/fentanyl	Subreddit for the discussion of fentanyl and its many analogues. We focus on harm reduction, and hope to dispel some common myths about these substances.	10,816
r/heroin	For the junkies and ex-junkies of Reddit: A subreddit for all things heroin. Topics include heroin appreciation, harm reduction, withdrawal, recovery, lifestyle discussions, and more.	31,322
r/microdosing	This is a community for discussion pertaining to microdosing research, experiments, regimens, and experiences. The most probable candidates for microdosing are psychedelics, but we encourage dialogue on the effects of any drugs at sub-threshold dosage. No sourcing of drugs allowed! Please have a look at the r/microdosing Sidebar.	42,766
r/opiates	Discussion of all things related to the narcotics known as opiates, from harm-reduction to pharmacology. We are participating in the blackout! We will not be adding approved users to the community as we will return to normal functions tomorrow	62,138
r/OpiatesRecovery	You are not alone in this fight. We are a group of people dedicated to helping each other stop and stay stopped.	76,888
r/OurOverUsedVeins	For those of us with an appreciation for heroin, to soothe, help, and give information on the subject of heroin use, and getting past our heroin addictions. Also, share your true stories! As heroin addicts, we’ve been in more terrifying situations than most people. OurOverUsedVeins wants to hear about them!	79,581

Table 2: Summary of the 7 Subreddits

Figure 1: A sample annotation with three types of entities.

Figure 3: Creation of the ReDose dataset.

	Training	Test	Total
Document	6,068	367	6,435
Sentence	14,260	1,209	15,469
Entity
DRUG	4,004	833	4,784
EFFECT	674	128	750
DOSE	647	98	733

Table 3: Description of ReDose .

BaseBERT
	Precision	Recall	F1-score
DRUG	0.891 (0.889, 0.893)	0.873 (0.871, 0.875)	0.882 (0.880, 0.884)
EFFECT	0.424 (0.411, 0.437)	0.231 (0.222, 0.240)	0.298 (0.288, 0.308)
DOSE	0.649 (0.640, 0.658)	0.752 (0.742, 0.761)	0.696 (0.688, 0.705)
micro avg	0.843 (0.840, 0.845)	0.810 (0.808, 0.812)	0.826 (0.824, 0.829)
BioBERT

Table 4: Performance Comparison for BERT-based models.

	Precision	Recall	F1-score
Llama-3 Instruct 8B
DRUG	0.71	0.44	0.55
EFFECT	0.03	0.11	0.05
micro avg	0.50	0.41	0.45
Llama-3 Instruct 70B
DRUG	0.74	0.84	0.79

Table 5: Performance Comparison for LLMs.

Explore similar work

RexDrug: Reliable Multi-Drug Combination Extraction through Reasoning-Enhanced LLMs
Mar 9, 2026Zhijun Wang, Ling Luo, Dinghao Pan +4Relation Extraction Clinical Information Extraction
School of Computer Science and Technology, Dalian University of Technology, No.2 Linggong Road, Ganjingzi District, Liaoning 116024, China · Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, No.44 Xiaoheyan Road, Dadong District, Liaoning 110042, China
LLM-Augmented Therapy Normalization and Aspect-Based Sentiment Analysis for Treatment-Resistant Depression on Reddit
Mar 12, 2026Yuxin Zhu, Sahithi Lakamana, Masoud Rouhizadeh +3Mental Health Sentiment Analysis
Department of Biomedical Informatics, Emory University, Atlanta, GA · Department of Pharmaceutical Outcomes and Policy, University of Florida, Gainesville, FL
Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking
Jun 24, 2026Huizi Yu, Jian Liu, Wenkong Wang +10KG Construction Healthcare
Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · Department of Psychiatry, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · School of Control Science and Engineering, Shandong University, Ji’nan, Shandong, China +5

Mar 9, 2026J/K move · Enter open · S save

RexDrug: Reliable Multi-Drug Combination Extraction through Reasoning-Enhanced LLMs

Zhijun Wang, Ling Luo, Dinghao Pan, Huan Zhuang+3

School of Computer Science and Technology, Dalian University of Technology, No.2 Linggong Road, Ganjingzi District, Liaoning 116024, China · Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, No.44 Xiaoheyan Road, Dadong District, Liaoning 110042, China

Relation Extraction Clinical Information Extraction

Open paper AskPDF arXiv

Automated Drug Combination Extraction (DCE) from large-scale biomedical literature is crucial for advancing precision medicine and pharmacological research. However, existing relation extraction methods primarily focus on binary interactions and struggle to model variable-length n-ary drug combinations, where complex compatibility logic and distributed evidence need to be considered. To address these limitations, we propose RexDrug, an end-to-end reasoning-enhanced relation extraction framework for n-ary drug combination extraction based on large language models. RexDrug adopts a two-stage training strategy. First, a multi-agent collaborative mechanism is utilized to automatically generate high-quality expert-like reasoning traces for supervised fine-tuning. Second, reinforcement learning with a multi-dimensional reward function specifically tailored for DCE is applied to further refine reasoning quality and extraction accuracy. Extensive experiments on the DrugComb dataset show that RexDrug consistently outperforms state-of-the-art baselines for n-ary extraction. Additional evaluation on the DDI13 corpus confirms its generalizability to binary drugdrug interaction tasks. Human expert assessment and automatic reasoning metrics further indicates that RexDrug produces coherent medical reasoning while accurately identifying complex therapeutic regimens. These results establish RexDrug as a scalable and reliable solution for complex biomedical relation extraction from unstructured text. The source code and data are available at https://github.com/DUTIR-BioNLP/RexDrug

Abstract

Explore similar work

RexDrug: Reliable Multi-Drug Combination Extraction through Reasoning-Enhanced LLMs

LLM-Augmented Therapy Normalization and Aspect-Based Sentiment Analysis for Treatment-Resistant Depression on Reddit

Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking