cs.CLMar 12, 2026

LLM-Augmented Therapy Normalization and Aspect-Based Sentiment Analysis for Treatment-Resistant Depression on Reddit

Authors: Yuxin Zhu, Sahithi Lakamana, Masoud Rouhizadeh, Selen Bozkurt, Rachel Hershenberg, Abeed Sarker

Organizations: Department of Biomedical Informatics, Emory University, Atlanta, GA · Department of Pharmaceutical Outcomes and Policy, University of Florida, Gainesville, FL

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Treatment-resistant depression (TRD) is a severe form of major depressive disorder in which patients do not achieve remission despite multiple adequate treatment trials. Evidence across pharmacologic options for TRD remains limited, and trials often do not fully capture patient-reported tolerability. Large-scale online peer-support narratives therefore offer a complementary lens on how patients describe and evaluate medications in real-world use. In this study, we curated a corpus of 5,059 Reddit posts explicitly referencing TRD from 3,480 subscribers across 28 mental health-related subreddits from 2010 to 2025. Of these, 3,839 posts mentioned at least one medication, yielding 23,399 mentions of 81 generic-name medications after lexicon-based normalization of brand names, misspellings, and colloquialisms. We developed an aspect-based sentiment classifier by fine-tuning DeBERTa-v3 on the SMM4H 2023 therapy-sentiment Twitter corpus with large language model based data augmentation, achieving a micro-F1 score of 0.800 on the shared-task test set. Applying this classifier to Reddit, we quantified sentiment toward individual medications across three categories: positive, neutral, and negative, and tracked patterns by drug, subscriber, subreddit, and year. Overall, 72.1% of medication mentions were neutral, 14.8% negative, and 13.1% positive. Conventional antidepressants, especially SSRIs and SNRIs, showed consistently higher negative than positive proportions, whereas ketamine and esketamine showed comparatively more favorable sentiment profiles. These findings show that normalized medication extraction combined with aspect-based sentiment analysis can help characterize patient-perceived treatment experiences in TRD-related Reddit discourse, complementing clinical evidence with large-scale patient-generated perspectives.

Figures & tables

Figure 1: Overview of the data processing pipeline from raw Reddit data to the final analysis sample. Posts were first collected from 28 relevant subreddits (21,826 posts in total). TRD-related post detection yielded 5,059 posts. Filtering by medication mentions resulted in 3,839 posts. These 3,839 posts form the TRD medication-subscriber cohort analyzed in this study. Each number ( nn ) represents the count of posts passing that stage.
CharacteristicValue
Number of TRD posts5,059
Number of unique subscribers3,480
Number of subreddits28
Time spanFeb 7, 2010 – Jul 12, 2025
Posts with at least one medication mention3,839 (75.9%)
Posts with no medication mention1,220 (24.1%)
Table 1: Characteristics of the Reddit TRD cohort and medication annotations.
Figure 2: Text of prompt used for LLM-based lexicon augmentation. {{therapy}} is replaced with the target therapy name, and {{example}} is replaced with in-domain Reddit/Twitter usage examples provided for stylistic guidance. [variant] was the placeholder for an LLM-generated variant.
DatasetNegativeNeutralPositiveTotal
Original train319 (10.6%)2,120 (70.4%)570 (18.9%)3,009
Augmented train1,914 (25.7%)2,120 (28.4%)3,420 (45.9%)7,454
Table 2: Label distribution in the original and augmented Twitter therapy-sentiment training sets used for aspect-based sentiment analysis classification.
Figure 3: Prompt used for LLM-based data augmentation. [text] was the placeholder for the original post, and [tweet] was the placeholder for an LLM-generated variant.
Figure 4: Annual volume and normalized distribution of Reddit posts mentioning treatment-resistant depression and the medication-mention subset, 2010 to 2025. Two cohorts are shown: all TRD posts and TRD posts with ≥\geq 1 medication mention. (A) Annual post volume: lines plot the number of distinct posts per year for each cohort. (B) Normalized annual share within cohort: lines plot each year’s proportion of the cohort, computed as #distinct posts in year y#distinct posts across all years in that cohort×100%\frac{\#\text{distinct posts in year }y}{\#\text{distinct posts across all years in that cohort}}\times 100\% . The vertical dashed marker indicates that 2025 reflects a partial year through the end of data collection (July 2025).
Figure 5: Distribution of the number of distinct medications mentioned per subscriber in the TRD cohort. For each medication-mentioning subscriber ( N=2,700N=2{,}700 ), we counted the number of unique generic (nonproprietary) drug names referenced. The distribution is right-skewed, with most subscribers mentioning fewer than four medications.
Figure 6: Distribution of therapy-mention frequency in the Reddit TRD cohort. The plot shows total mention counts for each of 81 distinct nonproprietary therapy names that appeared at least once in the corpus, after collapsing spelling variants, brand names, and aliases to a single canonical form. Bars correspond to total mentions aggregated across all posts; therapies are ordered from most to least frequently mentioned. The head of the distribution is dominated by rapid-acting glutamatergic/NMDA-targeting therapies (e.g., ketamine and esketamine) and stimulant/ADHD-adjunct therapies (e.g., amphetamine–dextroamphetamine), whereas the remainder forms a long tail spanning diverse antidepressants, augmentation agents, neuromodulation procedures, and other adjunctive therapies.
Figure 7: Subscriber reach of each therapy in the Reddit TRD cohort. For each nonproprietary therapy name, we counted the number of unique subscribers who mentioned it at least once. Horizontal bars show these subscriber counts ( uiu_{i} ). The overlaid line with points (top x-axis) shows the corresponding normalized reach among therapy-mentioning subscribers, pi=ui/Up_{i}=u_{i}/U , where UU is the number of subscribers who mentioned at least one therapy in the cohort. Therapies are ordered from highest to lowest uiu_{i} .
ModelMicro- F1F_{1} (95% CI)Macro-F 1F1F_{1} (negative)F1F_{1} (neutral)F1F_{1} (positive)
BERT-base (baseline)0.758 (0.738, 0.780)0.5440.2250.8580.549
RoBERTa-base (baseline)0.782 (0.762, 0.802)0.6330.4160.8710.612
DeBERTa-v3-base (baseline)0.780 (0.760, 0.801)0.6560.4660.8690.633
BERT-base (augmented)0.768 (0.748, 0.789)0.6190.4200.8600.577
RoBERTa-base (augmented)0.785 (0.765, 0.805)0.6390.4610.8690.587
DeBERTa-v3-base (augmented)0.800 (0.780, 0.820)0.6590.4670.8740.636
Table 3: Performance of transformer-based classifiers on the SMM4H 2023 Task 2 test set (N = 1,602 tweets) before (baseline) and after LLM data augmentation (augmented). Micro-F 1 equals accuracy. Macro-F 1 is the unweighted mean of class-specific F 1 scores.
Figure 8: Therapy-specific sentiment in TRD-focused Reddit discourse. For each normalized medication, the horizontal stacked barshows the proportion of mentions labeled negative , neutral , or positive , summing to 100%. Negative proportions extend left, positive right, with neutral centered. Therapies are ordered by decreasing positive proportion.
Figure 9: Sentiment composition of therapy mentions by therapeutic class in TRD-focused Reddit discourse. Each bar represents the proportion of Reddit mentions within a therapeutic class labeled negative (red, extending left), neutral (orange, centered), or positive (green, extending right). Classes are ordered by descending positive share (top to bottom).
MedicationPositive nnNegative nnp^\hat{p} (Positive)95% CI for p^\hat{p}pFDRp_{\mathrm{FDR}}
Nefazodone1001.000[0.692, 1.000]9.28×10−39.28\times 10^{-3}
Lisdexamfetamine179440.803[0.744, 0.853]1.25×10−181.25\times 10^{-18}
Pregabalin97280.776[0.693, 0.846]6.38×10−96.38\times 10^{-9}
Phenelzine100620.617[0.538, 0.692]1.57×10−21.57\times 10^{-2}
Vilazodone7300.189[0.080, 0.352]1.21×10−31.21\times 10^{-3}
Citalopram11470.190[0.099, 0.314]2.57×10−52.57\times 10^{-5}
Table 4: Selected significant therapy-level binomial tests from classifier predictions.
ClassMedicationMisspellings and Variants
Anxiolytics / Sedativesalprazolamalprazlam, alprazolm, alprzolam, alparzolam, alpraozlam, alprazalam, alprazloam, alprazoalm, xanax bars, zannies, zanz, bars, xanax, xanex
Anxiolytics / Sedativesclonazepamclnazepam, clonazepm, clonazpam, clonzepam, clanazepam, clnoazepam, cloanzepam, clonaezpam, klonies, klons, clonazepam, klonopin, bromazepam, klonipin, clonazapam, clonopin
Anxiolytics / Sedativesdiazepamdazepam, diazepm, diazpam, dizepam, daizepam, deazepam, diaezpam, diazeapm, vallies, diazepam, diazapam, aluminum
Anxiolytics / Sedativeslorazepamlorazepm, lorazpam, lorzepam, lrazepam, larazepam, loarzepam, loraezpam, lorazeapm, lorazepam, atavan
Anxiolytics / Sedativesbuspironebuspiron, buspar, bspirone, buspirne, busprone, bospirone, bsupirone, bupsirone, busiprone, busperone, Buspar, buscopan
Anxiolytics / Sedativeshydroxyzinehydroxyzene, hydroxazine, hydroxyzin, hydroxyzne, hydrxyzine, hdyroxyzine, hidroxizine, hydorxyzine, hydraxyzine, hydroxyizne, Vistaril, Atarax, hydroxyzine
Table S1: The full list of medications included in the Reddit TRD corpus and their misspellings/lexical variants.
CategorySubredditNotes
General postsr/mentalhealth 
General postsr/MentalHealthUK 
General postsr/depressionregimens 
General postsr/depression 
General postsr/mentalillness 
General postsr/depression_help 
Table S2: Subreddits queried for TRD-related data collection. Subreddits are grouped by the categories used during data collection: general mental health/advice forums, medication-focused communities, and comorbid-condition communities. Notes indicate communities flagged as NSFW or professional-perspective forums.
CategoryKeyword/phrase
Core clinical termstreatment-resistant depression
Core clinical termstreatment resistant depression
Core clinical termstreatment-resistant MDD
Core clinical termsdifficult-to-treat depression
Core clinical termsdifficult-to-treat MDD
Abbreviations & related termsTRD
Table S3: Keyword/phrase lexicon used to identify TRD-related posts. Phrases were used as heuristic indicators of TRD contexts, including formal clinical terms, abbreviations, and common colloquial expressions describing repeated treatment failure.
MedicationPositive nnNegative nnNon-neutral nnp^\hat{p} (Positive)95% CI for p^\hat{p}pFDRp_{\mathrm{FDR}}
Lisdexamfetamine179442230.803[0.744, 0.853]1.25×10−181.25\times 10^{-18}
Sertraline521742260.230[0.177, 0.291]5.48×10−155.48\times 10^{-15}
Venlafaxine461582040.225[0.170, 0.289]3.59×10−143.59\times 10^{-14}
Mirtazapine491471960.250[0.191, 0.317]2.72×10−112.72\times 10^{-11}
Pregabalin97281250.776[0.693, 0.846]6.38×10−96.38\times 10^{-9}
Citalopram1147580.190[0.099, 0.314]2.57×10−52.57\times 10^{-5}
Table S4: Therapies with significant asymmetry between positive and negative evaluations among non-neutral mentions (two-sided exact binomial test). The positive share is the proportion of positive mentions among non-neutral mentions; 95% exact binomial confidence intervals are shown. Benjamini–Hochberg adjusted pFDRp_{\mathrm{FDR}} values are reported.
Figure S1: Distribution of TRD-relevant posts across subreddits. Horizontal bars report the number of posts in each of the 28 included subreddits; subreddits are ordered by post count (highest to lowest). Values at bar ends give exact counts, highlighting concentration in general depression forums (e.g., r/depression: 1,079 posts) and therapy-focused communities (e.g., r/TherapeuticKetamine: 607; r/MAIOs: 587).
Figure S2: Subreddit-level volume and contributor breadth of TRD-relevant discussion. For each subreddit, paired bars show the number of TRD-relevant posts (dark) and the number of unique authors contributing those posts (teal), with subreddits ordered by post count. Counts at bar ends provide exact values (e.g., r/depression: 1,079 posts from 905 unique authors), enabling comparison of discussion volume versus concentration among repeat contributors.
Figure S3: Year-by-year composition of medication mentions by therapeutic class in TRD-relevant posts (2010–2025). Each stacked bar corresponds to a calendar year and shows the proportion of medication mentions assigned to each therapeutic class). For each year, the denominator is the total number of medication mentions in that year, and each segment numerator is the number of mentions in that therapeutic class in that same year. Within each year, segments sum to 100%, hence the figure reflects changes in the relative mix of classes over time.
CategoryKeyword/phrase
Core clinical termstreatment-resistant depression
Core clinical termstreatment resistant depression
Core clinical termstreatment-resistant MDD
Abbreviations & related termsTRD
Abbreviations & related termsrefractory depression
Abbreviations & related termsSSRI-resistant depression
Table S5: Keyword/phrase lexicon used to identify TRD-related posts. Phrases were used as heuristic indicators of TRD contexts, including formal clinical terms, abbreviations, and common colloquial expressions describing repeated treatment failure.
Therapy (generic name)Paraphrased first-person example (therapy in bold )LabelP(y^)P(\hat{y})
alprazolamI have tried 16 different antidepressants and failed. I was prescribed alprazolam at age 16 which caused me to develop a dependence (successfully came off that except PRN last year). Last year, my doctor prescribed TMS and Ketamine therapy.Negative0.989
alprazolamdrinking alcohol? no problemo!, you can spend as much money as you want on that, as long as big gov is making some money, pick up some smokes while your at it!, taking some diazepam/ alprazolam ? where is your prescription, you are a drug addict, why are you self medicating, you need a fine, criminal record …Neutral0.989
alprazolamNot exactly where I want to be but an improvement for sure. My Dr prescribed .5 alprazolam for anxiety as needed. I have found it sooo helpful.Positive0.999
amitriptylineHas anyone had a similar experience? or had any luck with amitriptyline ? I just hate feeling like this.Negative0.995
amitriptylineHave you tried the tcas? specifically, amitriptyline , imipramine and clomipramine? 3.Neutral0.990
amitriptylineI’ve wondered why tricyclic medications have never been brought up as a possible choice for me. I do take 25mg of amitriptyline at bedtime for migraine prevention; it has been extremely helpful for this. I have never noticed any adverse effects but I know it’s a low dose.Positive0.999
Table S6: Paraphrased first-person excerpts illustrating mention-level sentiment classifications for each therapy. For each generic-name therapy, we show up to two examples selected as the highest-confidence predictions from the DeBERTa-based classifier (typically one Positive and one Negative ; when only one polarity was available, a Neutral example is shown instead). Excerpts are researcher-generated paraphrases (not verbatim Reddit text) that preserve the original first-person perspective while removing identifying details. Sentiment labels reflect the evaluation of the target therapy in its local context window (the sentence containing the mention with surrounding context; rather than the overall post tone). P(y^)P(\hat{y}) denotes the model’s predicted probability for the assigned label.

Explore similar work

CardsList
Apr 8, 2026

Curation and Extraction of Drug-Related Entities from Reddit Platform

Physicians learn primarily about illicit drugs from clinical overdose cases, limiting their understanding of real-world usage. Meanwhile, drug users share first-hand experiences online, offering insights into dosage and effects of drugs. To bridge this gap, we introduce ReDose (REddit Drug DOSe and Effect), a dataset of 6,435 Reddit posts on substance use. A board-certified toxicologist primarily annotated both the training and test sets, while two medical science students contributed to the test set, labeling DRUG, DOSE, and EFFECT entities. We benchmarked 6,267 annotations using BERT-based, large language model (LLM)-based, and Retrieval-Augmented Generation (RAG) models. BiomedBERT achieved an F1-score of 0.843 for DRUG, while Llama-3 70B outperformed GPT-4 (F1 = 0.79 vs. 0.72). EFFECT extraction remains challenging, with GPT-4 achieving a recall of 0.41. ReDose captures patient-curated narratives to advance medical data extraction from social media.

Zewei Wang, Zihan Xu, Yishu Wei +2

Population Health Sciences, Weill Cornell Medicine, New York City, USA · School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · Emergency Medicine, Weill Cornell Medicine, New York City, USA

Named Entity RecognitionClinical Information ExtractionLLM Information ExtractionSocial Media Analysis
AskPDF
Apr 21, 2026

Depression Risk Assessment in Social Media via Large Language Models

Depression is one of the most prevalent and debilitating mental health conditions worldwide, frequently underdiagnosed and undertreated. The proliferation of social media platforms provides a rich source of naturalistic linguistic signals for the automated monitoring of psychological well-being. In this work, we propose a system based on Large Language Models (LLMs) for depression risk assessment in Reddit posts, through multi-label classification of eight depression-associated emotions and the computation of a weighted severity index. The method is evaluated in a zero-shot setting on the annotated DepressionEmo dataset (~6,000 posts) and applied in-the-wild to 469,692 comments collected from four subreddits over the period 2024-2025. Our best model, gemma3:27b, achieves micro-F1 = 0.75 and macro-F1 = 0.70, results competitive with purpose-built fine-tuned models (BART: micro-F1 = 0.80, macro-F1 = 0.76). The in-the-wild analysis reveals consistent and temporally stable risk profiles across communities, with marked differences between r/depression and r/anxiety. Our findings demonstrate the feasibility of a cost-effective, scalable approach for large-scale psychological monitoring.

Giorgia Gulino, Manuel Petrucci

1Guglielmo Marconi University, Department of Human Sciences, Rome, Italy

Social Media AnalysisDepression DetectionMental Health
AskPDF
Jul 30, 2026

The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials

Depression is a major mental disorder for which diagnosis relies primarily on clinical assessments. Automated methods to support its detection via the psychiatric MADRS scale are getting more and more attention. While existing solutions primarily focus on detecting the disorder from different text sources (e.g., online text, social media), there is still limited support for clinical trials, where clinical assessments are conducted through structured interviews based on standard guidelines such as SIGMA. In this work, we develop a LLM pipeline specifically designed to support clinicians in supporting the assessment of depression in patients enrolled in clinical trials. Our pipeline converts audio interviews into transcripts, maps them into the ten MADRS symptom items, estimates their severity, and identify problematic clinical ratings associated with them. Evaluation on real clinical interviews shows a strong overall correlation of 0.867 with expert ratings, providing interpretable support for future assessments in clinical trials.

Mila Fodor, Katalin Ócsai, Francesco Periti +2

Clario, part of Thermo Fisher Scientific

Depression DetectionClinical Decision SupportDepression Severity EstimationMedical LLMs
AskPDF