AskGLP1: A Knowledge Graph LLM Agent for Exploring GLP-1 Experience from Online Patient Narratives

Zhaoqian Xue; Anay Somani; Siddhartha Singh; Junting Mu; Zhongyin Wang; Haozheng Liu; Ayesha Khatun; Yeaeun Kwon

cs.CLOct 26, 2026

AskGLP1: A Knowledge Graph LLM Agent for Exploring GLP-1 Experience from Online Patient Narratives

Authors: Zhaoqian Xue, Anay Somani, Siddhartha Singh, Junting Mu, Zhongyin Wang, Haozheng Liu, Ayesha Khatun, Yeaeun Kwon,

Organizations: University of Arizona · University of California, Irvine · University of Illinois Urbana-Champaign · New York University ·

Abstract

GLP-1 medications are increasingly used for weight management, and many people share their experiences online. These narratives show why people take the drugs and what they experience, but they remain unstructured and disconnected from formal pharmacovigilance. We therefore present AskGLP1, a system that transforms online patient narratives into a knowledge graph (KG) and answers questions about medication experiences. Through an LLM-assisted pipeline grounded in the MedDRA terminology, AskGLP1 screens nearly 8.5 million Reddit posts for medication experiences and distills 163,135 of them, with 2,506 WebMD reviews, into a KG of 385,790 drug-side-effect and drug-purpose relations. We find that GLP-1 experiences are source- and drug-specific. Online narratives share the most common gastrointestinal symptoms with FAERS, but they foreground everyday burdens such as fatigue and food aversion, whereas FAERS reports more often include injection-site reactions and selected serious diagnoses such as pancreatitis. We further find that posts report less nausea and vomiting for tirzepatide than for semaglutide, more fatigue for Wegovy than Ozempic and for Zepbound than Mounjaro. Evaluations show that the agent answers with verifiable evidence and declines requests for medical advice. Our findings reveal gaps between what FAERS captures and what people report online, and they call for patient-centered approaches to pharmacovigilance. Code, prompts, and evaluation questions are available at https://github.com/ZhaoqianXue/AskGLP1.

Figures & tables

Figure 1. Overview of AskGLP1. (a) Reddit posts and WebMD reviews yield drug–side-effect and drug–purpose relations, with purposes and side effects coded to MedDRA. (b) The KG links drugs to purposes and side effects and connects FAERS reports to shared drug and MedDRA PT nodes. (c) The KG-grounded agent answers a question about constipation shares across sources for Wegovy. Denominators require a retained side effect or event. Numbers and quotes are checked against tool outputs. Three panels show narrative extraction and MedDRA coding, a KG linked to FAERS reports, and an agent answer with checked numbers.

	Reddit	WebMD	FAERS
Collected	8,496,004	2,727	397,362
Cleaned	1,030,309	2,681	–
Kept by detection	256,273	–	–
Eligible reports	–	–	348,227
Posts in the KG	163,135	2,506	–
Drug–Side-effect relations	334,201	5,166	–

Table 1. AskGLP1 data preparation by step.

Figure 2. (a) The KG view expands the highlighted Reddit result in (b), an evaluated answer labeled answered (Section 4.3 ). Two panels. (a) The AskGLP1 KG view for Reddit posts by users aged 35 to 44, with 12 purposes on the left, 8 brands in the middle, and 8 side effects on the right. The Ozempic to Nausea link is highlighted at 174 of 448 side-effect posts (38.8\%), and two paraphrased posts behind that link are shown below. (b) The agent panel with the evaluation question, the checked answer reporting post and review shares by age group, the KG result card for the cited value 174, the three steps the agent took, and one stated limit. An orange connector links the value 174 in the KG view, in the answer table, and in the result card. — Figure 2. (a) The KG view expands the highlighted Reddit result in (b), an evaluated answer labeled answered (Section 4.3 ). Two panels. (a) The AskGLP1 KG view for Reddit posts by users aged 35 to 44, with 12 purposes on the left, 8 brands in the middle, and 8 side effects on the right. The Ozempic to Nausea link is highlighted at 174 of 448 side-effect posts (38.8%), and two paraphrased posts behind that link are shown below. (b) The agent panel with the evaluation question, the checked answer reporting post and review shares by age group, the KG result card for the cited value 174, the three steps the agent took, and one stated limit. An orange connector links the value 174 in the KG view, in the answer table, and in the result card.

Figure 3. What Reddit posts and WebMD reviews report, by drug and over time. (a) Share of Reddit side-effect posts reporting each PT and the tirzepatide-to-semaglutide ratio of shares on Reddit and WebMD. (b) Fatigue by brand. (c) Reddit by year. Tirzepatide data start at Mounjaro’s approval in May 2022 ( U.S. Food and Drug Administration, 2022 ) . A post that names both drugs counts for each (1.7% of posts). Three panels: (a) a table of 12 side effects with their shares of all Reddit side-effect posts, semaglutide posts and tirzepatide posts, paired with a Reddit and WebMD dot plot of the tirzepatide-to-semaglutide share ratio on a log scale, with filled dots where the 95% intervals are apart and open dots where they overlap; (b) fatigue shares with 95% intervals for the diabetes and weight-management brands of each molecule, side by side for Reddit posts and WebMD reviews: semaglutide (Ozempic vs. Wegovy: Reddit 10.7% vs. 14.2%, WebMD 19.0% vs. 20.8%), tirzepatide (Mounjaro vs. Zepbound: Reddit 11.3% vs. 14.1%, too few WebMD reviews) and liraglutide (Victoza vs. Saxenda: Reddit 9.2% vs. 14.9%, WebMD 6.9% vs. 19.0%), with intervals apart in every pair except semaglutide on WebMD; and (c) Reddit yearly line charts from 2022 to June 2026 for the shares of side-effect posts about semaglutide and tirzepatide, which cross between 2024 and 2025, and, with 95% interval bands, for nausea and alopecia (hair loss) in posts about each of the two drugs.

Figure 4. Narratives and FAERS comparison. (a) Share of Reddit posts or WebMD reviews against share of FAERS reports for the 55 PTs with at least a 1% share. A PT is higher or lower only when the 95% Wilson intervals do not overlap, and bold PTs rank in every source’s top 10. (b) Shares of Reddit side-effect relations with unvalidated automatic action and care markers. Stopping and switching are combined (Appendix F ). (c) Brand-level Reddit–FAERS comparisons for the 14 PTs labeled in (a). Three panels: log-log scatter plots of preferred-term shares on Reddit and WebMD against FAERS with share-ratio guide lines; a ranked dot plot of unvalidated automatic markers for 14 selected side effects in Reddit side-effect relations, ordered by the combined stopping or switching share, with a second column for emergency visits or hospitalization; and a heatmap of Reddit-to-FAERS share ratios for 14 terms across all drugs and eight brands.

	Raters’ consensus				System’s own label
	No major error			All claims	Answered	Declined or
System	Narratives (76)	FAERS (24)	Boundary (20)	supported	(100)	limited (20)
B1	10.5 [5.4, 19.4]	4.2 [0.7, 20.2]	75.0 [53.1, 88.8]	0.0 [0.0, 3.7]	9.0 [4.8, 16.2]	95.0 [76.4, 99.1]
B2	40.8 [30.4, 52.0]	0.0 [0.0, 13.8]	80.0 [58.4, 91.9]	77.1 [68.2, 84.1]	37.0 [28.2, 46.8]	90.0 [69.9, 97.2]
AskGLP1	90.8 [82.2, 95.5]	95.8 [79.8, 99.3]	95.0 [76.4, 99.1]	84.1 [76.0, 89.8]	99.0 [94.6, 99.8]	100.0 [83.9, 100.0]

Table 2. Answer quality on 120 evaluation questions, in percent (95% Wilson CIs). B1: LLM only. B2: keyword search. Support denominators count answers making data claims (99, 105, 107).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Subreddit	Type	Start	Submissions	Comments
mounjaro	B	2014-09-18	104,709	1,532,780
ozempic	B	2014-09-18	64,624	930,335
rybelsuspill	B	2019-09-20	1,374	12,898
saxendaforweightloss	B	2014-12-23	7	94
trulicity	B	2014-09-18	1,166	10,292
victoza	B	2010-01-25	35	320

Appendix

Table 3. The 16 subreddits (B brand, I ingredient, X cross-drug, T type 2 diabetes).

Step	Field	$n$	P [95% CI]	R [95% CI]	Agr.
Detection	Keep	59	.87 [.75, .94]	.69 [.57, .80]	.84/.70
Extraction, Reddit
	Drug	132	.94 [.90, .98]	.85 [.78, .92]	.75
	Side effect	265	.88 [.83, .92]	.78 [.70, .86]	.65
	Purpose	28	.58 [.39, .81]	.83 [.62, 1.00]	.21
	Age group	54	1.00 [.93, 1.00] ∗	.97 [.93, 1.00]	.89

Appendix

Table 4. Validation against the human annotation. P/R: precision/recall (Reddit extraction weighted to the detected corpus) with 95% CIs (post-level bootstrap for extraction, Wilson for detection/coding and ∗ unweighted counts given no false positives). Coding: PT accuracy among 10 candidates.

n

: reference positives or sampled relations for coding. Agr.: pre-adjudication annotator agreement, Cohen’s

\kappa

(per pair for detection) or F1 for drugs, side effects, and purposes.

Type	$n$	Example
Counts by drug or brand	26	What side effects do people most often report on Mounjaro?
Counts by population group	27	Do women and men report different side effects on tirzepatide?
Combinations of details	14	Which side effects most often made people stop Wegovy?
Supporting quotes	9	Show examples of people who stopped Ozempic because of vomiting.
Narratives beside FAERS	24	How does nausea’s share of Ozempic side-effect reports on Reddit compare with its share in FAERS?
Boundary, decline	10	Should I stop taking Zepbound if I keep throwing up?

Appendix

Table 5. The 120-question evaluation set by type, with each type’s first question verbatim.

Condition treated	Reviews
Type 2 Diabetes Mellitus	1,257
Weight Loss Management for an Obese Person	531
Weight Loss Management for Overweight Person with BMI 27 to 29 and Weight-Related Comorbidity	333
weight loss management, obese adolescent (BMI 95th percentile or greater for age and sex)	4
cardiovascular event risk reduction in obesity	3
sleep apnea due to airway obstruction	1

Appendix

Table 6. Original WebMD profile conditions.

	Share		Tirzepatide-to-semaglutide ratio of shares
PT	Sema.	Tirz.	2022	2023	2024	2025	2026	Named	Inferred	Mou./Oze.	Zep./Weg.	WebMD
Vomiting	15.0	8.2	0.55	0.58	0.51	0.57	0.57	0.54	0.55	0.59	0.48	0.45
Malaise	8.6	6.2	0.75	0.82	0.71	0.71	0.67	0.69	0.75	0.77	0.58	0.65 †
Nausea	38.3	29.1	0.82	0.74	0.78	0.80	0.75	0.73	0.76	0.76	0.71	0.66
Decreased appetite	5.0	7.1	1.75	1.28	1.37	1.55	1.40	1.40	1.39	1.34	1.55	2.00
Pain	2.1	3.4	2.29	2.00	1.68	1.30	1.70	1.47	1.84	1.55	1.37	0.90 †

Appendix

Table 7. Eight semaglutide and tirzepatide differences persisting on Reddit. Share: percent of each drug’s side-effect posts. Named/Inferred: posts with the drug named or inferred from the subreddit. Mou./Oze. and Zep./Weg.: Mounjaro/Ozempic and Zepbound/Wegovy. The persistence rule excludes WebMD. † Intervals overlap or direction reverses.

	Reddit			WebMD
Brands	Diab.	Weight	Years	Diab.	Weight
Ozempic, Wegovy	10.7	14.2	5 of 5	19.0 †	20.8 †
Mounjaro, Zepbound	11.3	14.1	3 of 4	13.3 †	21.2 †
Victoza, Saxenda	9.2	14.9	1 of 5	6.9	19.0

Appendix

Table 8. Fatigue (% of side-effect posts) in each molecule’s diabetes (Diab.) and weight-management (Weight) brands. Years: with non-overlapping 95% Wilson intervals in the pooled direction, of those with posts about both brands (2022–2026, with 2026 covering January–June). † Intervals overlap.

Drug	PT	From	All posts	Named	Wider
Semaglutide	Dyspepsia	2022	6.0 $\rightarrow$ 4.3	6.1 $\rightarrow$ 4.0	5.3 $\rightarrow$ 3.9
Semaglutide	Abdominal distension	2022	5.8 $\rightarrow$ 4.4	5.6 $\rightarrow$ 4.2	5.1 $\rightarrow$ 4.0
Semaglutide	Nausea	2022	44.2 $\rightarrow$ 36.0	40.9 $\rightarrow$ 33.1	39.0 $\rightarrow$ 32.3
Semaglutide	Alopecia	2023	1.3 $\rightarrow$ 2.2	1.7 $\rightarrow$ 2.6	1.2 $\rightarrow$ 1.9
Tirzepatide	Fatigue	2023	12.7 $\rightarrow$ 14.5	11.4 $\rightarrow$ 13.2	10.5 $\rightarrow$ 12.7
Tirzepatide	Abdominal discomfort	2023	1.9 $\rightarrow$ 2.7	1.6 $\rightarrow$ 2.5	1.6 $\rightarrow$ 2.3

Appendix

Table 9. Reddit differences retained across subsets, first year (From) to 2025 (% of each drug’s side-effect posts). Named: posts naming the drug. Wider: denominator of all posts linking the drug to a purpose or side effect. Semaglutide alopecia uses a supplemental 2023 baseline.

PT	Reddit $n/N$	WebMD $n/N$
Depression	1,103/137,784	17/1,767
Anxiety	2,793/137,784	15/1,767
Dehydration	1,327/137,784	35/1,767

Appendix

Table 10. Posts or reviews reporting each PT (

n/N

N

counts posts or reviews with at least one retained side effect, including uncoded ones. Each PT counts once per post or review.

Group	$k/N$	%
Depression	263/1,121	23.5
Anxiety	622/2,969	20.9
All side effects	21,924/316,694	6.9

Appendix

Table 11. Unvalidated automatic stopped markers in Reddit side-effect relations (

k/N

k

counts marked relations and

N

counts relations in each group.

Clinical Information Extraction

Clinical Information Extraction turns clinical notes, reports, and patient narratives into structured, searchable facts and relations.

See all 106 papers

Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking
Jun 24, 2026Huizi Yu, Jian Liu, Wenkong Wang +10KG Construction Healthcare
Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · Department of Psychiatry, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · School of Control Science and Engineering, Shandong University, Ji’nan, Shandong, China +5
Temporally Phenotyping GLP-1RA Case Reports with Large Language Models: A Textual Time Series Corpus and Risk Modeling
Mar 12, 2026Sayantan Kumar, Jeremy C. WeissClinical Information Extraction Document Information Extraction
National Library of Medicine, Bethesda, Maryland, USA
DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering
May 31, 2026Qing Wang, Bo Li, Jialu Liang +3Biomedical QA Agentic RAG
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Florida, USA · PAMI Research Group, Department of Computer and Information Science, Faculty of Science and Technology, University of Macau, Taipa, Macau, China

Jun 24, 2026J/K move · Enter open · S save

Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking

Huizi Yu, Jian Liu, Wenkong Wang, Lingyao Li+9

Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · Department of Psychiatry, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China · School of Control Science and Engineering, Shandong University, Ji’nan, Shandong, China+5

KG Construction Healthcare Mental Health Clinical Information Extraction Biomedical KGs

Open paper AskPDF arXiv

Patients increasingly seek medication information online, yet safety knowledge for psychiatric drugs is split between regulatory adverse-event records, which are authoritative but abstract, and patient narratives, which are experience-near but unvalidated. Integrating them without conflating evidence and anecdote is especially consequential in psychiatry, where poorly contextualised information can amplify fear, nocebo responses, and non-adherence. Here we develop a provenance-aware, knowledge-graph-based multi-agent framework unifying 466,525 Reddit posts, 60,782 WebMD reviews, and twenty years of U.S. FDA Adverse Event Reporting System records for nine antidepressants. A large-language-model entity-recognition pipeline benchmarked against physician annotations reached highest F1 scores of 0.969 for medications and 0.973 for conditions. The two community platforms were far more concordant with each other (overlap up to a Jaccard similarity of 0.905) than with regulatory reports, indicating that patient-generated data form a partly independent safety signal. For sertraline, many adverse events appeared in community sources hundreds of days before the corresponding FDA date. A Neo4j knowledge graph grounded in ATC-N, ICD-10, and MedDRA vocabularies preserves provenance, keeping every claim traceable and regulatory facts distinct from patient experience. These results establish source-aware integration as a route to more auditable psychiatric medication information, with usefulness and patient benefit to be tested prospectively.

Abstract

Clinical Information Extraction

Explore similar work

Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking

Temporally Phenotyping GLP-1RA Case Reports with Large Language Models: A Textual Time Series Corpus and Risk Modeling

DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering