HerHealthGPT: An Evidence-Grounded Conversational AI System for Women's Reproductive Health Copy title

Hassan Saeed Hassan Albattra; Mazen Mohammed Bahgat; Rahatara Ferdousi; Hana Essam Sayed Ahmed Amrya; Mariam Mousa

cs.CLOct 24, 2026

HerHealthGPT: An Evidence-Grounded Conversational AI System for Women's Reproductive Health Copy title

Authors: Hassan Saeed Hassan Albattra, Mazen Mohammed Bahgat, Rahatara Ferdousi, Hana Essam Sayed Ahmed Amrya, Mariam Mousa

Organizations: School of Computing, Queen’s University, Kingston, Ontario, Canada

Abstract

"Why is my period late?" Sometimes, the most useful response begins with another question. Yet conversational AI systems may respond to incomplete health queries without seeking essential context or grounding their guidance in verifiable evidence. This paper introduces HerHealthGPT, an evidence-grounded conversational AI system designed to address these challenges in women's reproductive health. HerHealthGPT integrates retrieval-augmented generation (RAG), context-aware clarification, and source validation within an interactive web interface. Rather than responding immediately to incomplete queries, the system maintains conversational context, asks targeted follow-up questions, checks for premature conclusions, and grounds its responses in retrieved evidence. We evaluate the AI components using 540 English messages derived from 90 reproductive-health cases and conduct a prototype questionnaire study with 30 adults. Evidence retrieval improves concern-category accuracy from 54.1% to 59.3% compared with fine-tuning alone. The clarification router achieves 85.6% recall and 86.2% specificity on constructed targets. In the user study, 83.3% of participants preferred the retrieval-enabled configuration, while 25 of 30 considered its follow-up questions relevant. These findings suggest that integrating evidence retrieval with interactive clarification offers a promising approach to more transparent, context-aware reproductive-health information systems, while further clinical validation remains necessary.

Figures & tables

Fig. 1 : Four-layer architecture of HerHealthGPT, integrating the consumer interface, application services, model inference and retrieval, and persistent storage.

Fig. 2 : HerHealthGPT conversational workflow. Each turn combines conversation history and retrieved evidence to generate a structured decision. A premature-conclusion check may trigger one revision before validation, storage, and response delivery.

Panel of Figure 3 — Fig. 3 : Prototype screens. 1 1 1 The interactive prototype of the proposed system is available at: https://herhealthgpt.org/ . (a) Onboarding on a phone-sized screen; (b) one follow-up question per turn, each with its sources; (c) a routine-appointment conclusion with the quoted source sentences and disclaimer; (d) an inform answer to a general question, with citations and no urgency level. The examples show interface behavior, not clinical endorsements.

System	Configuration
A	Base Qwen3.5-9B with instructions
B	QLoRA-adapted Qwen3.5-9B
C	QLoRA-adapted Qwen3.5-9B + RAG

TABLE I : Configurations in the consumer study.

Configuration	Category	Compatible	Risk	Under-triage $\downarrow$	Clar. recall	Clar. spec.
ZS	53.9	62.2	66.7	71.8	85.6	86.2
Base+RAG	56.1	64.7	65.0	73.6	66.7	94.7
FT	54.1	62.4	68.5	52.9	0.0	100.0
FT+Instruction	54.3	62.6	67.0	58.6	0.0	100.0
FT+Shuffled	53.1	61.3	67.6	59.2	0.0	100.0
FT+Real RAG	59.3	68.4	68.1	56.9	0.0	100.0

TABLE II : English component results. Category accuracy: 540 messages (full) and 468 (compatible); risk: 540; under-triage: 174; clarification recall: 90 and specificity: 450. All values are percentages.

Measure	Median [Q1,Q3]	EN	AR
Easy to understand	4 [4,5]	21	5
Response relevant	4 [4,5]	20	5
Response useful	4 [4,5]	21	5
Hesitate to rely	3 [2,4]	7	1
Follow-ups relevant	4 [4,5]	22	3
Understood question ∗	5 [4,5]	21	2

TABLE III : Consumer ratings (

n=30

). Counts indicate ratings

\geq 4

Panel of Figure 7 — Fig. 4 : Consumer’s preferences. (n=30): (a) preferred system configuration and (b) intended actions.

Healthcare

Healthcare AI develops and evaluates models and systems for clinical care, medical research, public health, and healthcare operations.

See all 617 papers

HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering
Apr 29, 2026Md Biplob Hosen, Md Alomgeer Hussein, Md Akmol Masud +3LLM Grounding Healthcare
University of Maryland Baltimore County, Baltimore, MD, USA · 2Jahangirnagar University, Savar, Dhaka, Bangladesh
ThReadMed-QA: A Multi-Turn Medical Dialogue Benchmark from Real Patient Questions
Mar 11, 2026Monica Munnangi, Saiph SavageMedical QA Multi-Turn Dialogue Evaluation
Khoury College of Computer Sciences Northeastern University Boston
mamabench and mamaretrieval: Benchmarks for Evaluating Medical Retrieval-Augmented Generation in Maternal, Neonatal, and Reproductive Health
Jun 28, 2026Yi RenHealthcare Medical QA
École Polytechnique Fédérale de Lausanne, Switzerland

Apr 29, 2026J/K move · Enter open · S save

HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering

Md Biplob Hosen, Md Alomgeer Hussein, Md Akmol Masud, Omar Faruque+2

University of Maryland Baltimore County, Baltimore, MD, USA · 2Jahangirnagar University, Savar, Dhaka, Bangladesh

LLM Grounding Healthcare Clinical QA Evidence-Grounded Generation EHR QA Evidence Retrieval

Open paper AskPDF arXiv

Patient portals now give individuals direct access to their electronic health records (EHRs), yet access alone does not ensure patients understand or act on the complex clinical information contained in these records. The ArchEHR-QA 2026 shared task addresses this challenge by focusing on grounded question answering over EHRs, and this paper presents the system developed by the HealthNLP_Retrievers team for this task. The proposed approach uses a multi-stage cascaded pipeline powered by the Gemini 2.5 Pro large language model to interpret patient-authored questions and retrieve relevant evidence from lengthy clinical notes. Our architecture comprises four integrated modules: (1) a few-shot query reformulation unit which summarizes verbose patient queries; (2) a heuristic-based evidence scorer which ranks clinical sentences to prioritize recall; (3) a grounded response generator which synthesizes professional-caliber answers restricted strictly to identified evidence; and (4) a high-precision many-to-many alignment framework which links generated answers to supporting clinical sentences. This cascaded approach achieved competitive results. Across the individual tracks, the system ranked 1st in question interpretation, 5th in answer generation, 7th in evidence identification, and 9th in answer-evidence alignment. These results show that integrating large language models within a structured multi-stage pipeline improves grounding, precision, and the professional quality of patient-oriented health communication. To support reproducibility, our source code is publicly available in our GitHub repository

Abstract

Healthcare

Explore similar work

HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answering

ThReadMed-QA: A Multi-Turn Medical Dialogue Benchmark from Real Patient Questions

mamabench and mamaretrieval: Benchmarks for Evaluating Medical Retrieval-Augmented Generation in Maternal, Neonatal, and Reproductive Health