PriHA: A RAG-Enhanced LLM Framework for Primary Healthcare Assistant in Hong Kong

Richard Wai Cheung Chan; Shanru Lin; Ya-nan Ma; Hao Chen; Liangjun Jiang; Wenqi Fan

cs.IRApr 10, 2026

PriHA: A RAG-Enhanced LLM Framework for Primary Healthcare Assistant in Hong Kong

Authors: Richard Wai Cheung Chan, Shanru Lin, Ya-nan Ma, Hao Chen, Liangjun Jiang, Wenqi Fan

Organizations: The Hong Kong Polytechnic University, Hong Kong, China · Haikou Affiliated Hospital of Central South University Xiangya School of Medicine, Haikou, China · Sun Yat-Sen University Cancer Center, Guangzhou, China

Abstract

To address the unsustainable rise in public health expenditures, the Hong Kong SAR Government is shifting its strategic focus to primary healthcare and encouraging citizens to use community resources to self-manage their health. However, official clinical guidelines are fragmented across disparate departments and formats, creating significant access barriers. While general-purpose Large Language Models (LLMs) such as ChatGPT and DeepSeek offer potential solutions for information accessibility, they are prone to generating factually inaccurate content due to a lack of localized and domain-specific knowledge. To this end, we propose a Retrieval-Augmented Generation-Enhanced LLM system as Primary Healthcare Assistant (PriHA) in Hong Kong. Specifically, a tri-stage pipeline is proposed that leverages a query optimizer to generalize user intent-oriented sub-queries, followed by a novel Dual Retrieval Augmented Generation (DRAG) architecture for mixed-source retrieval and context-reorganized generation. Comprehensive experiments and a detailed case study demonstrate that our proposed method can outperform both ablations and baseline in terms of accuracy and clarity. Our research provides a reliable and traceable dialogue retrieval framework for exploring other high-risk, localized application scenarios.

Figures & tables

Figure 1: The illustration of scattered primary healthcare information and the existing limitations of Large Language Models (LLMs)-based Healthcare Assistant in Hong Kong.

Figure 2: Overall framework of the proposed PriHA. The framework processes the initial user input through a query optimizer, yielding a clarified intention and refined sub-queries, then passes to dual retrieval module that fetches and re-ranks content from a local knowledge base and web searchers. Finally, the summarization stage uses a reconciler to synthesize search results into a structured response with proper references.

Figure 3: Distribution of the HK-PriHCQA dataset by category.

Metric	Definition
Accuracy	The answer is factually correct and consistent with ground truth.
Completeness	The response covers all aspects of the query and provides the necessary context.
Trustworthiness	The response cites specific, verifiable, and authoritative sources.
Clarity	The response uses clear, simple language, adopts an empathetic tone, and is well-structured and easy to read.
Relevance	The response directly and completely addresses the specific question asked.

Table 1: Evaluation Metrics Definition

System	Acc.	Comp.	Trust.	Clar.	Rel.	Avg.
Zero-shot LLM	3.23	3.11	3.35	4.75	4.24	3.74
Web-Only RAG	2.54	2.23	2.64	4.66	3.88	3.19
Local-Only RAG	3.50	3.35	3.43	4.71	4.34	3.87
DRAG	3.95	4.03	3.98	4.86	4.18	4.20

Table 2: Mean scores by metric for each system (

N=80

)

Figure 4: Comparative performance of system configurations across five metrics.

Figure 5: Qualitative comparison on a query regarding voucher acceptance at Zhuhai People’s Hospital.

Medical LLMs

Medical large language models are AI systems adapted to clinical and biomedical data for health reasoning, information extraction, and communication.

See all 397 papers

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering
Jul 7, 2026Felix Feldman, Joshua Harris, Timothy Laurence +7Retrieval-Augmented Generation Question Answering
Evaluating the Utility of Personal Health Records in Personalized Health AI
May 18, 2026Rory Sayres, Kejia Chen, Ayush Jain +19LLM Personalization Healthcare
1Google Research · †Work done while at Google Research
A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
Aug 12, 2026Praveen Reddy, Charuta Mandke, Suvrankar Datta +4Retrieval-Augmented Generation LLM Grounding

Jul 7, 2026J/K move · Enter open · S save

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

Felix Feldman, Joshua Harris, Timothy Laurence, Leo Loman+6

Retrieval-Augmented Generation Question Answering Hybrid IR RAG Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hallucinations and the rapid evolution of official guidance. Retrieval-Augmented Generation (RAG) mitigates these risks by grounding responses in an explicitly maintained corpus, but end-to-end performance depends critically on retrieval configuration and on evaluation beyond multiple-choice formats. We extend PubHealthBench, a question answering (QA) benchmark of 7,929 questions derived from UK Government public health guidance, into a retrieval-augmented setting and systematically evaluate retrieval and generation choices. We compare dense, sparse, and hybrid retrieval across multiple embedding models and corpus variants, and show that hybrid retrieval consistently improves recall and ranking quality, with chunk length and topic interacting with ranking performance. Providing retrieved context substantially increases multiple-choice accuracy across a diverse set of LLMs, enabling smaller open-weight models to match or outperform larger models used without retrieval, with gains primarily driven by retrieval quality and careful context selection. To assess realistic free-form answering, we introduce a rubric-based LLM-as-a-judge covering faithfulness, completeness, clarity, and factual consistency, and validate it against dual human annotations. Judge-human agreement is strongest for faithfulness and completeness, while factual consistency and clarity are less reliably reproduced, motivating caution when interpreting those dimensions at scale. Overall, our results highlight retrieval as a primary lever for reliable public health QA and provide practical guidance for building and evaluating RAG systems grounded in official guidance.

Abstract

Medical LLMs

Explore similar work

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

Evaluating the Utility of Personal Health Records in Personalized Health AI

A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench