ClinQueryAgent: A Conversational Agent for Population Health Management

Joseph S. Boyle; Anthony Dranfield; Mike O'Neil; Maria Liakata; Alison Q. Smithard

cs.IRApr 13, 2026

ClinQueryAgent: A Conversational Agent for Population Health Management

Authors: Joseph S. Boyle, Anthony Dranfield, Mike O'Neil, Maria Liakata, Alison Q. Smithard

Organizations: Canon Medical Research Europe · Queen Mary University of London · Nottingham and Nottinghamshire ICB, NHS England · University of Edinburgh

Abstract

In this paper we introduce ClinQueryAgent, a system for translating natural language population health questions into executable database queries using agents with access to both local and external knowledge bases. Our novel architecture enables the use of powerful cloud-based language models whilst ensuring that no patient data leaves the secure environment. To combat inaccuracies over the course of longer dialogues due to context rot, information retrieval is delegated to a sub-agent. We deploy the system via a chat window embedded within an existing population health management platform where it has been used by 128 staff from 15 healthcare practices covering a total of 148,319 patients in the UK's National Health Service (NHS). We evaluate the system's capacity to autonomously handle a range of health informatics tasks on a constructed dataset and via a beta-testing phase. Our results show that both analysts and clinicians are able to easily generate actionable information from patient health records using natural language requests requiring no programming expertise to verify. We make a public demo of the system available at: https://demo-899965260288.europe-west1.run.app/

Figures & tables

Workflow description	Real-world Example User Request
List each patient in a cohort	[General Practitioner] “List the new hypertension diagnoses in
(returns a list)	the last 12 months”
	Intention: Clinical audit
Summarise age and sex statistics across a patient cohort	[Pharmacist] “number of patients on semaglutide”
(returns the count, mean age, and sex)	Intention: Check financial spend [Follow-up: Check if budget is targeting desired patient population e.g. high deprivation patients who cannot afford medication]
Show distribution of a variable for a cohort	[Mental Health Commissioning Manager] “Can you find out the

Table 1: The five categories (workflows) of data analysis that may be handled and displayed by eHealthScope, with corresponding real-world user request examples. The agent implicitly classifies the user request into a workflow description, which it typically describes at the beginning of its answer process: ‘To answer this question I will first search for hypertension and then create a cohort from those patients, and then link you to a register of those patients’. ‘SMI QOF register’ refers to a specific concept for patients with one or more Serious Mental Illness.

Figure 4: Cumulative message count for ClinQueryAgent in its beta deployment phase from 2025-06-26 – 2026-02-13. Clinical staff include GPs, care-coordinators, and pharmacists; Management and Commissioning includes operations staff, practice managers; Data & Technology staff include population health analysts, software developers, and data management staff.

		Patient Overlap			Mean Per Question
Type	Method	R	P	F1	Tokens	Time (s)	Tools
EpiCoho-M Dataset
	QueryAgent	0.72	0.68	0.64	35,299	15.14	3.39
Single	+ Retrieval Agent	0.76	0.63	0.63	38,693	23.18	3.36
	+ Retrieval Agent + UMLS	0.80	0.68	0.68	38,509	18.97	3.47
	QueryAgent	0.45	0.53	0.43	855,427	64.82	2.13

Table 2: Performance metrics on the EpiCoho-M epidemiological cohort questions, the Real World beta-test questions and the UK brands datasets, with/without the concept retrieval sub-agent, and with/without access to UMLS. We assess accuracy according to how closely the retrieved patient cohort matches the expected patient cohort in a real-world database, using recall (R), precision (P) and F1-score (F1). We also show the mean number of tokens used, time, and number of tool calls per question.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Concept	Code ID	Code Description	Pseudocode
	44054006	Type II Diabetes	SELECT pat_id
Diabetes	472969004	History of diabetes type 2	FROM Conditions
type II	443694000	Uncontrolled type 2 diabetes	WHERE code IN (
	…	+160 more	44054006, 472969004, ...)
	777514008	Semaglutide only product	SELECT pat_id
Semaglutide	782102009	Semaglutide 1.34 mg/mL	FROM Medications

Appendix

Table 3: Example NHS Nottingham and Nottinghamshire ICB concept definitions. The concept codesets are described in columns 2 & 3, and the final “Pseudocode” column illustrates the rules to combine them; these are usually boolean logic, sometimes including time constraints. The corresponding patient cohort comprises all patients with this code assignment pattern present in their record.

Feedback	count	url clicked
positive	29	24/29
negative	18	9/18
no response	307	197/307
Total	354	230/354

Appendix

Table 4: User labels annotated during the beta testing deployment. Users were prompted with the message ‘Was this useful?’ followed by thumbs-up and thumbs-down options to give positive and negative feedback.

User Role	Intent	Analysis
Practice Manager 1	“how do I add a new user”	This action is not supported by the agent
Practice Manager 2	“edit a user”	This action is not supported by the agent
Practice Manager 2	“edit a user”	This action is not supported by the agent
Practice Manager 3	“how do I modify user permissions”	This action is not supported by the agent
Practice Manager 4	"permissions log"	This action is not supported by the agent
Receptionist	"speak to a human being"	This action is not supported by the agent

Appendix

Table 5: Error analysis for each instance of negative feedback received in the four months beta deployment. The system was updated to support the requested actions related to user management in general practices.

Healthcare

Healthcare AI develops and evaluates models and systems for clinical care, medical research, public health, and healthcare operations.

See all 616 papers

ClinAgent: A ReAct-Based Agent for Conversational Access to Clinical Trial Information
Sep 12, 2026Antonino Vaccarella, Riccardo Cantini, Domenico Talia +4Clinical QA LLM Agents
Department of Computer Science, University of Pisa, Pisa, Italy · Department of Computer Engineering, Modeling, Electronics and Systems, University of Calabria, Rende, Italy. · Institute of Information Science and Technologies “Alessandro Faedo”, National Research Council, Pisa, Italy. +2
Large Language Models for Structured Clinical Data Analysis: Dual-Agent Grounding and Validation
Sep 28, 2026Erfan D. Dehkalani, Seetha Shankaran, Abbot R. Laptook +3Multi-Agent LLM Systems LLM Grounding
Fetal-Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, and Harvard Medical School, Boston, MA, USA · Department of Pediatrics, Wayne State University School of Medicine, Detroit, MI, USA · Department of Pediatrics, Women & Infants Hospital of Rhode Island, and Warren Alpert Medical School of Brown University, Providence, RI, USA +1
A Self-Evolving Agent for Longitudinal Personal Health Management
Jul 15, 2026Haoran Li, Jiebi Deng, Tong Jin +10Agent Memory Healthcare
School of Data Science, Fudan University, Shanghai, China · School of Life Sciences, Beijing University of Chinese Medicine, Beijing, China · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China +4

Sep 12, 2026J/K move · Enter open · S save

ClinAgent: A ReAct-Based Agent for Conversational Access to Clinical Trial Information

Antonino Vaccarella, Riccardo Cantini, Domenico Talia, Paolo Trunfio+3

Department of Computer Science, University of Pisa, Pisa, Italy · Department of Computer Engineering, Modeling, Electronics and Systems, University of Calabria, Rende, Italy. · Institute of Information Science and Technologies “Alessandro Faedo”, National Research Council, Pisa, Italy.+2

Clinical QA LLM Agents Agentic RAG LLM Agent Evaluation

Open paper AskPDF arXiv

Querying clinical trial registries remains a manual and error-prone process, requiring researchers to navigate large volumes of semi-structured data without support for natural language interaction or cross-source synthesis. To address this, we introduce ClinAgent, a conversational system based on agentic Retrieval-Augmented Generation (RAG) that enables clinicians and researchers to query clinical trial information in plain language and receive grounded, up-to-date responses across multi-turn interactions. The system centers on a Large Language Model (LLM) agent following the ReAct paradigm, which iteratively reasons over queries, selects among a set of integrated tools, and refines its actions based on intermediate outputs. These tools include a ClinicalTrials.gov search interface, a PubMed module, and a Python-based analyzer operating on a locally cached structured dataset of clinical trials. We evaluate the system using a three-phase framework assessing operational effectiveness, planning quality, tool-use efficiency, and expert qualitative judgments, comparing three LLM backends: Gemini 3.0 Flash and two variants of DeepSeek V3.2 (thinking and non-thinking). Results reveal complementary strengths, with DeepSeek (thinking mode) excelling in planning quality, while Gemini achieves the highest overall performance and strongest expert ratings. Overall, our findings highlight the potential of agentic AI systems to improve the accessibility and synthesis of clinical trial information, supporting more efficient and user-centered biomedical research workflows.

Abstract

Healthcare

Explore similar work

ClinAgent: A ReAct-Based Agent for Conversational Access to Clinical Trial Information

Large Language Models for Structured Clinical Data Analysis: Dual-Agent Grounding and Validation

A Self-Evolving Agent for Longitudinal Personal Health Management