COACH meets QUORUM: A Framework and Pipeline for Aligning User, Expert and Developer Perspectives in LLM-generated Health Counselling

Yee Man Ng; Bram van Dijk; Pieter Beynen; Otto Boekesteijn; Joris Jansen; Gerard van Oortmerssen; Max van Duijn; Marco Spruit

cs.CLMar 9, 2026

COACH meets QUORUM: A Framework and Pipeline for Aligning User, Expert and Developer Perspectives in LLM-generated Health Counselling

Authors: Yee Man Ng, Bram van Dijk, Pieter Beynen, Otto Boekesteijn, Joris Jansen, Gerard van Oortmerssen, Max van Duijn, Marco Spruit

Organizations: Leiden University · Leiden University Medical Center · Healthy Chronos

Abstract

Systems that collect data on sleep, mood, and activities can provide valuable lifestyle counselling to populations affected by chronic disease and its consequences. Such systems are, however, challenging to develop; besides reliably extracting patterns from user-specific data, systems should also contextualise these patterns with validated medical knowledge to ensure the quality of counselling, and generate counselling that is relevant to a real user. We present QUORUM, a new evaluation framework that unifies these developer-, expert-, and user-centric perspectives, and show with a real case study that it meaningfully tracks convergence and divergence in stakeholder perspectives. We also present COACH, a Large Language Model-driven pipeline to generate personalised lifestyle counselling for our Healthy Chronos use case, a diary app for cancer patients and survivors. Applying our framework shows that overall, users, medical experts, and developers converge on the opinion that the generated counselling is relevant, of good quality, and reliable. However, stakeholders also diverge on the tone of the counselling, sensitivity to errors in pattern-extraction, and potential hallucinations. These findings highlight the importance of multi-stakeholder evaluation for consumer health language technologies and illustrate how a unified evaluation framework can support trustworthy, patient-centered NLP systems in real-world settings.

Figures & tables

Figure 1: Left: user interface for submitting queries. Right: generated counselling for query ‘How can I sleep better?’ Blue underlining illustrate claims about the user data ; orange indicates contextualisation statements that contextualise found patterns with information from the knowledge database. Claims and statements are labelled with (1), (A), etc. but this is hidden from the user.

Perspective (N)	Statement evaluated	Variable	Measurement	Interpretation
Users (19)	1 The response aligns with my situation	Alignment	5-point Likert scale	Avg. score indicating alignment with user’s situation (mood, energy, etc.)
	2 I will follow the tips in the response	Follow-up	5-point Likert scale	Avg. score indicating whether user is likely to act on the response
	3 The response has the right tone	Tone	5-point Likert scale	Avg. score of response tone quality
	4 The response has the right length	Length	5-point Likert scale	Avg. score of response length
Experts (6)	1 The contextualisation is correct	Correctness	5-point Likert scale	Avg. score of contextualisation quality
	2 The response has the right tone	Tone	5-point Likert scale	Avg. score of response tone quality

Table 1: Contents (translated from Dutch) of the QUORUM evaluation framework. Five-point Likert scales are structured as strongly disagree (1) , disagree (2) , neutral (3) , agree (4) , strongly agree (5) .

Figure 2: Overview of the pipeline flow and inputs.

Figure 4: Visualization of QUORUM outcomes. ‘Averages’ graph summarises relevance, quality, and reliability; ‘Relevance’ and ‘Quality’ summarize their respective dimensions by averaging over variable scores; ‘Reliability’ contains rescaled variable ratios ( Table 1 .)

Var.
Alignm.	$3.90_{\pm.94}$
Fol. up	$3.90_{\pm 1.15}$
Tone	$4.32_{\pm.82}$
Length	$4.16_{\pm.69}$

Table 2: Main evaluation results. For users, we report averages over 19 users; for experts, averages over in total 100 evaluations (50 response-query pairs evaluated by two experts); for developers, ratios over in total 66 claims about user data (faithfulness/completeness) and 31 contextualisation statements (hallucination).

Annotator	A (n=25)	B (n=25)	C (n=12)	D (n=13)	E (n=12)	F (n=13)
Correctness	4.08 (±0.63)	4.88 (±0.33)	3.33 (±0.47)	4.15 (±0.36)	2.75 (±0.73)	3.92 (±0.47)
Tone	4.12 (±0.59)	3.8 (±0.63)	3.42 (±0.76)	3.7 (±0.92)	3 (±0.71)	3.85 (±0.36)
Length	4.16 (±0.46)	4.04 (±0.34)	3.92 (±0.28)	4.46 (±0.63)	3.25 (±0.6)	3.92 (±0.47)

Table 3: Mean and standard deviation of each expert and variable. N shows the number of query-response pairs evaluated per expert, each was labelled twice, totalling 100 observations.

Figure 5: Examples of various errors in claims about data and contextualisation statements. A): overgeneralization of common-sense patterns: B) missing pattern in the user data; C): hallucination: advice is not strictly present in chunks.

Clinical Language Model Evaluation

Clinical language model evaluation measures whether AI systems are accurate, safe, fair, and useful across medical tasks and care settings.

See all 418 papers

Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs
Jun 7, 2026Rahul Gorijavolu, Kaushik Madapati, Pritika Vig +7LLM Evaluation LLM Sycophancy
MIT Critical Data, Massachusetts Institute of Technology, Cambridge, MA, USA · School of Medicine, Johns Hopkins University, Baltimore, MD, USA · Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA +10
Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration
Jul 26, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman +3Mental Health Counseling LLM Prompting
Ahsanullah University of Science and Technology · Bangladesh University of Business and Technology · The University of New South Wales +2
Challenges of Auditing: Variability in Outputs of Large Language Models for Health
Sep 15, 2026Yuan Pu, Yewon Chang, Furong Jia +4LLM Evaluation LLM Auditing
Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NC. · Department of Medicine, Duke University, 40 Duke Medicine Circle, Durham, 27710, NC. · Geriatrics and Extended Care, Durham VA Health System, 508 Fulton Street, Durham, 27705, NC. +2

Jun 7, 2026J/K move · Enter open · S save

Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs

Rahul Gorijavolu, Kaushik Madapati, Pritika Vig, Rawan Abulibdeh+6

MIT Critical Data, Massachusetts Institute of Technology, Cambridge, MA, USA · School of Medicine, Johns Hopkins University, Baltimore, MD, USA · Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA+10

LLM Evaluation LLM Sycophancy LLM Personalization LLM Auditing Medical LLMs

Open paper AskPDF arXiv

Background: Consumer-facing large language models are now a common source of health information, and they interpret and personalize responses rather than retrieve them. Whether their responses vary across users is a clinical, equity, and governance question, sharpened by evidence that sycophantic responses can alter judgment and increase trust. Objective: To evaluate response variation and sycophancy in consumer-facing health LLMs under conditions resembling ordinary patient use. Methods: We constructed simulated user profiles differing in geography, browsing context, expressed beliefs, and social determinants of health, drawing on literature linking social context to health attitudes. We adapted validated instruments, including the Vaccination Attitudes Examination scale and reproductive attitudes scales, into multi-turn prompts designed to elicit clinically meaningful variation across users. Results: The evaluation encountered five linked barriers. Factual prompts produced stable responses that masked sycophancy emerging over multi-turn conversation. Browser-based interfaces did not disclose which signals influence outputs and could not be reset to a clean baseline. Large-scale testing was restricted by terms of service, rate limits, and bot detection. Accuracy-based criteria could not capture tone, framing, or omission, and LLM-as-judge methods risked shared alignment bias. Models changed without traceable version identifiers, preventing reliable replication. Conclusions: No reliable independent evaluation framework yet exists for examining how consumer-facing health LLMs behave in ordinary use. Oversight requires disclosure of personalization signals, stable version identifiers, researcher safe harbor programs, and post-deployment monitoring of health-related outputs.

Abstract

Clinical Language Model Evaluation

Explore similar work

Testing the Black Box: Structural Barriers to Independent Evaluation of Consumer-Facing Health LLMs

Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration

Challenges of Auditing: Variability in Outputs of Large Language Models for Health