cs.CYMar 26, 2026
SaveSame Verdict, Different Reasons: LLM-as-a-Judge and Clinician Disagreement on Medical Chatbot Completeness
Organizations: Data Science and AI Institute, Johns Hopkins University
Abstract
LLM-as-a-Judge frameworks are increasingly trusted to automate evaluation in place of human experts, yet their reliability in high-stakes medical contexts remains unproven. We stress-test this assumption for detecting incomplete patient-facing medical responses, evaluating three rubric granularities (General-Likert, Analytical-Rubric, Dynamic-Checklist) and three backbone models across two clinician-annotated datasets, including HealthBench, the largest publicly available benchmark for medical response evaluation. LLM Judges discriminate complete from incomplete responses at and slightly above near chance (AUC --); at the threshold required to recall of incomplete responses, clinicians must still review the vast majority of the dataset, offering no triage utility. Even when model and clinician verdicts agree, they rarely cite the same explanation; and when they diverge, false positives stem from over-flagging non-essential gaps while false negatives reflect outright detection failures. These results reveal that LLM Judges and clinicians apply fundamentally different completeness standards; a finding that undermines their use as autonomous evaluators or triage filters in clinical settings.
Figures & tables
Figure 1: A true positive example in which both clinician and LLM-as-a-Judge rate the response as incomplete but identify entirely different omissions: the clinician flags missing advice to avoid weights that could drop on the abdomen, while the LLM flags missing safety-netting and contraindications such as preeclampsia. Shared verdicts do not imply shared reasoning.
Figure 2: Overview of the three rubrics evaluated in this work for judging the completeness of chatbot responses to medical questions. The rubrics are in increasing order of granularity: 3-point General-Likert, 5-point Analytical-Rubric, and the -point Dynamic-Checklist. The example is from HealthBench, judged by Llama 3.3 70B. “HCP” refers to Healthcare Provider.
| LLaMA 3.3-70B | OpenBioLLM | GPT-5 Mini | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | Metric | F1 | Prec | Rec | AUC | F1 | Prec | Rec | AUC | F1 | Prec | Rec | AUC |
| HealthBench | Dynamic-Checklist ∗ | 0.89 | 0.97 | 0.82 | 0.91 | 0.91 | 0.90 | 0.93 | 0.85 | 0.93 | 0.96 | 0.91 | 0.93 |
| Analytical-Rubric | 0.42 | 0.91 | 0.27 | 0.65 | 0.20 | 0.84 | 0.11 | 0.53 | 0.76 | 0.86 | 0.68 | 0.66 | |
| Analytical-Rubric (FSS) | 0.75 | 0.83 | 0.68 | 0.64 | 0.65 | 0.82 | 0.54 | 0.59 | 0.83 | 0.81 | 0.84 | 0.62 | |
| General-Likert | 0.07 | 0.95 | 0.04 | 0.53 | 0.01 | 0.83 | 0.01 | 0.50 | 0.54 | 0.89 | 0.39 | 0.64 | |
| General-Likert (FSS) | 0.61 | 0.85 | 0.47 | 0.63 | 0.39 | 0.82 | 0.26 | 0.54 | 0.67 | 0.82 | 0.57 | 0.58 | |
Table 1: Discriminative power of automated metrics against Clinician Judgment for identifying Incomplete responses measured with F1, Precision (Prec), and Recall (Rec), and area under the ROC curve (AUC). FSS variants denote few-shot prompted versions of General-Likert and Analytical-Rubric. Within each dataset, bold-underline marks the best value per column across all models and metrics; underline marks the best value per column within each backbone model. ∗ Dynamic-Checklist on HealthBench is excluded from highlighting: its scores are inflated by circularity in the ground-truth construction.
Figure 3: Omission-alignment heatmap by verdict category: TP and TN (model and clinician agree on verdict) and FP and FN (they disagree). Each cell shows the percentage of pairs with full ( Yes ), partial ( Partially ), or no ( No ) overlap in cited omissions. TP 8,725; TN 6,211; FP 3,440; FN 8,759.
Figure 4: Failure-mode distribution by metric for different-verdict pairs. Each bar shows the percentage of pairs assigned to each category by the GPT-5 Mini classifier. Top: false positives (FP01–FP04), where the model over-penalizes a clinician-approved response. Bottom: false negatives (FN01–FN05), where the model misses a clinician-identified omission. Category definitions are in Table 14 . FP 3,440; FN 8,759.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Distribution of automated completeness scores (columns) by clinician-labelled omission severity (rows) on MedExpert. Rows correspond to backbone LLM judges; columns correspond to metrics. Cell annotations show raw count and row-normalised percentage. An ideal grader would concentrate mass along the top-left to bottom-right diagonal.
Figure 6: Distribution of LLM-as-a-Judge completeness metric scores ([0,1]) for responses judged Complete vs. Incomplete by clinicians, faceted by dataset (rows) and backbone LLM (columns). Each pair of boxes shows the score distribution assigned by a given metric to clinician-labeled Complete (left) and Incomplete (right) responses; well-separated pairs indicate strong discriminative ability. Boxes that overlap substantially, or where Incomplete scores are not reliably lower than Complete scores, indicate that the metric fails to distinguish the two groups. Annotations show mean standard deviation for each group. Asterisk (*) above a pair indicates Mann-Whitney U for score separation between Complete and Incomplete groups.
Figure 7: Pairwise Pearson correlation between backbone LLM scores on the same responses, within each dataset metric combination. Asterisks (*) denote . Diagonal cells are blank (self-correlation). Low correlations indicate that the choice of backbone LLM substantially changes the assigned completeness score.
Figure 8: Sankey diagram showing the distribution of error types in the false positive and false negative cases aggregated by LLM Judge rubric.
| Metric Name | Definition | Annotation Setup |
|---|---|---|
| Completeness ( Xie et al., 2024 ) | Amount of salient details in the [generated answer] as recalled from a reference | Fine-grained checklist of the claims in the generated answer from the original response |
| Completeness ( Wang et al., 2024b ) | How well the generated text captures the key ideas of the reference | Rating Scale of 1-5 5 5 5 Wang et al. (2024b) use human evaluation to verify their automated approach, though the annotation details are not specified. |
| Thoroughness ( Allen et al., 2024 ) | Answers all questions posed by a patient | - |
| Completeness ∗ ( Arora et al., 2025 ) | All important information needed to be safe and helpful to the user. Even if accurate, a response that is incomplete (e.g., omitting key steps or red flags) can still result in low-quality advice or harm. | Fine-grained checklist of required information for a question |
| Informativeness ∗ ( Xu et al., 2024 ) | Comprehensive: answers include additional information beyond the expectations | Rating Scale of (0) incomplete (1) adequate (2) comprehensive |
| Completeness ∗ ( Yarmohammadi et al., 2025 ) | A complete answer contains all relevant medical information for the patient based on the provided question. Incomplete responses contain omissions, which are defined as important information whose omission could cause clinical harm to the patient. | List of omissions and their severity levels |
Table 2: Various definitions of “Completeness” in the AI-Medical chatbot literature and the manual evaluation setup (if applicable). Metrics marked with a ∗ have released manual annotations.
Table 3: Few-shot prompt used to classify whether a user is a Healthcare Professional (True) or a Patient/Layperson (False).
Table 4: Prompt for a rubric LLM-as-a-Judge from HealthBench.
Table 5: Prompt for a Likert score rubric LLM-as-a-Judge. Modified from HealthBench.
Table 6: Prompt for the Dynamic-Checklist Step 1: generating criteria for a question.
Table 7: Full grading prompt for the General-Likert rubric. The rubric criterion is from Diekmann et al. (2025b) .
Table 8: Few-shot grading prompt for the General-Likert FSS metric (Part 1): instructions, rubric, and Score 3 example.
Table 9: Few-shot grading prompt for the General-Likert FSS rubric (Part 2): Score 2 and Score 1 examples and task template.
Table 10: Full grading prompt for the Analytical-Rubric rubric. The rubric is modeled after the completeness annotation instructions from Yarmohammadi et al. (2025) .
Table 11: Few-shot grading prompt for the Analytical-Rubric FSS rubric. Two clinician-annotated examples per score level are included in the full prompt; one Score 1 example shown here for space.
Table 12: RQ2 system prompt for same-verdict pairs (TP and TN): determines whether model and clinician identified the same omissions. Applied by GPT-5 Mini to all TP and TN cases.
Table 13: RQ2 system prompt for different-verdict pairs (FP and FN): classifies the model’s failure mode into one of nine categories. Applied by GPT-5 Mini to all FP and FN cases.
| Code | Name | Description |
| False Positive categories (Model = Incomplete, Clinician = Complete) | ||
| FP01 | Over-flags minor gaps | The model identifies omissions the clinician considers non-essential or supplementary. The core clinical question is safely answered; the model applies a completeness standard stricter than clinical safety requires. |
| FP02 | Demands depth beyond patient-facing scope | The model expects provider-level or specialist-level detail (e.g., exact risk percentages, pharmacokinetics, procedure-specific protocols) that the clinician considers beyond what a patient-facing answer needs, especially for underspecified questions. |
| FP03 | Misapplies safety-netting to non-urgent context | The model inserts emergency or red-flag framing into questions that are informational, low-acuity, or underspecified. The clinician judges the answer adequate for the question as asked; the model penalizes for missing escalation criteria when no emergency context exists. |
| FP04 | Misinterprets question intent / misapplies rubric | The model fundamentally misunderstands what the question is asking or incorrectly applies evaluation criteria (e.g., penalizing an appropriate doctor referral for not directly answering, or evaluating against criteria unrelated to the question). |
| False Negative categories (Model = Complete, Clinician = Incomplete) | ||
Table 14: RQ3 failure-mode taxonomy for different-verdict pairs. FP codes apply when the model over-penalizes (Model = Incomplete, Clinician = Complete); FN codes apply when the model under-penalizes (Model = Complete, Clinician = Incomplete). Each pair is assigned exactly one code by the GPT-5 Mini classifier ( Table 13 ).
More in this topic
LLM-as-a-Judge
Large language model (LLM)-as-a-judge research uses language models to score, compare, and verify AI outputs as scalable alternatives to human evaluation.
See all 555 papers