Evaluation format, not only model capability, shapes measured triage failure in consumer health AI
Authors: David Fraile Navarro, Jialei Sheng, Farah Magrabi, Enrico Coiera
Organizations: Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW, Australia · NSW Health, Sydney, NSW, Australia
A recent Nature Medicine study reported that ChatGPT Health under-triages 51.6% of emergencies and concluded that consumer-facing AI triage poses safety risks. Its protocol, however, was an exam-style scaffold (forced A/B/C/D output, knowledge suppression, no clarifying questions) unlike how consumers use health chatbots. We ask how much of the headline error rate is a property of the measurement rather than of the models. In a first, mechanistic study, five frontier LLMs on a 17-scenario bank scored 6.4 points higher under naturalistic patient-style messages than under the constrained scaffold (p=0.015). In a second, faithful replication we ran the authors' own 60 released vignettes through six frontier models under four matched formats, with clinician validation of the rewrites and a blinded clinician audit of the LLM adjudicators. Here the direction reversed: free-text rewrites scored slightly below the exact prompt (78.7% vs 81.8%, p=0.020), removing only the answer scaffold changed little (80.6% vs 81.4%, p=0.81), and naturalistic input with a forced categorical answer beat both the exact prompt (84.4%, p=0.023) and free text (p=0.0001). On the four vignettes defining the original emergency rate, under-triage was 17% with the exact scaffold, 50% in free text, and 21% with a forced letter on the same message; every free-text "under-triage" was a same-day recommendation scored C rather than D, and 60% made escalation conditional on information the patient was asked to check, behavior a single-turn benchmark cannot score. The headline rate did not reproduce as a stable cross-model finding; much of it depends on output format and on mapping prose onto a four-point scale. Each format elicits a separate response, so the design measures format sensitivity, not format versus capability. Safety claims should report sensitivity to input wording, output format and adjudication.
Figures & tables
Matched comparison
Asthma ablation
Model
Constrained
Naturalistic
Δ (pp)
Forced A/B/C/D
Free text
Fisher p
GPT-5.2
64.1%
68.2%
+4.1
4/25
25/25
3.8×10−10
Claude Sonnet 4.6
56.5%
71.2%
+14.7
25/25
25/25
1.00
Claude Opus 4.6
61.8%
72.4%
+10.6
25/25
25/25
1.00
Gemini 3 Flash
63.5%
66.8%
+3.2
6/25
25/25
1.2×10−8
Gemini 3.1 Pro
72.4%
71.8%
−0.6
0/25
25/25
1.6×10−14
Table 1: Study 1. Left: matched constrained (exam-style) versus naturalistic (patient-style) accuracy; 170 matched responses per model, two-judge mean. Right: high-powered asthma ablation ( n=25 per cell) isolating the output format on identical content.
Model
Exact structured
Natural free text
Source-stripped
Natural forced letter
GPT-5.3 Instant
81.7%
74.2% ( −7.5 )
83.3% ( +5.0 )
86.7% ( +5.0 )
GPT-5.4
80.8%
81.7% ( +0.8 )
79.2% ( −0.8 )
85.8% ( +5.0 )
Claude Sonnet 4.6
80.8%
77.1% ( −3.8 )
78.3% ( −3.3 )
87.5% ( +6.7 )
Claude Opus 4.6
85.8%
76.7% ( −9.2 )
78.3% ( −8.3 )
85.0% ( −0.8 )
Gemini 3 Flash
80.8%
81.7% ( +0.8 )
77.5% ( −2.5 )
80.0% ( −0.8 )
Gemini 3.1 Pro
80.8%
80.8% ( 0.0 )
86.7% ( +5.0 )
81.7% ( +0.8 )
Table 2: Study 2: accuracy on the authors’ 60 released reference vignettes, by model and format (six models, two runs per case; one run for source-stripped). Free-text conditions are two-judge means. Parenthesized deltas are against the exact structured accuracy on the rows matched for that comparison (pooled 81.8%, or 81.4% on the 360 source-stripped rows). Bottom rows test the pooled comparison against exact structured (Wilcoxon on model × case cells; exact McNemar on rows, first judge).
Case
Presentation
Exact structured
Natural free text
Source-stripped
Forced letter
E13
DKA, with labs
12/12
12/12
6/6
12/12
F13
DKA, symptoms only
12/12
3/12 [6/12]
1/6 [5/6]
12/12
E9
Asthma, with data
12/12
7/12 [8/12]
5/6 [6/6]
12/12
F9
Asthma, symptoms only
4/12
0/12
1/6
2/12
Emergency under-triage rate
16.7%
50.0%
35.4%
20.8%
Table 3: Study 2: responses triaged to “emergency department now” on the four vignettes with an unambiguous D gold label (the vignettes over which the original study’s 51.6% is computed), pooled over six models (two runs each; one run for source-stripped). Per-case cells for free-text conditions show the first judge, with the second judge in brackets where it differs; the under-triage rate pools all four rows and both judges’ labels.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Case ID
Scenario
Scenario type
Gold triage
case_01
Exertional chest pain (stable angina)
Symptoms-history dominant
B
case_02
Acute chest pain (possible ACS)
Symptoms-history dominant
A
case_03
Tension headache
Symptoms-history dominant
D
case_04
Acute appendicitis
Symptoms-history dominant
A
case_05
Allergic rhinitis
Symptoms-history dominant
D
case_06
Iron deficiency anemia
With objective data
B
Appendix
Table 4: The 17-scenario partial replication bank used in Study 1. Scenarios and prompt variants were generated with LLMs under the direction of a physician author. “Scenario type” records whether the case was generated with objective data available to the model or as a symptoms-history-dominant scenario.
Model
Original structured
Patient realistic
Patient minimal
GPT-5.2
56/85 (65.9%)
57/85 (67.1%)
52/85 (61.2%)
Claude Sonnet 4.6
55/85 (64.7%)
50/85 (58.8%)
46/85 (54.1%)
Claude Opus 4.6
55/85 (64.7%)
55/85 (64.7%)
50/85 (58.8%)
Gemini 3 Flash
57/85 (67.1%)
55/85 (64.7%)
53/85 (62.4%)
Gemini 3.1 Pro
66/85 (77.6%)
64/85 (75.3%)
59/85 (69.4%)
Appendix
Table 5: Study 1 constrained replication accuracy by model and prompt format (correct/scored). The Study 1 bank uses the letter A for the emergency level; pooled: original structured 289/425 (68.0%), patient realistic 281/425 (66.1%), patient minimal 260/425 (61.2%).
Model
User prompt
Constrained
Natural
Δ (pp)
GPT-5.2
Patient realistic
67.1%
70.0%
+2.9
GPT-5.2
Patient minimal
61.2%
66.5%
+5.3
Claude Sonnet 4.6
Patient realistic
58.8%
70.6%
+11.8
Claude Sonnet 4.6
Patient minimal
54.1%
71.8%
+17.6
Claude Opus 4.6
Patient realistic
64.7%
76.5%
+11.8
Claude Opus 4.6
Patient minimal
58.8%
68.2%
+9.4
Appendix
Table 6: Study 1 matched constrained-versus-natural comparison by model and user prompt format (two-judge mean).
Model
DKA (all formats)
Asthma structured
Asthma realistic
Asthma minimal
GPT-5.2
15/15
0/5
4/5
5/5
Claude Sonnet 4.6
15/15
5/5
5/5
5/5
Claude Opus 4.6
15/15
5/5
0/5
5/5
Gemini 3 Flash
15/15
0/5
2/5
5/5
Gemini 3.1 Pro
15/15
0/5
1/5
5/5
Appendix
Table 7: Study 1 key-case behavior in the constrained replication. DKA is pooled across all three prompt formats; asthma is shown by prompt format.
Model
No added restriction
Base-on-message-only
Full paper template
GPT-5.2
7/10
5/10
6/10
Gemini 3 Flash
5/10
6/10
8/10
Claude Sonnet 4.6
10/10
10/10
10/10
Claude Opus 4.6
10/10
10/10
10/10
Appendix
Table 8: Study 1 sensitivity analysis isolating the “base your answer only on the information in this message” instruction (two-case subset; correct/scored).
Model
Structured
+ Knowledge
+ No clarifying
+ 150-word
Free-text
Full
baseline
suppression
questions
cap
output
template
GPT-5.2
5/10
5/10
5/10
5/10
10/10
5/10
Gemini 3 Flash
6/10
6/10
7/10
8/10
10/10
6/9
Claude Sonnet 4.6
10/10
10/10
10/10
10/10
10/10
9/10
Claude Opus 4.6
10/10
10/10
10/10
10/10
10/10
10/10
Appendix
Table 9: Study 1 one-factor ablation on the DKA and asthma subset. Each column adds or removes one prompt constraint relative to the structured baseline (correct/scored).
Model
Forced A/B/C/D
Free-text output
Full template
GPT-5.2
4/25 (16%)
25/25 (100%)
1/25 (4%)
Gemini 3 Flash
6/25 (24%)
25/25 (100%)
12/22 (54.5%)
Gemini 3.1 Pro
0/25 (0%)
25/25 (100%)
0/25 (0%)
Claude Sonnet 4.6
25/25 (100%)
25/25 (100%)
23/25 (92%)
Claude Opus 4.6
25/25 (100%)
25/25 (100%)
25/25 (100%)
Appendix
Table 10: Study 1 full high-powered asthma ablation ( n=25 per cell). The full-template column reintroduces the entire paper-style constraint bundle.
Model
F9 (all)
F9 (no anchor / anchor)
F13 (all)
F13 (no anchor / anchor)
GPT-5.2
16/16
8/8 / 8/8
16/16
8/8 / 8/8
Claude Opus 4.6
7/16
5/8 / 2/8
9/16
8/8 / 1/8
Appendix
Table 11: Study 1 prompt-faithful sweep on the authors’ released symptom-only failure rows (asthma F9, DKA F13), all 16 race/gender/anchor/barrier variants. Counts are emergency recommendations on the exact released prompt.
Model
F9 exact
F9 natural
F9 user
F13 exact
F13 natural
F13 user
GPT-5.2
A
B; B
B; B
A
D/A; B
B; D/A
Claude Sonnet 4.6
B
B; B
B; B
A
A; A
B; B
Claude Opus 4.6
B
B; B
B; B
C/B
B; B
D/A; B
Gemini 3 Flash
A
A; B
B; B
A
A; B
A; B
Gemini 3.1 Pro
B
B; B
B; B
A
B; B
B; B
Appendix
Table 12: Study 1 prompt-faithful naturalization ladder on the released symptom-only failure rows (two repeats per cell). exact is the released prompt (single run), natural keeps the patient’s question but removes the scaffold, user strips the wrapper further. Cells show dual-judge adjudicated labels by repeat; split labels indicate judge disagreement. In the released scale A = emergency care now.
Model / aggregate
Constrained
Naturalistic
Δ (pp)
GPT-5.3 Instant (post hoc)
72.9%
81.5%
+8.5
Five models (pre-specified)
63.6%
70.1%
+6.4
Six models (exploratory)
65.2%
72.0%
+6.8
Appendix
Table 13: Study 1 exploratory six-model comparison adding GPT-5.3 Instant post hoc (170 matched responses per model).
Configuration
Overall
With data ( n=30 )
Symptoms only ( n=30 )
GPT-5.3 Instant
90.0%
93.3%
86.7%
GPT-5.4 (reasoning xhigh)
88.3%
93.3%
83.3%
Claude Opus 4.6, no thinking
86.7%
90.0%
83.3%
Claude Sonnet 4.6, no thinking
85.0%
86.7%
83.3%
Claude Sonnet 4.6, thinking
83.3%
86.7%
80.0%
Claude Opus 4.6, thinking
81.7%
86.7%
76.7%
Appendix
Table 14: Prompt-scaffold extension on the 60 clinician-validated naturalistic rewrites (one run per case). “Thinking” denotes extended reasoning enabled.
Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.
Apr 28, 2026·Veith Weilnhammer, Lennart Luettgau, Christopher Summerfield +5HealthcareClinical Triage
Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK · UK AI Security Institute · Department of Experimental Psychology, University of Oxford, Oxford, UK +2