Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence

Mingzhan Yang; Weili Wu

cs.AIOct 3, 2026

Decide, Ask, or Defer: Clinical LLMs under Incomplete Evidence

Authors: Mingzhan Yang, Weili Wu

Organizations: Department of Computer Science University of Texas at Dallas Richardson, TX 75080

Abstract

Clinical LLMs must decide not only what diagnosis to produce, but also whether the available evidence is sufficient for autonomous decision making. Binary DECIDE/ABSTAIN formulations merge distinct non decision states and do not explicitly evaluate information acquisition. We introduce a DECIDE/ASK/DEFER formulation together with a blinded protocol that prevents models from using evidence completeness metadata. We evaluate Qwen, Gemini, and GPT on 200 matched clinical evidence states constructed from DDXPlus. The models show substantial differences in action selection under identical evidence, with disagreement in 137 of 200 states. For Qwen, a matched targeted versus random analysis shows that selected information changes the likelihood of a subsequent autonomous decision more clearly than diagnostic correctness. Its matched DECIDE/ABSTAIN baseline further reveals a safety autonomy tradeoff: the three action policy rescues some erroneous autonomous decisions but also removes some correct autonomous deci sions. These results show that separating information acquisition from clinician deferral exposes behavior that binary abstention hides, without yielding a uniformly improved decision policy.

Figures & tables

Figure 1: Comparison between binary abstention and the proposed three action formulation. In the binary setting, Abstain merges cases in which additional information may resolve uncertainty with cases that require clinician involvement. Our formulation separates these outcomes into Ask , which acquires one additional case specific evidence item, and Defer , which transfers the case to a clinician. After one Ask action, the model makes a final Decide or Defer action.

Metric	Numerator	Denominator
Conditional decision accuracy	Correct Decide outcomes	All Decide outcomes
Autonomous coverage	All Decide outcomes, correct or incorrect	All evaluated states $N$
Correct autonomous coverage	Correct Decide outcomes	All evaluated states $N$
Wrong decision rate	Incorrect Decide outcomes	All evaluated states $N$
Unsafe premature decision rate	Incorrect Decide outcomes with evidence level $<100\%$	All incomplete evidence states

Table 1: Primary decision metrics.

N

denotes the number of evaluated clinical states.

Figure 2: Stage 1 action rates across initial evidence levels for Qwen, Gemini, and GPT. Unfulfillable Ask denotes a Stage 1 Ask action when no valid item remains in the case specific hidden evidence pool.

Model pair	Disagreement	$\boldsymbol{\chi^{2}}$	Holm adjusted $p$
Qwen vs. Gemini	50.5%	93.15	$1.78\times 10^{-20}$
Qwen vs. GPT	61.5%	89.00	$9.46\times 10^{-20}$
Gemini vs. GPT	33.5%	46.59	$7.63\times 10^{-11}$

Table 2: Pairwise comparison of Stage 1 action policies across matched clinical states.

Metric	Targeted	Random	Diff.	95% CI	$p$
Post Ask Decide	67.62%	57.14%	+10.48 pp	[0.95, 20.00]	.052
Correct Decide	43.81%	38.10%	+5.71 pp	[-2.86, 14.29]	.263
Conditional accuracy	64.79%	66.67%	-1.88 pp
Defer	32.38%	42.86%	-10.48 pp

Table 3: Qwen targeted versus random information acquisition over 105 matched executable Ask states.

Figure 3: Matched comparison between the Qwen three action policy and its Decide / Abstain baseline. Panel (a) shows changes in safety and autonomy metrics under identical evidence states. Panel (b) summarizes matched transition outcomes. Error bars in (a) are case clustered 95% bootstrap confidence intervals. Defer is treated as a non decision outcome, and no ground truth appropriateness label is assumed.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Qwen provider	Groq
Qwen model ID	qwen/qwen3.8 27b
Qwen temperature	0
Qwen output limit	Provider default
Qwen other settings	Top $p$ and reasoning settings were not explicitly set.
Gemini provider	Vertex AI

Appendix

Table 4: Model invocation settings used in the formal evaluation.

Analysis	Targeted DECIDE	Random DECIDE	Targeted correct	Random correct
All pairs ( $N=105$ )	67.62%	57.14%	43.81%	38.10%
Distinct items ( $N=81$ )	62.96%	49.38%	43.21%	35.80%

Appendix

Table 5: Sensitivity analysis for targeted versus random information acquisition in Qwen. The primary analysis includes all 105 executable Ask pairs; the distinct item analysis excludes 24 pairs in which targeted and random evidence selections are identical.

Outcome	Count	Rate
Targeted win	16	19.75%
Tie	49	60.49%
Targeted loss	16	19.75%

Appendix

Table 6: Pair level targeted versus random outcome comparison for the 81 distinct item Qwen pairs. Wins, ties, and losses are defined from the targeted policy perspective using the ordering correct Decide

>

Defer

>

incorrect Decide .

Agreement pattern	Count	Rate
All three models agree	63	31.5%
Exactly two models agree	120	60.0%
All three models differ	17	8.5%
Any disagreement	137	68.5%

Appendix

Table 7: Cross model agreement across the 200 matched clinical evidence states.

Model pair	Disagreement	95% CI
Qwen vs. Gemini	50.5%	[42.5, 59.0]
Qwen vs. GPT	61.5%	[53.5, 69.0]
Gemini vs. GPT	33.5%	[25.0, 42.5]

Appendix

Table 8: Pairwise Stage 1 disagreement across matched states. Confidence intervals are estimated by case clustered bootstrap resampling.

Model pair	$\boldsymbol{\chi^{2}}$	df	Holm adjusted $p$
Qwen vs. Gemini	93.15	2	$1.78\times 10^{-20}$
Qwen vs. GPT	89.00	2	$9.46\times 10^{-20}$
Gemini vs. GPT	46.59	2	$7.63\times 10^{-11}$

Appendix

Table 9: Pairwise Stuart Maxwell tests for Stage 1 action distributions.

Model pair	Decide	Ask	Defer
Qwen Gemini	$-46.5$	$+31.5$	$+15.0$
Qwen GPT	$-35.0$	$+44.0$	$-9.0$
Gemini GPT	$+11.5$	$+12.5$	$-24.0$

Appendix

Table 10: Pairwise differences in Stage 1 action rates, expressed in percentage points.

Model pair	Action	Raw $p$	Holm adjusted $p$
Qwen vs. Gemini	Decide	$2.019\times 10^{-28}$	$1.818\times 10^{-27}$
Qwen vs. Gemini	Ask	$3.425\times 10^{-15}$	$2.055\times 10^{-14}$
Qwen vs. Gemini	Defer	$2.272\times 10^{-7}$	$9.089\times 10^{-7}$
Qwen vs. GPT	Decide	$5.138\times 10^{-16}$	$3.596\times 10^{-15}$
Qwen vs. GPT	Ask	$1.399\times 10^{-23}$	$1.119\times 10^{-22}$
Qwen vs. GPT	Defer	$0.03846$	$0.03846$

Appendix

Table 11: Action specific exact McNemar tests across matched Stage 1 states. Holm correction is applied across all nine comparisons.

Qwen	Gemini	GPT	Count
Ask	Decide	Decide	45
Defer	Decide	Decide	21
Ask	Ask	Defer	19

Appendix

Table 12: Most frequent cross model Stage 1 action patterns.

Initial evidence	Decide	Abstain
25%	21	29
50%	33	17
75%	39	11
100%	44	6
Overall	137	63

Appendix

Table 13: Qwen binary baseline action counts across the four evidence levels.

Metric	Value
Autonomous coverage	68.5%
Abstain rate	31.5%
Conditional decision accuracy	76.64%
Correct autonomous coverage	52.5%
Wrong decision rate	16.0%
Unsafe premature decision rate	15.33%

Appendix

Table 14: Qwen binary baseline performance metrics.

Metric	Binary	Three action	Difference
Autonomous coverage	68.5%	54.0%	$-14.5$ pp
Correct autonomous coverage	52.5%	39.0%	$-13.5$ pp
Wrong decision rate	16.0%	15.0%	$-1.0$ pp

Appendix

Table 15: Matched Qwen final policy comparison. Differences are three action minus binary baseline.

Explore similar work

Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
May 6, 2026Oriana Presacan, Andreea Grama, Larisa Irimină +6Mental Health Diagnosis
National University of Science and Technology Politehnica Bucharest · Psychiatric Hospital Doctor Gheorghe Preda · Oslo Metropolitan University +3
Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support
Jul 29, 2026Shayndhan Sivanathan, Shravan Nageswaran, Mehdi Zadem +10Triage Clinical Decision Support
Atman Labs · Oxford University Hospitals · Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford +11
Quantifying and Mitigating Premature Closure in Frontier LLMs
May 14, 2026Rebecca Handler, Suhana Bedi, Nigam ShahLarge Language Models Fail Closure
Department of Medicine, Stanford University · Department of Biomedical Data Science, Stanford University

May 6, 2026, cs.CLJ/K move · Enter open · S save

Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry

Oriana Presacan, Andreea Grama, Larisa Irimină, Alireza Nik+5

National University of Science and Technology Politehnica Bucharest · Psychiatric Hospital Doctor Gheorghe Preda · Oslo Metropolitan University+3

Mental Health Diagnosis Psychology

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used for decision support in healthcare, but clinical evidence is often incomplete or evolving. When the available information is insufficient to support a reliable answer, models should request clarification or abstain rather than provide unsupported responses. Existing medical benchmarks, however, typically assume that complete information is available upfront. We introduce Safe-Psych, a sequential benchmark for evaluating how LLMs handle evolving diagnostic uncertainty in clinical psychiatry. Safe-Psych contains over 1,000 real-world psychiatric clinical notes segmented to simulate incremental evidence disclosure, with psychiatrist-derived action labels at each stage: DIAGNOSE, CLARIFY, or ABSTAIN. We evaluate multiple state-of-the-art LLMs in full-information and sequential settings. Our findings show that capability does not ensure calibration: even strong models struggle under incomplete clinical information, with under-abstention exceeding 60% for most models and safety-aware prompting reducing premature commitment only by shifting errors toward excessive abstention. In sequential evaluation, models frequently diagnose before sufficient evidence is available and rarely seek clarification unless explicitly prompted; these premature diagnoses are less accurate than on-time diagnoses. Overall, Safe-Psych reveals a limitation across the evaluated models: recognizing when clinical evidence is incomplete and additional information is needed. We release Safe-Psych to support research on improving LLM safety in healthcare.

Abstract

Explore similar work

Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

Quantifying and Mitigating Premature Closure in Frontier LLMs