From Uncertainty to Clinical Risk: Severity-Aware Conformal Planning for Interactive Medical Diagnosis
Authors: Yue Zhou, Haiyang Zhou, Jin Zhang, Kong Wang, Yongxin Ni, Youhua Li, Hanwen Du
Organizations: Lanzhou University, China · Hunan University, China · National University of Singapore, Singapore · City University of Hong Kong, Hong Kong, China · The Ohio State University, USA
Interactive medical diagnosis dynamically acquires patient information through multiple rounds of questioning, supporting accurate, efficient, and safe clinical decisions under incomplete evidence. Existing methods commonly guide information acquisition with predictive uncertainty or label ambiguity, but overlook the asymmetric clinical risk of missing severe diseases and lack unified long-horizon planning over whether to continue asking questions or commit to a diagnosis. To address these limitations, we propose Severity-Aware Conformal Clinical Planning, which formulates interactive diagnosis as a risk-sensitive sequential decision problem. The framework maintains complementary diagnostic, safety, and masked-evidence beliefs; calibrates turn-specific diagnostic prediction sets and severity-weighted differential-diagnosis risk on held-out diagnostic trajectories; and introduces the calibrated clinical risk into Monte Carlo Tree Search to jointly evaluate long-horizon Ask and Commit trajectories. Experiments on DDXPlus and MediQ show that our method achieves more accurate diagnoses with fewer questions across multiple large language models, while improving differential-diagnosis quality and reducing high-risk errors in severe cases. These findings validate the value of using clinical risk, rather than predictive uncertainty alone, as a planning signal and demonstrate the effectiveness of the proposed framework for information acquisition and risk-aware diagnostic decision making. They also motivate future work on clinical-risk-oriented interactive diagnosis and information-acquisition methods.
Figures & tables
Figure 1: Overview of the proposed framework.
Model
Method
Top-1 ↑
Top-3 ↑
Avg. Queries ↓
DDF1 ↑
DSF1 ↑
Severe Error ↓
Qwen3-4B
NA/DC
32.24%
54.49%
0
0.385
0.279
61.18%
ATD
54.90%
84.90%
4
0.366
0.265
45.29%
UoT
65.92%
87.35%
5.38
0.374
0.247
32.35%
MISQ-HF
66.94%
87.14%
6.14
0.372
0.238
32.35%
C-IP
68.57%
82.45%
9.28
0.360
0.236
23.53%
ACTMED
89.39%
97.96%
7.46
0.354
0.184
8.24%
Table 1: Overall performance on DDXPlus across three LLM backbones. NA/DC denotes No Ask / Direct Commit. The best and second-best values within each backbone are highlighted in bold and underlined, respectively.
Method
Top-1 ↑
Avg. Queries ↓
DDF1 ↑
DSF1 ↑
Severe-case Error ↓
w/o Tri-Belief
96.33%
3.90
0.333
0.205
1.76% (3/170)
w/o Conformal Calibration
98.16%
5.10
0.629
0.570
0.59% (1/170)
w/o Severity Weighting
97.96%
4.71
0.612
0.564
1.18% (2/170)
w/o Multi-Step MCTS ( H=1 )
97.76%
4.87
0.590
0.581
1.18% (2/170)
Full Model ( H=2 )
98.57%
4.77
0.596
0.592
0.00% (0/170)
Table 2: Ablation results on DDXPlus.
Metric
α=0.1
α=0.2
α=0.3
Top-1 ↑
98.57%
95.10%
90.61%
Top-3 ↑
99.59%
98.78%
93.67%
Avg. Queries ↓
4.77
5.05
4.12
DDF1 ↑
0.596
0.588
0.583
DSF1 ↑
0.592
0.569
0.558
Severe Error ↓
0.00%
3.53%
8.24%
Table 3: Sensitivity to the conformal miscoverage level α .
Figure 2: Case study of severe epiglottitis. Despite an initial Top-1 prediction of upper respiratory tract infection, calibrated clinical risk prevents premature commitment. Risk-guided questioning elicits discriminative evidence, shifts the Top-1 diagnosis to epiglottitis, and enables risk-aware commitment while retaining a clinically meaningful differential.
Method
Accuracy ↑
Avg. Queries ↓
Coverage ↑
Acc.-Turn AUC@10 ↑
Full-Info Gap ↓
Direct
49.76%
0
91.87%
49.76%
17.72 pp
Random
63.09%
6.07
94.15%
59.22%
4.39 pp
C-IP
63.09%
7.32
93.33%
58.73%
4.39 pp
Entropy-Greedy
64.88%
6.08
94.63%
60.09%
2.60 pp
UoT
64.88%
6.03
93.01%
59.57%
2.60 pp
Ours
66.18%
6.57
91.54%
60.76%
1.30 pp
Table 4: Overall performance on MediQ. Higher values are better for Accuracy, Coverage, and Acc.-Turn AUC@10, while lower values are better for Avg. Queries and Full-Info Gap. Best and second-best values among non-oracle interactive methods are shown in bold and underlined, respectively.