Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue

Chenxuan Li; Xinrong Chen; Luyan Zhang; Peidong Jia; Runfan Zheng; Zhongyu Zhao; Xuecheng Shang; Peixing Wan

cs.CLAug 25, 2026

Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue

Authors: Chenxuan Li, Xinrong Chen, Luyan Zhang, Peidong Jia, Runfan Zheng, Zhongyu Zhao, Xuecheng Shang, Peixing Wan

Organizations: Peking University, Beijing, China · Northeastern University, Boston, MA, USA · Sun Yat-sen University, Guangzhou, China · Department of Medical Bioinformatics, School of Basic Medical Sciences, Peking University, Beijing, China

Abstract

Proactive medical dialogue requires an agent to decide what to ask from incomplete patient information. Existing information-seeking approaches commonly prioritize questions that most reduce diagnostic uncertainty, but this criterion overlooks an important property of medical diagnosis: different diagnostic errors can carry substantially different consequences. The most informative question may therefore differ from the one most valuable for the downstream decision. We propose Expected-Severity-Risk (ESR), a consequence-aware question-supervision objective that values each candidate by its expected reduction in severity-aware terminal risk. Because questions must be selected before their answers are observed, ESR marginalizes over possible answers using train-only population statistics. Its rankings are then distilled into a prefix-only language policy, requiring no teacher-side risk computation at deployment. Across three matched Qwen3-4B training seeds on DDxPlus, ESR reduces mean high-severity diagnostic miss from 0.0645 to 0.0455 (29.5% relative reduction) and improves mean diagnostic accuracy from 0.9123 to 0.9320 while requiring only 0.14 additional questions per dialogue. Fixed-budget analyses show that the distinction persists when question count is controlled, while a matched expected-0/1-risk student control further isolates the contribution of asymmetric severity weighting. These results support moving proactive medical dialogue beyond uncertainty reduction toward consequence-aware evidence acquisition.

Figures & tables

Figure 1: Information value and diagnostic consequence can prioritize different questions. ESR evaluates candidate questions before their answers are observed and prioritizes evidence according to expected reduction in severity-aware terminal risk. The current patient’s hidden answer is not used during question ranking.

Method	Acc. $\uparrow$	Q	HSM $\downarrow$	Sev. $\downarrow$	Pop.-HS $\downarrow$	Cost $\downarrow$	Cap $\downarrow$
E-Entropy	$.9123\!\pm\!.0015$	$2.182\!\pm\!.020$	$.0645\!\pm\!.0000$	$.0447\!\pm\!.0003$	$.0430\!\pm\!.0000$	$.2377\!\pm\!.0014$	$.0073\!\pm\!.0015$
ESR	$\mathbf{.9320\!\pm\!.0017}$	$2.322\!\pm\!.025$	$\mathbf{.0455\!\pm\!.0048}$	$\mathbf{.0333\!\pm\!.0015}$	$\mathbf{.0303\!\pm\!.0032}$	$\mathbf{.1886\!\pm\!.0008}$	$.0177\!\pm\!.0038$
$\Delta$ ( ESR $-$ E-Entropy )	$+.0197\!\pm\!.0025$	$+.140\!\pm\!.015$	$-.0190\!\pm\!.0048$	$-.0113\!\pm\!.0019$	$-.0127\!\pm\!.0032$	$-.0491\!\pm\!.0021$	$+.0103\!\pm\!.0051$

Table 1: Main shared-stopping comparison across three seeds. Values are mean

\pm

SD over seeds 42/43/44; only next-question supervision differs.

\Delta

is the seed-wise ESR

-

E-Entropy difference.

Method	Acc. $\uparrow$	Q	HSM $\downarrow$	Sev. $\downarrow$
E-Entropy	.911	2.161	.0645	.0447
Exp.-0/1-Risk	.913	2.509	.0690	.0447
ESR	.931	2.294	.0435	.0337

Table 2: Frozen seed-42 student objective decomposition. All methods use expected-answer marginalization.

Explore similar work

PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue
Sep 23, 2026Chenxuan Li, Jiayi Wan, Xinrong Chen +3Clinical Reasoning Training Dialogue Benchmarks
Peking University, Beijing, China · Department of Medical Bioinformatics, School of Basic Medical Sciences, Peking University, Beijing, China
CARE-Bench: Benchmarking Patient-Facing LLM Triage
Aug 4, 2026Yining Hua, Hongbin Na, Cyrus AyubchaTriage Disclosures
1Harvard University · University of Technology Sydney
ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs
Aug 19, 2025Hongxin Ding, Baixiang Huang, Yue Fang +8Offline Reinforcement Learning
National Engineering Research Center of Software Engineering, Peking University, China · School of Computing and Data Science, The University of Hong Kong · School of Computer Science, Peking University, Beijing, China +2

Sep 23, 2026, cs.LGJ/K move · Enter open · S save

PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue

Chenxuan Li, Jiayi Wan, Xinrong Chen, Zhongyu Zhao+2

Peking University, Beijing, China · Department of Medical Bioinformatics, School of Basic Medical Sciences, Peking University, Beijing, China

Clinical Reasoning Training Dialogue Benchmarks

Open paper AskPDF arXiv

Large language models (LLMs) have made substantial progress on medical question-answering, yet effective medical dialogue also requires learning to ask questions that uncover relevant patient information. To train such dialogue policies, a common pipeline combines supervised fine-tuning with reinforcement learning (RL) based on final diagnostic correctness. However, this outcome-based supervision does not directly distinguish the contributions of individual questions and provides no question-level feedback for unexecuted alternatives. To address this gap, we introduce PCQC (Privileged Counterfactual Question Credit), which uses privileged patient information during training to learn from questions never asked. During training, PCQC makes alternative questions directly comparable at the same dialogue state by using privileged patient facts to construct their answers. A frozen diagnostic scorer evaluates the diagnostic utility of each resulting question-answer pair by how strongly it supports the correct diagnosis. PCQC turns these comparisons into relative question credit that teaches the policy which questions to favor, directly supervising both executed and unexecuted questions alongside outcome-based RL without requiring complete rollouts for the unexecuted alternatives. Extensive experiments across four medical benchmarks demonstrate that PCQC achieves 63.10% mean diagnostic accuracy, outperforming GRPO and ATPO by 4.38 and 4.21 percentage points, respectively. These gains are achieved with 33.1% fewer inquiry turns than GRPO.

Abstract

Explore similar work

PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue

CARE-Bench: Benchmarking Patient-Facing LLM Triage

ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs