Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Culverhouse College of Business, The University of Alabama, Tuscaloosa, AL, USA · Business Administration, Chang’an University, Xi’an, Shaanxi, China
While online medical consultation broadens access, unprompted patient disclosures and topic shifts frequently disrupt clinician inquiry, leaving clinical states progressively revealed. We characterize these dynamics using both existing evaluation metrics and task-specific interaction metrics designed in this work. Through an empirical study across 16 medical dialogue datasets, we show that authentic online interactions exhibit substantially lower contextual continuity and higher unpredictability than offline and simulated benchmarks. We propose PRISM (Policy Reasoning with Inferred States for Medicine), a framework inspired by partially observable Markov decision processes that decouples latent state estimation from action selection to address drifting dialogue trajectories. Specifically, PRISM factorizes clinical triage into belief estimation, policy optimization, and response realization. By maintaining an evolving posterior belief over consultation states and conditioning action selection on the maximum a posteriori state, PRISM advances clinical goals while accommodating patient-initiated shifts. Extensive evaluations show that PRISM substantially outperforms frontier LLMs and clinical agents, achieving a 22.40 percentage point gain in clinical decision accuracy alongside decisive improvements in response generation, surpassing the strongest baseline by 5.51 points in BERTScore_F1. Furthermore, ablation studies show that providing oracle consultation states consistently boosts response quality across all evaluated baselines by guiding their action reasoning, yielding average gains of 5.58 points in BERTScore_F1.
Figures & tables
Attributes
Samples used
Utterance
Discourse (Patient)
Discourse (Physician)
Corpus
Dataset
Type
Mode
Dial.
Utt.
ECoh ↓
BPB ↑
Discl. ↑
Redir. ↑
Any ↑
Cond. ↑
Colla. ↑
MAUVE ↑
MedSynth
Sim.
Off.
5.00k
242.38k
0.924
/
0.044
0.007
0.050
0.375
0.019
/
ACI-Bench
Sim.
Off.
0.21k
11.21k
0.809
/
0.079
0.013
0.086
0.387
0.034
/
MediTOD
Sim.
Off.
0.21k
20.31k
0.905
/
0.079
0.003
0.081
0.457
0.035
/
NoteChat
Sim.
Off.
5.00k
142.90k
0.903
/
0.088
0.005
0.091
0.432
0.040
/
MDD-5k ∗
Sim.
Off.
4.63k
214.36k
0.932
0.761
0.168
0.004
0.168
0.256
0.043
0.005
Table 1. Interaction dynamics of the evaluated samples (rates in [0,1] , BPB in bits/byte). ∗ denotes Chinese corpora. Arrows indicate support directions. Shaded rows report group means.
Figure 1. Overview of PRISM. Overview of PRISM.
Model
EM
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
chrF
Sem.
BertScore P
BertScore R
BertScore F1
Qwen3.8-27B
0.00
22.07
8.53
18.88
3.07
8.52
58.96
61.97
73.44
67.12
Qwen3.8-Max
0.00
21.55
8.53
18.54
3.28
8.87
59.03
61.35
73.30
66.71
DeepSeek-V4-Pro
0.00
22.76
9.46
19.72
3.48
9.32
59.01
61.79
73.63
67.10
MiniMax-M3
0.00
22.03
9.37
19.27
3.40
9.38
59.49
60.65
73.17
66.23
GPT-6-Astra
0.04
21.00
8.57
17.96
2.93
8.59
58.62
60.06
72.63
65.65
Codex
0.00
23.30
8.92
19.85
3.61
9.21
60.05
62.43
73.71
67.53
Table 2. Next-question generation performance of PRISM and baseline systems.
Model
Acc. ↑
Recall ↑
Precision ↑
F1 ↑
Qwen3-8B-SFT
43.07
43.07
47.88
40.95
Qwen3.8-27B
46.19
46.19
51.70
44.40
Qwen3.8-Max
51.76
51.76
60.68
48.31
DeepSeek-V4-Pro
53.65
53.65
60.18
51.82
MiniMax-M3
53.99
53.99
60.44
51.68
GPT-6-Astra
50.98
50.98
58.58
48.24
Table 3. Clinical decision performance.
Figure 2. Response generation with and without oracle state policy guidance. Labels indicate score gains. Response generation with and without oracle state policy guidance. Labels indicate score gains.
Large language models (LLMs) have made substantial progress on medical question-answering, yet effective medical dialogue also requires learning to ask questions that uncover relevant patient information. To train such dialogue policies, a common pipeline combines supervised fine-tuning with reinforcement learning (RL) based on final diagnostic correctness. However, this outcome-based supervision does not directly distinguish the contributions of individual questions and provides no question-level feedback for unexecuted alternatives. To address this gap, we introduce PCQC (Privileged Counterfactual Question Credit), which uses privileged patient information during training to learn from questions never asked. During training, PCQC makes alternative questions directly comparable at the same dialogue state by using privileged patient facts to construct their answers. A frozen diagnostic scorer evaluates the diagnostic utility of each resulting question-answer pair by how strongly it supports the correct diagnosis. PCQC turns these comparisons into relative question credit that teaches the policy which questions to favor, directly supervising both executed and unexecuted questions alongside outcome-based RL without requiring complete rollouts for the unexecuted alternatives. Extensive experiments across four medical benchmarks demonstrate that PCQC achieves 63.10% mean diagnostic accuracy, outperforming GRPO and ATPO by 4.38 and 4.21 percentage points, respectively. These gains are achieved with 33.1% fewer inquiry turns than GRPO.
Chenxuan Li, Jiayi Wan, Xinrong Chen +3
Peking University, Beijing, China · Department of Medical Bioinformatics, School of Basic Medical Sciences, Peking University, Beijing, China
Personalized dialogue requires more than recalling explicit user histories: systems also need to infer hidden user states that evolve through interaction and shape appropriate response strategies. Existing memory- and profile-based methods primarily reuse observable user information, offering limited support for modeling user-state dynamics or selecting actions based on how they shape future user states. We propose PUMA (Prospective User-state Modeling for Action selection), a framework grounded in the Free Energy Principle (FEP) that formulates personalization as decision-making under partial observability, centered on an explicit user state model that captures latent user states and their action-conditioned dynamics. At each turn, PUMA maintains a belief over the user's hidden state, refines the user state model for observation generation and action-conditioned state transition, and selects dialogue actions by minimizing expected free energy, balancing epistemic and pragmatic objectives under a unified criterion. This formulation shifts personalization from passive memory retrieval to model-based decision-making over user evolution. We instantiate PUMA on healthcare-oriented counseling and motivational interviewing benchmarks with latent state annotations for rigorous evaluation. Experiments show that PUMA improves long-horizon dialogue outcomes while maintaining strong response quality, and a cross-dataset study demonstrates more reliable user-state estimation and next-state prediction.
Jiani Luo, Xiaoyan Zhao, Yang Zhang +4
School of Computing, National University of Singapore, Singapore · School of Artificial Intelligence, Beihang University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China +1
Large language models (LLMs) are increasingly used for medical consultation and health information support, where safety depends not only on medical knowledge but also on robust responses to unclear, inconsistent, or misleading patient input. However, most existing medical LLM evaluations assume idealized and well-posed patient questions, limiting their realism. We study challenging patient behaviors that commonly arise in real medical consultations and complicate safe clinical reasoning. We define four clinically grounded categories of such behaviors: information contradiction, factual inaccuracy, self-diagnosis, and care resistance. For each behavior, we specify concrete failure criteria that capture unsafe responses. Building on four existing medical dialogue datasets, we introduce CPB-Bench (Challenging Patient Behaviors Benchmark), a bilingual (English and Chinese) benchmark of multi-turn dialogues annotated for these behaviors. We find that although models perform well overall, they exhibit consistent behavior-specific failures, especially when handling contradictory or medically implausible patient information. We further evaluate four intervention strategies and find inconsistent improvements, with some interventions introducing unnecessary corrections.