Organizations: School of Computer Science and Technology, Wuhan University of Science and Technology, China · Department of Cardiology, Tongji Hospital, Huazhong University of Science and Technology, China · Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands · School of Computer Science and Technology, Huazhong University of Science and Technology, China
Pre-consultation medical dialogues are essential for collecting comprehensive patient information, but are costly to obtain at scale due to privacy and ethical constraints. Existing LLM-based synthesis methods often rely on prompt engineering or rigid rules, limiting their ability to balance clinical relevance with realistic patient diversity. We propose CoRPS-Dial, a controllable dual-agent framework for pre-consultation medical dialogue synthesis that integrates patient personas with guideline-aligned clinical workflow rules. CoRPS-Dial generates dialogues through interactions between a persona-driven patient agent and a process-guided doctor agent, followed by post-hoc quality control for role consistency, workflow completeness, and textual consistency. Experiments on coronary heart disease demonstrate that CoRPS-Dial improves alignment between synthetic and real dialogues in information coverage, clinical compliance, and dialogue naturalness. Downstream fine-tuning experiments further confirm the utility of the generated synthetic data for medical dialogue tasks. These results highlight the effectiveness of CoRPS-Dial for clinically grounded and controllable medical dialogue synthesis.
Figures & tables
Figure 1: Overview of dialogue generation approaches. (a) Prompt-driven generation with limited clinical reliability. (b) Clinical workflow-driven generation with costly expert verification. (c) Intent-driven multi-agent simulation with real-time rule validation and iterative optimization requiring repeated LLM calls. (d) CoRPS-Dial, a patient persona and clinical process dual-driven framework with post-hoc validation and selective refinement.
Figure 2: CoRPS-Dial Framework. (a) Patient Persona Construction , deriving diverse profiles from real consultations capturing demographics, chief complaints, emotions, communication styles, and expression abilities; (b) Clinical Process Modeling , abstracting real consultations with guidelines into symptom elaboration, present illness history, past history, risk factors, social history and family history; (c) Dual-Agent Dialogue Generation , enabling multi-turn structured interactions between a persona-based patient agent and a process-based doctor agent; (d) Post-hoc Quality Control & Selective Refinement , applying rule-based validation, reward-based scoring, and targeted revisions for high-quality, grounded clinical dialogues.
Metric
CoRPS-Dial
Raw Doctor
Free Patient
Single Agent
Minimal Agents
No QC/SR
Structure
Turns/Dialog
44.00 ± 0.52
18.37 ± 2.41
41.26 ± 1.13
43.12 ± 1.39
27.84 ± 6.92
44.00 ± 0.00
Sent Len
52.99 ± 64.00
21.43 ± 18.92
47.85 ± 51.76
21.11 ± 8.21
38.14 ± 42.07
66.27 ± 158.96
Unique Chars
1106
742
983
579
1287
1209
Tokens/Turn
36.51
15.82
33.94
13.77
29.17
44.41
Semantic
Table 1: Dialogues generation quality comparison of structure, semantic, and diversity. “Sent Len” means sentence length. “Local Coh” means local coherence. “xxx ± xxx” denotes mean ± standard deviation (std).
Role Consistency
Process Consistency
Contextual Coherence
Basic Text Quality
Human
CoRPS-Dial
9.17 ± 0.38
9.43 ± 0.29
9.17 ± 0.38
9.56 ± 0.22
Raw Doctor
8.47 ± 0.62
7.61 ± 0.83
7.94 ± 0.67
8.83 ± 0.51
Free Patient
7.93 ± 0.71
9.11 ± 0.37
8.32 ± 0.59
9.12 ± 0.36
Single Agent
8.50 ± 0.68
6.28 ± 1.76
7.70 ± 0.79
8.64 ± 0.56
Minimal Agents
6.18 ± 1.24
5.84 ± 1.36
6.03 ± 1.41
7.41 ± 1.08
No QC/SR
8.95 ± 0.49
9.03 ± 0.21
8.71 ± 0.56
8.92 ± 0.47
Table 2: Human and Deepseek-v2 evaluation results compared with baselines across role consistency, process consistency, contextual coherence, and basic text quality. “xxx ± xxx” denotes mean ± std.
Metric
Real (256)
Syn (256)
Syn (1024)
Syn (2048)
Syn (4096)
Syn (8192)
Syn (9714)
Structure
Turns/Dialog
34.26 ± 12.3
42.51 ± 4.32
42.39 ± 4.32
42.33 ± 4.46
42.33 ± 4.49
42.24 ± 4.48
42.25 ± 4.50
Sent Len
33.53 ± 74.29
57.60 ± 64.80
58.30 ± 65.81
56.88 ± 64.18
57.02 ± 64.51
57.22 ± 64.81
57.29 ± 64.95
Unique Chars
1413
1510
1837
1999
2162
2311
2350
Tokens/Turn
23.64
37.05
37.42
36.48
36.60
36.72
36.77
Semantic
Table 3: Evaluation of dialogue quality across structure, semantics, and diversity for synthetic and real dialogues. “Syn” means synthetic. “Pairwise Sim” means pairwise dialogue similarity. “Paired Sim” means paired dialogue similarity.
Metric
LLaMA-8B
LLaMA-8B-SFT
DeepSeek-R1-8B
DeepSeek-R1-8B-SFT
DeepSeek-R1-14B
DeepSeek-R1-14B-SFT
Effective Coverage ↑
16.48%
46.83%
36.2%
23.17%
29.33%
44.65%
Efficiency ↑
0.75
2.13
1.65
1.05
1.33
2.03
Turns
22
21.98
22
22
22
22
Context Utilization ↑
23.35%
26.28%
27.13%
38.25%
26.86%
23.73%
Repetition Rate ↓
79.64%
19.33%
27.52%
66.60%
30.57%
7.55%
Invalid Question Rate ↓
90.12%
38.67%
38.52%
75.69%
45.27%
32.36%
Table 4: Performance comparison of three models before and after SFT. ↑ means higher values are better, and ↓ means lower values are better.
Figure 3: Human and LLM-as-a-Judge evaluations of 50 real and 50 synthetic doctor–patient dialogues on a 10-point Likert scale. Human scores are averaged over 3 domain experts (mean ± std), and LLM scores are from Deepseek-v2 across 3 epochs (mean ± std).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Examples of surface-level noise filtering in clinical dialogues.
Type
Scoring criteria
Score
Scoring Rule
Role Consistency
Perfect consistency, where each role behaves completely according to its expected identity and duties, displaying professional and appropriate actions throughout.
10
Flawless role adherence.
High consistency, where most dialogues align with role expectations, with only minor deviations in a few
7-9
9: No role violation, only trivial phrasing imperfections may occur.
instances.
8: 1–2 minor out-of-character instances (e.g., patient repeats medical terms).
7: ≥ 3 minor or 1 moderate deviation (e.g., patient self-diagnosis).
Minor inconsistencies, where most dialogues are role-appropriate, but occasional deviations occur.
4-6
6: Clear role overstepping but primary roles remain identifiable.
5: Frequent role overstepping ( > 3 moderate instances).
Appendix
Table 5: Specific scoring criteria for GLM-4-9B. We assess the quality of the synthetic dialogues in terms of role consistency, process consistency, contextual coherence, and basic text quality, then score them according to different criteria, and finally average them.
Metric
Real (256)
Syn (256)
Syn (1024)
Syn (2048)
Syn (4096)
Syn (5120)
Structure
Turns/Dialog
34.26 ± 12.30
42.18 ± 4.26
42.50 ± 4.40
42.10 ± 4.53
42.16 ± 4.52
42.14 ± 4.52
Sent Len
33.53 ± 74.29
56.41 ± 63.78
57.82 ± 64.68
58.15 ± 65.33
57.74 ± 65.33
57.66 ± 65.20
Unique Chars
1413
1514
1861
2014
2164
2211
Tokens/Turn
23.64
36.13
37.11
37.27
37.03
36.98
Semantic
Appendix
Table 6: Comparison of augmented profile-based dialogue generation quality in terms of structure, semantics, and diversity, with random sampling from augmented patient profiles, selecting one dialogue per profile. “Syn” means synthetic, “Sent Len” means sentence length. “Pairwise Sim” means pairwise dialogue similarity. “Paired Sim” means paired dialogue similarity. “Local Coh” means local coherence.
Figure 5: Cohen’s kappa inter-rater agreement across four evaluation dimensions for real and synthetic doctor–patient dialogues, comparing humans and human–LLM consistency.
Figure 6: Qualitative comparison of dialogues generated by baselines. Deep red indicates relevance, light red indicates weak relevance, and a yellow background indicates an issue.
University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India