Organizations: School of Computer Science and Technology, Wuhan University of Science and Technology, China · Department of Cardiology, Tongji Hospital, Huazhong University of Science and Technology, China · Department of Computer Science, Vrije Universiteit Amsterdam, Netherlands · School of Computer Science and Technology, Huazhong University of Science and Technology, China
Pre-consultation medical dialogues are essential for collecting comprehensive patient information, but are costly to obtain at scale due to privacy and ethical constraints. Existing LLM-based synthesis methods often rely on prompt engineering or rigid rules, limiting their ability to balance clinical relevance with realistic patient diversity. We propose CoRPS-Dial, a controllable dual-agent framework for pre-consultation medical dialogue synthesis that integrates patient personas with guideline-aligned clinical workflow rules. CoRPS-Dial generates dialogues through interactions between a persona-driven patient agent and a process-guided doctor agent, followed by post-hoc quality control for role consistency, workflow completeness, and textual consistency. Experiments on coronary heart disease demonstrate that CoRPS-Dial improves alignment between synthetic and real dialogues in information coverage, clinical compliance, and dialogue naturalness. Downstream fine-tuning experiments further confirm the utility of the generated synthetic data for medical dialogue tasks. These results highlight the effectiveness of CoRPS-Dial for clinically grounded and controllable medical dialogue synthesis.
Figures & tables
Figure 1: Overview of dialogue generation approaches. (a) Prompt-driven generation with limited clinical reliability. (b) Clinical workflow-driven generation with costly expert verification. (c) Intent-driven multi-agent simulation with real-time rule validation and iterative optimization requiring repeated LLM calls. (d) CoRPS-Dial, a patient persona and clinical process dual-driven framework with post-hoc validation and selective refinement.
Figure 2: CoRPS-Dial Framework. (a) Patient Persona Construction , deriving diverse profiles from real consultations capturing demographics, chief complaints, emotions, communication styles, and expression abilities; (b) Clinical Process Modeling , abstracting real consultations with guidelines into symptom elaboration, present illness history, past history, risk factors, social history and family history; (c) Dual-Agent Dialogue Generation , enabling multi-turn structured interactions between a persona-based patient agent and a process-based doctor agent; (d) Post-hoc Quality Control & Selective Refinement , applying rule-based validation, reward-based scoring, and targeted revisions for high-quality, grounded clinical dialogues.
Metric
CoRPS-Dial
Raw Doctor
Free Patient
Single Agent
Minimal Agents
No QC/SR
Structure
Turns/Dialog
44.00 ± 0.52
18.37 ± 2.41
41.26 ± 1.13
43.12 ± 1.39
27.84 ± 6.92
44.00 ± 0.00
Sent Len
52.99 ± 64.00
21.43 ± 18.92
47.85 ± 51.76
21.11 ± 8.21
38.14 ± 42.07
66.27 ± 158.96
Unique Chars
1106
742
983
579
1287
1209
Tokens/Turn
36.51
15.82
33.94
13.77
29.17
44.41
Semantic
Table 1: Dialogues generation quality comparison of structure, semantic, and diversity. “Sent Len” means sentence length. “Local Coh” means local coherence. “xxx ± xxx” denotes mean ± standard deviation (std).
Role Consistency
Process Consistency
Contextual Coherence
Basic Text Quality
Human
CoRPS-Dial
9.17 ± 0.38
9.43 ± 0.29
9.17 ± 0.38
9.56 ± 0.22
Raw Doctor
8.47 ± 0.62
7.61 ± 0.83
7.94 ± 0.67
8.83 ± 0.51
Free Patient
7.93 ± 0.71
9.11 ± 0.37
8.32 ± 0.59
9.12 ± 0.36
Single Agent
8.50 ± 0.68
6.28 ± 1.76
7.70 ± 0.79
8.64 ± 0.56
Minimal Agents
6.18 ± 1.24
5.84 ± 1.36
6.03 ± 1.41
7.41 ± 1.08
No QC/SR
8.95 ± 0.49
9.03 ± 0.21
8.71 ± 0.56
8.92 ± 0.47
Table 2: Human and Deepseek-v2 evaluation results compared with baselines across role consistency, process consistency, contextual coherence, and basic text quality. “xxx ± xxx” denotes mean ± std.
Metric
Real (256)
Syn (256)
Syn (1024)
Syn (2048)
Syn (4096)
Syn (8192)
Syn (9714)
Structure
Turns/Dialog
34.26 ± 12.3
42.51 ± 4.32
42.39 ± 4.32
42.33 ± 4.46
42.33 ± 4.49
42.24 ± 4.48
42.25 ± 4.50
Sent Len
33.53 ± 74.29
57.60 ± 64.80
58.30 ± 65.81
56.88 ± 64.18
57.02 ± 64.51
57.22 ± 64.81
57.29 ± 64.95
Unique Chars
1413
1510
1837
1999
2162
2311
2350
Tokens/Turn
23.64
37.05
37.42
36.48
36.60
36.72
36.77
Semantic
Table 3: Evaluation of dialogue quality across structure, semantics, and diversity for synthetic and real dialogues. “Syn” means synthetic. “Pairwise Sim” means pairwise dialogue similarity. “Paired Sim” means paired dialogue similarity.
Metric
LLaMA-8B
LLaMA-8B-SFT
DeepSeek-R1-8B
DeepSeek-R1-8B-SFT
DeepSeek-R1-14B
DeepSeek-R1-14B-SFT
Effective Coverage ↑
16.48%
46.83%
36.2%
23.17%
29.33%
44.65%
Efficiency ↑
0.75
2.13
1.65
1.05
1.33
2.03
Turns
22
21.98
22
22
22
22
Context Utilization ↑
23.35%
26.28%
27.13%
38.25%
26.86%
23.73%
Repetition Rate ↓
79.64%
19.33%
27.52%
66.60%
30.57%
7.55%
Invalid Question Rate ↓
90.12%
38.67%
38.52%
75.69%
45.27%
32.36%
Table 4: Performance comparison of three models before and after SFT. ↑ means higher values are better, and ↓ means lower values are better.
Figure 3: Human and LLM-as-a-Judge evaluations of 50 real and 50 synthetic doctor–patient dialogues on a 10-point Likert scale. Human scores are averaged over 3 domain experts (mean ± std), and LLM scores are from Deepseek-v2 across 3 epochs (mean ± std).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Examples of surface-level noise filtering in clinical dialogues.
Type
Scoring criteria
Score
Scoring Rule
Role Consistency
Perfect consistency, where each role behaves completely according to its expected identity and duties, displaying professional and appropriate actions throughout.
10
Flawless role adherence.
High consistency, where most dialogues align with role expectations, with only minor deviations in a few
7-9
9: No role violation, only trivial phrasing imperfections may occur.
instances.
8: 1–2 minor out-of-character instances (e.g., patient repeats medical terms).
7: ≥ 3 minor or 1 moderate deviation (e.g., patient self-diagnosis).
Minor inconsistencies, where most dialogues are role-appropriate, but occasional deviations occur.
4-6
6: Clear role overstepping but primary roles remain identifiable.
5: Frequent role overstepping ( > 3 moderate instances).
Appendix
Table 5: Specific scoring criteria for GLM-4-9B. We assess the quality of the synthetic dialogues in terms of role consistency, process consistency, contextual coherence, and basic text quality, then score them according to different criteria, and finally average them.
Metric
Real (256)
Syn (256)
Syn (1024)
Syn (2048)
Syn (4096)
Syn (5120)
Structure
Turns/Dialog
34.26 ± 12.30
42.18 ± 4.26
42.50 ± 4.40
42.10 ± 4.53
42.16 ± 4.52
42.14 ± 4.52
Sent Len
33.53 ± 74.29
56.41 ± 63.78
57.82 ± 64.68
58.15 ± 65.33
57.74 ± 65.33
57.66 ± 65.20
Unique Chars
1413
1514
1861
2014
2164
2211
Tokens/Turn
23.64
36.13
37.11
37.27
37.03
36.98
Semantic
Appendix
Table 6: Comparison of augmented profile-based dialogue generation quality in terms of structure, semantics, and diversity, with random sampling from augmented patient profiles, selecting one dialogue per profile. “Syn” means synthetic, “Sent Len” means sentence length. “Pairwise Sim” means pairwise dialogue similarity. “Paired Sim” means paired dialogue similarity. “Local Coh” means local coherence.
Figure 5: Cohen’s kappa inter-rater agreement across four evaluation dimensions for real and synthetic doctor–patient dialogues, comparing humans and human–LLM consistency.
Figure 6: Qualitative comparison of dialogues generated by baselines. Deep red indicates relevance, light red indicates weak relevance, and a yellow background indicates an issue.
Physician-physician discussions of patient cases represent a rich source of clinical knowledge and reasoning that could feed AI agents to enrich and even participate in subsequent interactions. However, privacy regulations and ethical considerations severely restrict access to such data. While synthetic data generation using Large Language Models offers a promising alternative, existing approaches primarily focus on patient-physician interactions or structured medical records, leaving a significant gap in physician-to-physician communication synthesis. We present SynDocDis, a novel framework that combines structured prompting techniques with privacy-preserving de-identified case metadata to generate clinically accurate physician-to-physician dialogues. Evaluation by five practicing physicians in nine oncology and hepatology scenarios demonstrated exceptional communication effectiveness (mean 4.4/5) and strong medical content quality (mean 4.1/5), with substantial interrater reliability (kappa = 0.70, 95% CI: 0.67-0.73). The framework achieved 91% clinical relevance ratings while maintaining doctors' and patients' privacy. These results place SynDocDis as a promising framework for advancing medical AI research ethically and responsibly through privacy-compliant synthetic physician dialogue generation with direct applications in medical education and clinical decision support.
Beny Rubinstein, Sergio Matos
University of Aveiro, Portugal · IEETA, DETI, LASI, University of Aveiro, Portugal
Conversational artificial intelligence has the potential to assist users in preliminary medical consultations, particularly in settings where access to healthcare professionals is limited. However, many existing medical dialogue systems operate in a single-turn question--answering paradigm or rely on template-based datasets, limiting conversational realism and multilingual applicability. In this work, we introduce MedAidDialog, a multilingual multi-turn medical dialogue dataset designed to simulate realistic physician--patient consultations. The dataset extends the MDDial corpus by generating synthetic consultations using large language models and further expands them into a parallel multilingual corpus covering seven languages: English, Hindi, Telugu, Tamil, Bengali, Marathi, and Arabic. Building on this dataset, we develop MedAidLM, a conversational medical model trained using parameter-efficient fine-tuning on quantized small language models, enabling deployment without high-end computational infrastructure. Our framework additionally incorporates optional patient pre-context information (e.g., age, gender, allergies) to personalize the consultation process. Experimental results demonstrate that the proposed system can effectively perform symptom elicitation through multi-turn dialogue and generate diagnostic recommendations. We further conduct medical expert evaluation to assess the plausibility and coherence of the generated consultations.
University of Birmingham, Dubai, United Arab Emirates · Heritage Institute of Technology, Kolkata, India · Madan Mohan Malaviya University of Technology, India
Medical conversations offer insights into clinical communication often absent from Electronic Health Records. However, developing reliable clinical Natural Language Processing (NLP) models is hampered by the scarcity of domain-specific datasets, as clinical data are typically inaccessible due to privacy and ethical constraints. To address these challenges, we present a pipeline for generating synthetic Dutch medical dialogues using a Dutch fine-tuned Large Language Model, with real medical conversations serving as linguistic and structural reference. The generated dialogues were evaluated through quantitative metrics and qualitative review by native speakers and medical practitioners. Quantitative analysis revealed strong lexical variety and overly regular turn-taking, suggesting scripted rather than natural conversation flow. Qualitative review produced slightly below-average scores, with raters noting issues in domain specificity and natural expression. The limited correlation between quantitative and qualitative results highlights that numerical metrics alone cannot fully capture linguistic quality. Our findings demonstrate that generating synthetic Dutch medical dialogues is feasible but requires domain knowledge and carefully structured prompting to balance naturalness and structure in conversation. This work provides a foundation for expanding Dutch clinical NLP resources through ethically generated synthetic data.
Cecilia Kuan, Aditya Kamlesh Parikh, Henk van den Heuvel
Center for Language Studies, Radboud University, Nijmegen, Netherlands