SynDocDis: A Metadata-Driven Framework for Generating Synthetic Physician Discussions Using Large Language Models
Organizations: University of Aveiro, Portugal · IEETA, DETI, LASI, University of Aveiro, Portugal
Abstract
Physician-physician discussions of patient cases represent a rich source of clinical knowledge and reasoning that could feed AI agents to enrich and even participate in subsequent interactions. However, privacy regulations and ethical considerations severely restrict access to such data. While synthetic data generation using Large Language Models offers a promising alternative, existing approaches primarily focus on patient-physician interactions or structured medical records, leaving a significant gap in physician-to-physician communication synthesis. We present SynDocDis, a novel framework that combines structured prompting techniques with privacy-preserving de-identified case metadata to generate clinically accurate physician-to-physician dialogues. Evaluation by five practicing physicians in nine oncology and hepatology scenarios demonstrated exceptional communication effectiveness (mean 4.4/5) and strong medical content quality (mean 4.1/5), with substantial interrater reliability (kappa = 0.70, 95% CI: 0.67-0.73). The framework achieved 91% clinical relevance ratings while maintaining doctors' and patients' privacy. These results place SynDocDis as a promising framework for advancing medical AI research ethically and responsibly through privacy-compliant synthetic physician dialogue generation with direct applications in medical education and clinical decision support.
Figures & tables
| Metadata | Example |
|---|---|
| Chat Name | Pancreas ATM |
| Participant Doctors | 52, 35, 6, 65, 46 |
| Specialty of Participants | Head of Department in a large hospital; Head of Service in a peripheral hospital; Head of Unit in a large hospital; Head of Unit in a large hospital; Head of Unit in a large hospital. |
| Patient Case | Male, age 69, PS1, Aug 2023 diagnosed with pancreatic adenocarcinoma, one lesion in pancreatic head and another in tail. Underwent total pancreatectomy, pathologic stage T2mN0M0, now is on adjuvant FOLFIRINOX. Genetic testing shows germinal mutation ATM C.103C>T heterozygote. The question is whether to give him maintenance PARP inhibitor. |
| Number of responses | 4 |
| Are answers valuable? | Valuable. Standard of care |
| Category | Description | |
|---|---|---|
| Medical Content Quality | 1.1 Clinical Accuracy | Assess whether the information shared is medically correct and up to date. |
| 1.2 Evidence-Based | Evaluate if physicians are using evidence-based medicine principles in their discussion. | |
| 1.3 Relevance | Is the discussion directly applicable to the patient case at hand? | |
| 1.4 Comprehensiveness | Check if all relevant aspects of the patient’s condition are being addressed. | |
| Communication Effectiveness | 2.1 Clarity & Coherence | Assess whether the discussion is clear, well-structured, and easy to follow. |
| 2.2. Medical Terminology | Evaluate appropriate use of medical terms and explanations when necessary. |
Synthetic Data Generation
Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.
See all 623 papers