Generating High Quality Synthetic Data for Dutch Medical Conversations
Organizations: Center for Language Studies, Radboud University, Nijmegen, Netherlands
Abstract
Medical conversations offer insights into clinical communication often absent from Electronic Health Records. However, developing reliable clinical Natural Language Processing (NLP) models is hampered by the scarcity of domain-specific datasets, as clinical data are typically inaccessible due to privacy and ethical constraints. To address these challenges, we present a pipeline for generating synthetic Dutch medical dialogues using a Dutch fine-tuned Large Language Model, with real medical conversations serving as linguistic and structural reference. The generated dialogues were evaluated through quantitative metrics and qualitative review by native speakers and medical practitioners. Quantitative analysis revealed strong lexical variety and overly regular turn-taking, suggesting scripted rather than natural conversation flow. Qualitative review produced slightly below-average scores, with raters noting issues in domain specificity and natural expression. The limited correlation between quantitative and qualitative results highlights that numerical metrics alone cannot fully capture linguistic quality. Our findings demonstrate that generating synthetic Dutch medical dialogues is feasible but requires domain knowledge and carefully structured prompting to balance naturalness and structure in conversation. This work provides a foundation for expanding Dutch clinical NLP resources through ethically generated synthetic data.
Figures & tables
| Metric | Mean | SD |
|---|---|---|
| Alternation rate | 0.973 | 0.021 |
| Role consistency | 0.012 | 0.007 |
| ASL | 16.18 | 4.03 |
| Average SPT | 2.14 | 0.37 |
| Topic coverage | 0.889 | 0.132 |
| TTR | 0.377 | 0.04 |
| Measure | D9 | D3 | D6 |
|---|---|---|---|
| Length | 610 | 972 | 1266 |
| MSTTR | 0.85 | 0.825 | 0.843 |
| TTR | 0.44 | 0.384 | 0.321 |
| ASL | 10.97 | 11.61 | 14.79 |
| Turns | 26 | 61 | 50 |
| SPT | 2.44 | 1.62 | 2.44 |
| Coherency | |
| Points | Definition and Examples |
| 1-2 | Frequent order/timing mistakes (non-sequential turns, confusing switches, inconsistent tense). Dialogue feels unnatural and hard to follow. Ex - doctor and patient turns are frequently mixed up, inserted after goodbye:: "Arts: Goed, laten we nu eens kijken naar wat u al doet en wat we kunnen verbeteren." |
| 3 | Mostly sequential and moderately coherent, but some abrupt or unnatural/unclear transitions between topics (e.g. sudden topic change, unclear turn boundaries). Ex - "Patiënt: Ik probeer elke twee uur een half kopje te drinken. Arts: En wat betreft uw voeding?" |
| 4-5 | Logically flows, correct turn-taking, consistent tenses, naturally progressing conversation; mostly coherent with minor jumps between topics without clear transition.. Ex - "Patiënt: Ik probeer elke twee uur een half kopje te drinken. Arts: En wat betreft uw voeding?" |
| Consistency | |
| Points | Definition and Example |
| 1-2 | Frequent factual errors, hallucinated medical info, missing or contradictory details with previous statement and context. Lacks accuracy in medical facts, treatment options, or role actions. Ex - contains hallucinated diseases/treatment - "Arts: Je zegt dat je allergisch bent en Diazepam vijf keer per week neemt." |
| 3 | Some inconsistencies, minor errors or missing pieces, e.g., partial topic coverage, minor fact mistakes, not linking blood pressure to kidney function after discussing both. Ex - "Patiënt: Ik probeer mijn waterinname te verhogen, maar soms vergeet ik." (General lifestyle is relevant, but lacks specific detail or contradicts earlier plan.) |
| 4-5 | Consistent and accurate; mostly or all facts align with scenario and medical knowledge, topics covered. |
| Fluency | |
| Points | Definition and Example |
| 1-2 | Use of Dutch is awkward or ungrammatical, regardless of content. Poor quality sentences, long sentences, repetition in sentences. Translations-like errors. Ex - "Het rug pijn is niet goed voelen en ik zijn moe altijd is van pijn." |
| 3 | Readable with occasional awkwardness, odd phrasing, or grammar slips; generally understandable. Ex - "Ja, ik ben nemen de pillen en ik voel minder, maar niet blij altijd." |
| 4-5 | Fluent, natural, and idiomatic Dutch; well-structured and appropriate medical register. Ex - "Ik neem ibuprofen en probeer rust te nemen. Maar het helpt niet echt." |
| Relevance | |
| Points | Definition and Example |
| 1-2 | Major topics missing (symptoms, medication use, lifestyle, and laboratory results.), many irrelevant details, or missing clinical actions. Ex - "Patiënt: Ik houd van voetbal en mijn huisdier is een kat." (Unrelated information to context and/or of topics being discussed.) |
| 3 | Some or all target topics touched but detail may be shallow, or topic coverage uneven; some off-topic content. Lacks full coverage. Ex - "Arts: Wat betreft beweging, misschien kunt u een beetje meer wandelen." |
| 4-5 | Every target topic is addressed with sufficient detail; all info relevant to consultation. Ex - "Arts: We kunnen naar fysiotherapie of yoga kijken. Ook uw labresultaten wijzen op een geleidelijk probleem." (Addresses symptoms, medication, lifestyle, and lab results directly.) |
| Clinical Use | |
| Points | Definition and Example |
| 1-2 | Dialogue is unrealistic for actual clinical setting (e.g., unsafe advice, fundamental errors, implausible behaviors). Ex - unsafe or implausible advice is given ex - "Arts: Neem gewoon meer pijnstillers, zoveel als u wilt, en de rest is niet belangrijk." |
| 3 | Could work in clinic with edits; mostly safe and realistic, but has notable weaknesses. Ex - "Arts: Probeer rustig te blijven en misschien wat meer water drinken." (Safe but incomplete; lacks specific clinical recommendation or next steps.) |
| 4-5 | Realistic, safe, plausible for clinical scenario; could be useful for annotation/training. Ex - Arts: We zullen uw bloeddruk monitoren en indien nodig medicatie voorschrijven. Houd een dagboek bij van uw symptomen en neem contact op als ze verergeren." |
Synthetic Data Generation for Language Models
Synthetic data generation for language models creates artificial text, tasks, and interaction traces to train or evaluate models.
See all 292 papers