Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization

Yanis Labrak; David Grünert; Séverin Baroudi; Jiyun Chun; Pawel Cyrta; Sergio Burdisso; Ahmed Hassoon; David Liu

cs.SDApr 7, 2026

Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization

Authors: Yanis Labrak, David Grünert, Séverin Baroudi, Jiyun Chun, Pawel Cyrta, Sergio Burdisso, Ahmed Hassoon, David Liu,

Organizations: Idiap Research Institute, Switzerland · University of Zurich, Switzerland · Université de Toulon, Aix Marseille Univ, LIS, CNRS, France · The Ohio State University, USA ·

Abstract

Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant to long-context reasoning pose well-known challenges for automatic evaluation. We propose a synthetic data generation pipeline designed to serve both as a training resource and as a controlled evaluation environment, and instantiate it for first-visit doctor-patient conversations with SOAP note generation as the task. The pipeline has three stages, persona-driven dialogue generation, multi-speaker audio synthesis with overlap/pause modeling, room acoustics, and sound events, and LLM-based reference SOAP note production, built entirely on open-weight models. We release 8,800 synthetic conversations with 1.3k hours of corresponding audio and reference notes. Evaluating current open-weight systems, we find that cascaded approaches still substantially outperform end-to-end models.

Figures & tables

Dataset	Doctor		Patient		Num Turns
Dataset	Turn Length	Fog Index	Turn Length	Fog Index	Num Turns
Ours	$49.9\pm 16.0$	$10.4$	$56.0\pm 15.2$	$6.9$	$28.4\pm 7.5$
PriMock57	$18.8\pm 4.7$	$6.6$	$12.3\pm 5.8$	$5.5$	$97.3\pm 17.7$
Mocks	$29.6\pm 6.3$	$7.0$	$13.6\pm 3.1$	$7.0$	$54\pm 6.2$

Table 1: Dialogue statistics comparing our generated synthetic conversations against baseline and reference datasets.

Figure 1: Overview of the audio generation pipeline, integrating Text-To-Speech, sound event generation, temporal scene composition, and environmental acoustic simulation.

	Train	Dev	Test	Overall
Personas (Doc/Pat)	60/120	20/20	20/60	100/200
Dialogues	7,200	400	1,200	8,800
Hours	1,087	59	183	1,329
Words in dialogues	10.9M	582K	1.8M	13.3M
Turns/dialogue	28.4	28.7	28.0	28.4
Duration/dialogue (s)	544	530	548	544

Table 2: Synth-DoPaCo dataset statistics.

Architecture		Faith.	Cov.	Struct.	Conc.
Qwen3	-ASR+Thinking	3.1 (±0.7)	3.9 (±0.9)	4.0 (±1.0)	3.4 (±0.8)
	-Omni-Instruct	2.7 (±0.8)	4.3 (±0.8)	4.3 (±0.9)	3.0 (±0.7)
	-Omni-Thinking	2.7 (±0.8)	4.2 (±0.8)	4.0 (±0.9)	3.1 (±0.7)
Whisper Large V3+		3.3 (±0.8)	4.3 (±0.8)	4.0 (±0.9)	3.5 (±0.7)
Qwen3-Thinking
Reference (Oracle		5.0 (±0.2)	4.2 (±0.9)	4.5 (±0.9)	3.8 (±0.8)

Table 3: Comparison of different note generation methods. Metrics: (Faith)fulness, (Cov)erage, (Struct)ure, and (Conc)iseness

Model	R-2	R-3	R-L	Open	#Wrd
Whisper+Qwen3	$11.8$ $\pm 0.3$	$4.16$ $\pm 0.20$	$22.7$ $\pm 0.4$	$29.1$ $\pm 0.5$	255
Qwen3-ASR+Qwen3	$11.0$ $\pm 0.3$	$3.84$ $\pm 0.22$	$21.9$ $\pm 0.5$	$27.9$ $\pm 0.6$	254
Qwen3-Omni-Thinking	$12.7$ $\pm 0.3$	$5.03$ $\pm 0.21$	$21.3$ $\pm 0.4$	$27.5$ $\pm 0.5$	487
Qwen3-Omni-Instruct	$14.3$ $\pm 0.4$	$6.24$ $\pm 0.27$	$22.9$ $\pm 0.4$	$30.9$ $\pm 0.6$	545

Table 4: Dev-set SOAP-note F1 (%);

\pm

is the bootstrapped 95% CI half-width.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 609 papers

Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 2026
Aug 25, 2026Enes Yavuz Ugan, Fabian Retkowski, Yuka Ko +4Medical Report Generation Text Summarization
ISL, Karlsruhe Institute of Technology · AI4LT, Karlsruhe Institute of Technology · InterACT, Carnegie Mellon University
Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge
Jul 19, 2026Aivo Olev, Tanel AlumäeLLM Hallucination Mitigation Text Summarization
Tallinn University of Technology Tallinn, Estonia
Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation
Sep 13, 2026Ziyu Zhang, Mingchen Shao, Wenjie Tian +3Medical Report Generation Clinical Speech Processing

Aug 25, 2026J/K move · Enter open · S save

Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 2026

Enes Yavuz Ugan, Fabian Retkowski, Yuka Ko, Thai-Binh Nguyen+3

ISL, Karlsruhe Institute of Technology · AI4LT, Karlsruhe Institute of Technology · InterACT, Carnegie Mellon University

Medical Report Generation Text Summarization

Open paper AskPDF arXiv

With the advent of Large Language Models and its instruction following capabilities a promising application is the task of summarization. Within this domain of task the extractive sub-task of clinical protocolling has emerged as a topic of particular interest as it can significantly reduce the downtime and protocolling burden of health-care workers thus enabling them to focus on their core work helping humans. A further step towards automation is the direct generation of clinical notes from speech without intermediate transcripts, reducing processing time while preserving information such as coughing or other paralinguistic cues that may be lost in transcript-based systems. To this end, we present KIT's submission to this years BeTraC challenge in the lightweight track. Our main contribution is a scalable data augmentation pipeline that unifies heterogeneous medical dialogue datasets through synthetic speech generation and automatically generated SOAP supervision, enabling robust adaptation of a speech foundation model for end-to-end speech-to-SOAP generation.

Abstract

Synthetic Data Generation

Explore similar work

Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 2026

Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge

Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation