Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarization

Ambre Marie; Thomas Bertin; Guillaume Dardenne; Gwenolé Quellec

cs.CLFeb 16, 2026

Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarization

Authors: Ambre Marie, Thomas Bertin, Guillaume Dardenne, Gwenolé Quellec

Organizations: LaTIM · LaTIM UMR 1101 INSERM, Brest, France · University of Western Brittany, Brest, France · DySoLab ·

Abstract

Automatic speech recognition for French medical conversations remains challenging, with word error rates often exceeding 30% in spontaneous clinical speech. This study proposes a multi-pass LLM post-processing architecture alternating between Speaker Recognition and Word Recognition passes to improve transcription accuracy and speaker attribution. Ablation studies on two French clinical datasets (suicide prevention telephone counseling and preoperative awake neurosurgery consultations) investigate four design choices: model selection, prompting strategy, pass ordering, and iteration depth. Using Qwen3-Next-80B, Wilcoxon signed-rank tests confirm significant WDER reductions on suicide prevention conversations (p<0.05, n=18), while maintaining stability on awake neurosurgery consultations (n=10), with zero output failures and acceptable computational cost (RTF 0.32), suggesting feasibility for offline clinical deployment, pending validation on larger corpora.

Figures & tables

Dataset size
	AN	SP
Total duration	5h49m	6h32m
Total words	57,668	66,144
Conversations	10	18
Unique speakers (corpus)	15	13
Conversational structure

Table 1: Dataset statistics

Model	WER	DER	WDER	WER	DER	WDER	Avg	Outliers
	AN			SP
Baseline	32.67	24.96	43.03	30.03	28.28	42.45	42.74	0
GPT4omini	32.62	24.96	42.93	30.04	28.21	42.44	42.69	0
QwenVL	34.68	26.02	45.46	30.43	33.17	42.55	44.01	3
Qwen80B	32.66	24.96	43.02	30.03	28.19	42.44	42.73	0

Table 2: Model comparison in single-pass configuration.

Strategy	WER	DER	WDER	WER	DER	WDER	Avg	Outliers
	AN			SP
Baseline	32.67	24.96	43.03	30.03	28.28	42.45	42.74	0
Zero-shot	32.68	27.29	43.76	29.99	26.89	39.96	41.86	0
Few-shot	32.40	25.67	42.12	30.03	26.32	40.44	41.28	1

Table 3: Zero-shot vs Few-shot prompting with Qwen80B.

Strategy	N	WER	DER	WDER	WER	DER	WDER	Avg	Outliers
		AN			SP
Baseline	0	32.67	24.96	43.03	30.03	28.28	42.45	42.74	0
Joint-pass	2	32.68	24.96	43.04	30.03	28.19	42.43	42.74	0
Joint-pass	3	32.69	24.96	43.05	30.03	28.19	42.43	42.74	0
SR-led	2	32.68	27.29	43.76	29.99	26.89	39.96	41.86	0
WR-led	2	32.71	27.84	44.62	29.99	26.55	39.72	42.17	0

Table 4: SR-led vs WR-led pass ordering with Qwen80B, including joint multi-pass baseline.

N pass	WER	DER	WDER	WER	DER	WDER	Avg	RTF
	AN			SP
0	32.67	24.96	43.03	30.03	28.28	42.45	42.74	0.09
1	32.66	24.96	43.02	30.03	28.19	42.44	42.73	0.12
2	32.68	27.29	43.76	29.99	26.89	39.96	41.86	0.22
3	32.68	27.45	43.79	29.99	26.32	39.57	41.68	0.32
4	32.68	27.48	43.96	29.99	26.46	39.33	41.65	0.45

Table 5: Iteration depth analysis with Qwen80B.

	AN		SP
vs 3P-S	$\boldsymbol{\Delta}$ WDER	p-value	$\boldsymbol{\Delta}$ WDER	p-value
Baseline	-0.76	1.00	+2.88*	0.02
1P-S (GPT4omini)	-0.86	0.77	+2.87*	0.02
1P-S (QwenVL)	+0.98	0.13	+4.02*	0.02
1P-S	-0.77	1.00	+2.87*	0.02
2P-S	-0.03	0.31	+0.39	0.40

Table 6: Statistical comparison of 3P-S vs alternatives.

Figure 2: Representative examples of qualitative improvements.

Speech Processing

Speech processing develops computational methods for recognizing, synthesizing, analyzing, and transforming spoken audio.

See all 412 papers

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models
Jan 7, 2024Quan Wang, Yiling Huang, Guanlong Zhao +3Speaker Diarization
Google LLC, USA
Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset
Mar 25, 2026Mohammed Nowshad Ruhani Chowdhury, Mohammed Nowaz Rabbani Chowdhury, Sakari LukkarinenLLM Fine-Tuning Medical LLMs
School of ICT and Industrial Management, Metropolia University of Applied Sciences, Helsinki, Finland · Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge
Jul 19, 2026Aivo Olev, Tanel AlumäeLLM Hallucination Mitigation Text Summarization
Tallinn University of Technology Tallinn, Estonia

Jan 7, 2024J/K move · Enter open · S save

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

Quan Wang, Yiling Huang, Guanlong Zhao, Evan Clark+2

Google LLC, USA

Speaker Diarization

Open paper AskPDF arXiv

In this paper, we introduce DiarizationLM, a framework to leverage large language models (LLM) to post-process the outputs from a speaker diarization system. Various goals can be achieved with the proposed framework, such as improving the readability of the diarized transcript, or reducing the word diarization error rate (WDER). In this framework, the outputs of the automatic speech recognition (ASR) and speaker diarization systems are represented as a compact textual format, which is included in the prompt to an optionally finetuned LLM. The outputs of the LLM can be used as the refined diarization results with the desired enhancement. As a post-processing step, this framework can be easily applied to any off-the-shelf ASR and speaker diarization systems without retraining existing components. Our experiments show that a finetuned PaLM 2-S model can reduce the WDER by rel. 55.5% on the Fisher telephone conversation dataset, and rel. 44.9% on the Callhome English dataset.

Abstract

Speech Processing

Explore similar work

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset

Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge