Text-Only LLM Correctors Break Correct Transcripts: A Repair-Damage Analysis of Audio Grounding in ASR Error Correction

Vasiliy Kudryavtsev; Kirill Borodin; Nikita Vasiliev; Mikhail Gorodnichev; Grach Mkrtchian

cs.SDOct 21, 2026

Text-Only LLM Correctors Break Correct Transcripts: A Repair-Damage Analysis of Audio Grounding in ASR Error Correction

Authors: Vasiliy Kudryavtsev, Kirill Borodin, Nikita Vasiliev, Mikhail Gorodnichev, Grach Mkrtchian

Organizations: BitmanagerAI, Dubai, UAE · MTUCI, Moscow, Russia · lab260, Yerevan, Armenia

Abstract

LLM correctors of ASR output are judged by aggregate WER, which hides what they fix and what they break. We decompose every WER change exactly into repaired tokens, damaged tokens and insertions, and use this to ask what the source audio adds to a corrector. We compare correction with and without the audio, for the model's own draft and for the output of an independent recogniser, on three corpora. With Gemini 2.5 Flash, text-only correction of its own draft breaks 13-32% of already-correct transcripts; with the audio, 1-3%. Damage falls in both settings. Repair falls against the own draft and rises against the recogniser. On medical dictation, kept insertions cancel the gain in net WER. The open-weight Voxtral, which rarely edits, does not show this pattern. Prompts differ between the two arms, so the contrasts bound the effect of the audio and do not isolate it.

Figures & tables

input systems (WER, %)
	TTS RU	RuLibri RU	Medical EN
	$n{=}2{,}260$	$n{=}270$	$n{=}284$
ASR output (recogniser)	3.95	7.29	16.06
Draft (audio LLM)	2.39	10.04	17.65
control, against the draft ( $\Delta$ WER, pt)
B audio, no hypothesis	$+$ 0.01	$+$ 0.18	$+$ 2.53

Table 1: Gemini 2.5 Flash. Top: WER (%) of the two input systems. Below:

\Delta\mathrm{WER}

(pt) of each cell against the system that supplied its input; positive is an improvement. Bold: 95% interval excludes zero.

self-correction: A (audio) vs. C (no audio)
	rep	dam	ins	net	break-rate
TTS RU	$-$ 0.15	$-$ 2.83	$+$ 0.24	$+$ 2.92	31.9 $\to$ 2.6
RuLibri RU	$-$ 0.50	$-$ 0.88	$+$ 0.08	$+$ 0.45	16.3 $\to$ 1.0
Medical EN	$-$ 0.74	$-$ 1.72	$-$ 1.08	$-$ 0.09	12.8 $\to$ 2.3
cross-system: F (audio) vs. T (no audio)
TTS RU	$+$ 0.09	$-$ 1.31	$+$ 0.08	$+$ 1.48	18.1 $\to$ 2.6

Table 2: Effect of the audio, Gemini 2.5 Flash: grounded minus text-only cell, pt; net

=

rep

-

dam

+

ins. Negative dam: less damage; positive ins: fewer insertions left. Bold: 95% interval excludes zero. Break-rates (%, text-only

\to

grounded) have no intervals.

#	Reference	Input hypothesis	Output without audio	Output with audio
1	loss of hair	loss of hair	loss of hearing	loss of hair
2	postoperative imaging …	post operative imaging …	postoperative imaging …	post operative imaging …
3	forxiga 10mg tablet	forxa 10 mg tablet	forxiga 10 mg tablet	forxa 10 mg tablet
4	seizure	see you soon	see you soon	seizure
5	oxiconazole	oxyconosol	oxycontin	oxiconazole

Table 3: Illustrative examples from the medical corpus (Gemini 2.5 Flash, normalized text); italics mark tokens changed by the corrector. Rows 1–3: self-correction, the input is the draft; row 2 shows the first tokens of a 32-token utterance. Rows 4–5: cross-system, the input is the Whisper output.

	ASR	T	F	draft	C	A	B
WER	3.79	3.26	1.74	2.37	5.12	2.18	2.75
CER	0.79	1.23	0.42	0.67	2.57	0.67	1.18
DTR	65.5	67.5	69.7	69.0	66.1	69.3	69.1
WER, original	–	–	–	–	5.03	2.19	2.76
WER, 15 dB	–	–	–	–	6.16	3.20	4.05
WER, 8 dB	–	–	–	–	8.91	6.02	7.45

Table 4: Synthetic corpus, Gemini 2.5 Flash, raw rates in % (WER and CER: lower is better; not

\Delta

WER). Top: each system over its own complete outputs (

n=2{,}262

–

3{,}000

); columns cover different utterance sets and are not paired. Bottom: WER on 250 clips, original and with reverberation and noise. No intervals are available.

input systems (WER, %)
	TTS RU	RuLibri RU	Medical EN
	$n{=}467$	$n{=}96$	$n{=}96$
ASR output (recogniser)	5.17	8.02	17.03
Draft (audio LLM)	7.90	26.31	43.86
cells ( $\Delta$ WER, pt)
B audio, no hypothesis	$-$ 1.61	$+$ 6.09	$-$ 21.15

Table 5: Voxtral-Small-24B on subsets of the same clips; layout and units as in Tables 1 and 2 . The ins rows are obtained by subtraction and have no intervals. Bold: 95% interval excludes zero.

LLM Self-Correction

Large language model self-correction studies how language models detect and repair errors in their own answers, reasoning, and actions.

See all 159 papers

Select or Rewrite: Oracle-Guided Fine-Tuning for Generative ASR Post-Editing from N-Best Hypotheses
Oct 10, 2026Shiran Liu, Gaohui Xu, Sergio Burdisso +3Language Model Post-Training Fine-Tuning
Idiap Research Institute, Martigny, Switzerland · KTH Royal Institute of Technology, Stockholm, Sweden · Brno University of Technology, Brno, Czech Republic
Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian
May 19, 2026Yun Hao, Reihaneh Amooie, Wietse de Vries +2Automatic Speech Recognition Data Contamination in Language Models
University of Groningen, The Netherlands
Measuring Local Acoustic Contribution in Speech LLMs via Word-Span Counterfactual Interventions
Oct 21, 2026Hardik B. Sailor, Qiongqiong Wang, Dheemant Rastogi +2Audio-Language Model Evaluation Speech Language Models
†Institute of Advanced Intelligence and Computing, Agency for Science, Technology and Research, Singapore · ⋆Nanyang Technological University, Singapore

Oct 10, 2026J/K move · Enter open · S save

Select or Rewrite: Oracle-Guided Fine-Tuning for Generative ASR Post-Editing from N-Best Hypotheses

Shiran Liu, Gaohui Xu, Sergio Burdisso, Thibault Bañeras-Roux+2

Idiap Research Institute, Martigny, Switzerland · KTH Royal Institute of Technology, Stockholm, Sweden · Brno University of Technology, Brno, Czech Republic

Language Model Post-Training Fine-Tuning Automatic Speech Recognition

Open paper AskPDF arXiv

Large language models can post-edit ASR by using linguistic context to compare N-best hypotheses and rewrite the transcript when needed, often improving WER without retraining the recognizer. However, prompt-only correction is brittle: it may over-edit when top-1 is correct and hallucinate when the reference is absent, and often needs very large models. We propose oracle-guided fine-tuning that turns N-best lists into supervision via three curated pools: corrections (reference in the list but not top-1) to teach selection, regularizers (top-1 correct) to discourage unnecessary rewrites, and hard negatives (reference absent) to suppress hallucination. Fine-tuning on this data provides more effective supervision than random subset fine-tuning for learning a hybrid select-or-rewrite policy, with the clearest gains from around 3B parameters on the proprietary dataset but less conclusive results on CommonVoice, a divergence suggesting that public read-speech text may overlap with LLM pretraining and mask genuine post-editing gains.

Abstract

LLM Self-Correction

Explore similar work

Select or Rewrite: Oracle-Guided Fine-Tuning for Generative ASR Post-Editing from N-Best Hypotheses

Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian

Measuring Local Acoustic Contribution in Speech LLMs via Word-Span Counterfactual Interventions