Transcribing Children's Speech: ASR Performance and Obtaining Reliable Orthographic Transcriptions

Gus Lathouwers; Lingyun Gao; Catia Cucchiarini; Helmer Strik

cs.CLApr 10, 2026

Transcribing Children's Speech: ASR Performance and Obtaining Reliable Orthographic Transcriptions

Authors: Gus Lathouwers, Lingyun Gao, Catia Cucchiarini, Helmer Strik

Organizations: Radboud University, the Netherlands

Abstract

Automatic speech recognition (ASR) has the potential to substantially reduce manual annotation effort in child speech research by generating automatic transcriptions. However, obtaining reliably high-quality ASR transcriptions for child speech remains challenging in low-resource languages due to limited child-specific pre-trained models and highly diverse noise conditions. This study investigates the effectiveness of state-of-the-art ASR models on child speech through two research questions, by evaluating nine ASR models from three model families (Whisper, Parakeet, and Wav2Vec2) on two Dutch child speech datasets, JASMIN and DART. Research question 1 examines the performance of ASR-models applied to child speech. The fine-tuned Whisper-medium model achieves the best overall performance, with a WER of 5.54% on JASMIN and 70.37% on DART, showing that the noisy DART data are clearly more challenging. Research question 2 examines to what extent it is possible to select a subset for which reliable orthographic transcriptions can be obtained automatically, without the need for manual verification. We use an utterance-level selection method that compares ASR output with the original read prompt to identify correctly pronounced recordings. Using the proposed selection method, 42.0% [for JASMIN] and 18.1% [for DART] of the utterances can be automatically identified as correctly pronounced with high confidence, resulting in very low error rates on an utterance level (precisions of 98.3% and higher) and reducing the need for manual verification.

Figures & tables

Property	JASMIN	DART	DART (subset)
Total speech files (n)	10,642	2,343	1,049
Testing or Evaluation set	Train / Evaluate	Evaluate	Evaluate
Total Duration	9 hours 51 minutes	2 hours 54 minutes	0 hours 44 minutes
Sentences / Words ratio (%)	90.3 / 9.7	12.4 / 87.6	8.5 / 91.5
Reading Mistakes Manual (%)	53.5	45.4-53.9	19.8

Table 1: Overview of the datasets used in this study and their key properties. Reading Mistakes Manual (RMM) describes the total amount of utterances in the full datasets in which the children made at least one error pronouncing a prompt.

Model Name	Base Model	Params (M)	Finetuned
whisper-large	openai/whisper-large-v2	1550	-
whisper-large-prompted	openai/whisper-large-v2	1550	-
whisper-small-FT	openai/whisper-small	244	v
whisper-medium-FT	openai/whisper-medium	769	v
whisper-large-FT	openai/whisper-large-v2	1550	v
Wav2Vec2-gronlp	GroNLP/Wav2Vec2-dutch-large	317	-

Table 2: Overview of ASR models used in the experiments, including base model, number of parameters, and whether the model was custom finetuned on Dutch child speech in the current study.

PR	MO	AO	RMM	RMA	Outcome
zij heeft een klacht	zij heeft een klant	we heeft een klans	0	0	TN
duikplank	duikplank	duikplank	1	1	TP
schrapen	schrappen	schrapen	0	1	FP
hoge bomen	hoge bomen	handgebaren	1	0	FN

Table 3: Example comparisons for true positive, true negative, false positive, false negative for research question 2. PR refers to the original prompt the child read, which are included in the JASMIN and DART corpora. MO refers to the manual orthographic transcription, also included in the two corpora used. AO refers to the ASR-prediction generated by a model. RMA (Reading Mistakes Automatic) represents whether AO matches PR in its entirety, if so denoted with a 1. RMM (Reading Mistakes Manual) represents whether MO matches PR in its entirety.

Panel of Figure 1 — Figure 1: WER and UER of all models for JASMIN (top graph) and DART (bottom graph). The horizontal dotted line represents the baseline measure (a non-modified version of whisper-large-v2).

	JASMIN (n=1550)			DART (n=1047)
Model	WER	CER	UER	WER	CER	UER
whisper-large (baseline)	16.94	9.34	58.03	77.57	45.99	76.41
whisper-large-prompted	15.65	7.11	56.61	73.37	41.29	73.45
whisper-small-FT	8.94	3.33	37.72	75.62	35.44	70.01
whisper-medium-FT	5.54	2.08	27.14	70.37	33.27	63.13
whisper-large-FT	5.46	2.03	27.08	78.40	38.33	67.72

Table 4: ASR performance on the JASMIN and DART datasets measured in WER, CER, and UER. Bolded values represent the best performance for a metric/dataset combination. The -FT suffix denotes that the model has been finetuned on child audio.

	JASMIN (n=1550)				DART (n=1047)				DART (n=2343)
Model	P	R	F1	MCC	P	R	F1	MCC	P	R	F1	MCC
whisper-large	80.1	59.5	68.3	0.49	99.0	23.8	38.3	0.23	94.6	19.0	31.7	0.28
whisper-prompted	89.2	59.7	71.7	0.59	99.6	27.3	42.9	0.25	96.2	21.6	35.2	0.31
whisper-small-FT	97.9	77.8	86.7	0.79	99.6	29.6	45.6	0.27	95.4	25.6	40.3	0.32
whisper-medium-FT	98.3	88.9	93.4	0.88	100.0	36.6	53.6	0.32	99.3	32.9	49.4	0.42
whisper-large-FT	96.8	89.5	93.0	0.88	99.3	31.9	48.3	0.28	97.8	27.2	42.6	0.37

Table 5: Precision (P), Recall (R), F1-score (F1), and Matthews Correlation Coefficient (MCC) for utterance selection across datasets. Best performance for the individual models is displayed in bold (top 9 rows). Conditions in which output of the three finetuned models is combined (whisper-medium-FT, Wav2Vec2-FT, and parkeet-FT) are displayed in the bottom two rows, with the best value for combined strategies also bolded. † For DART n=1047, comb-and-condition agreement is a case in which at least two out of three models of [whisper-medium-FT, Wav2Vec2-FT, parakeet-FT] match the PR. For DART n=2343 and JASMIN n=1550, comb-and-condition are cases in which all three match the PR.

Error	MO $\rightarrow$ AO	Description	Frequency
S	hij $\rightarrow$ haar	AO transcribes ”haar” instead of ’hij’ for 3 cases in which child hesitated during pronouncing ’hij’.	3
S	een $\rightarrow$ hun	There are multiple case in which ”hun” is transcibed by ASR as ”een” and vice versa.	3
S	d’r $\rightarrow$ haar	”d’r” is a short-form common contraction of ”haar”, in multiple sentences instances of ”er”, ”d’r”, and ”haar” produce errors between MO and AO.	2
I	(none) $\rightarrow$ de	Insertion of article ”de”, in two instances the MO forgot to transcribe these.	4
I	(none) $\rightarrow$ een	Insertion of ”een”, due to hallucination (2 times) or MO error (1 time).	3
I	(none) $\rightarrow$ naar	Insertion of ’naar’ in cases in which child spoke very rapidly.	2

Table 6: Overview of common errors made for the JASMIN dataset.

Error	MO $\rightarrow$ AO	Description	Frequency
S	pluimpje $\rightarrow$ blijmpje	Failure to transcribe ”plui”, instead transcribing ”blij” in the word ”pluimpje”.	4
S	besproei $\rightarrow$ bestproe	Consistently transcribing regularly pronounced ’besproei’ as ’bestproe’	3
S	fraai $\rightarrow$ vrij	Three cases, of wich 1 recording has poor speech quality, one fringe case with deviant ’fraai’ pronunciation, and one regular case in which ASR-model failed to detect ’raai’.	3
I	(none) $\rightarrow$ de	Many cases in which normally pronounced word was split by ASR into two subwords (e.g., ”verhaal” transcribed by ASR-model as ”de” + ”haa”).	8
I	(none) $\rightarrow$ op	Cases in which ASR incorrectly split one spoken word into 2 subwords.	6
I	(none) $\rightarrow$ ja	A number of cases where caused by the ASR-model transcribing background noise of children yelling, or small partial pronunciations of the word ”ja”.	4

Table 7: Overview of common errors made for the DART dataset.

Reference (MO)	Hypothesis (AO)	Error	Explanation
net als ze zich willen omdraaien blijft de lichtstraal hangen bij een zoort rek	net als ze zich willen omdraaien blijft de lichtstraal hangen bij een soort rek	S	Manual annotator notated “zoort”, ASR-model transcribed “soort”.
hij w wenkt met zijn arm	hij * wenkt met zijn arm	D	ASR-model deleted extra consonant ”w” uttered before ”wenkt” in comparison with MO.
misschien zou ze eigenlijk liever onze hoofden tegen elkaar g aan slaan denkt sarah	misschien zou ze eigenlijk liever onze hoofden tegen elkaar * aan slaan denkt sarah	D	Deleted consonant ”g” inserted before ”aan slaan”.
robbie schuld zijn hoofd	robbie schudt zijn hoofd	S	“schudt” Instead of ”schuld”
even laten horen ze papa stem	even later horen ze papas stem	S	Missing genitive “’s” for ”papa”.
nu zorgt de postbode dat je brieft in de juiste bus terechtkomt	nu zorgt de postbode dat je brief in de juiste bus terechtkomt	S	“brief” Instead of “brieft”.

Table 8: False Positives in the JASMIN dataset (n=1550).

Reference (MO)	Hypothesis (AO)	Error	Explanation
braden*u	braden	S	Child pronounced the word ”braden” incorrectly (marked ’*u’ by annotator), however ASR-output transcribed ”braden”.
laaaager*u	lagen	S	Drawn out vowel ’a’ and ’n’ pronounced as ’r’ not caught by the ASR-model.
blazen*a	blazen	S	speech recording was cut-off (marked as ’*a’ by annotator), ASR-model transcribed full word.

Table 9: False Positives in the DART dataset (n=2343).

Explore similar work

Utterance-Level Methods for Identifying Reliable ASR-Output for Child Speech
Apr 10, 2026Gus Lathouwers, Lingyun Gao, Catia Cucchiarini +1Speech Processing ASR Evaluation
Centre for Language Studies, Radboud University, Netherlands · Department of Language and Communication, Radboud University, Netherlands
Enabling automatic transcription of child-centered audio recordings from real-world environments
Jun 13, 2025Daniil Kocharov, Azarias Galama, Okko RäsänenAutomatic Speech Recognition
Signal Processing Research Centre, Tampere University, Finland, Faculty of Information Technology and Communication Sciences, Department of Computing Sciences, P.O. Box 553, FI-33101 Tampere, Finland
Child ASR Adaptation with Adult Retention: An Empirical Study
Sep 25, 2026Houssam Eddine-Othman Lachemat, Shammur Absar ChowdhuryAutomatic Speech Recognition Domain Adaptation for ASR
Qatar Computing Research Institute

Apr 10, 2026J/K move · Enter open · S save

Utterance-Level Methods for Identifying Reliable ASR-Output for Child Speech

Gus Lathouwers, Lingyun Gao, Catia Cucchiarini, Helmer Strik

Centre for Language Studies, Radboud University, Netherlands · Department of Language and Communication, Radboud University, Netherlands

Speech Processing ASR Evaluation

Open paper AskPDF arXiv

Automatic Speech Recognition (ASR) is increasingly used in applications involving child speech, such as language learning and literacy acquisition. However, the effectiveness of such applications is limited by high ASR error rates. The negative effects can be mitigated by identifying in advance which ASR-outputs are reliable. This work aims to develop two novel approaches for selecting reliable ASR-output at the utterance level, one for selecting reliable read speech and one for dialogue speech material. Evaluations were done on an English and a Dutch dataset, each with a baseline and finetuned model. The results show that utterance-level selection methods for identifying reliably transcribed speech recordings have high precision for the best strategy (P > 97.4) for both read speech and dialogue material, for both languages. Using the current optimal strategy allows 21.0% to 55.9% of dialogue/read speech datasets to be automatically selected with low (UER of < 2.6) error rates.

Abstract

Explore similar work

Utterance-Level Methods for Identifying Reliable ASR-Output for Child Speech

Enabling automatic transcription of child-centered audio recordings from real-world environments

Child ASR Adaptation with Adult Retention: An Empirical Study