Huntington Disease Automatic Speech Recognition with Biomarker Supervision

Charles L. Wang; Cady Chen; Ziwei Gong; Julia Hirschberg

cs.LGMar 11, 2026

Huntington Disease Automatic Speech Recognition with Biomarker Supervision

Authors: Charles L. Wang, Cady Chen, Ziwei Gong, Julia Hirschberg

Organizations: Columbia University

Abstract

Automatic speech recognition (ASR) for pathological speech remains underexplored, especially for Huntington's disease (HD), where irregular timing, unstable phonation, and articulatory distortion challenge current models. We present a systematic HD-ASR study using a high-fidelity clinical speech corpus not previously used for end-to-end ASR training. We compare multiple ASR families under a unified evaluation, analyzing WER as well as substitution, deletion, and insertion patterns. HD speech induces architecture-specific error regimes, with Parakeet-TDT outperforming encoder-decoder and CTC baselines. HD-specific adaptation reduces WER from 6.99% to 4.95% and we also propose a method for using biomarker-based auxiliary supervision and analyze how error behavior is reshaped in severity-dependent ways rather than uniformly improving WER. We open-source all code and models.

Figures & tables

Figure 1: Overview of our adapter-based HD-ASR framework. A frozen Parakeet-TDT encoder is augmented with trainable PEFT adapters. Standard ASR supervision is retained, while a masked mean-pooled encoder representation is used to predict one auxiliary biomarker family per run (prosody, phonation, or articulation).

Model	Architecture Family	Parameters	WER (%)
Whisper-small	Encoder-decoder	244M	19.78
Whisper-medium	Encoder-decoder	769M	26.91
Whisper-large-v2	Encoder-decoder	1.55B	18.44
Parakeet-TDT 0.6B	TDT / Transducer	600M	6.99
Meta Omnilingual 300M	CTC	300M	30.46

Table 1: Zero-shot ASR performance across model families on the HD test set. Results are reported on 47 scored test utterances.

Model	Substitution Share (%)	Deletion Share (%)	Insertion Share (%)
Whisper-small	17.99	9.35	72.66
Whisper-medium	13.29	6.68	80.04
Whisper-large-v2	16.56	9.18	74.27
Parakeet-TDT 0.6B	41.90	29.68	28.43
Meta Omnilingual 300M	43.30	22.85	33.85

Table 2: Error composition across zero-shot ASR baselines on HD speech. Values are reported as percentages of each model’s total errors

(S+D+I)

Model	WER (%)	Substitutions (%)	Deletions (%)	Insertions (%)
Parakeet-HD	4.95	2.09	1.29	1.57
Parakeet-HD-Prosody	6.11	2.13	2.53	1.45
Parakeet-HD-Phonation	6.07	1.92	2.77	1.38
Parakeet-HD-Articulation	6.44	1.94	3.21	1.29

Table 3: ASR performance for biomarker-auxiliary Parakeet variants.

Model	Stage	$\Delta$ WER	$\Delta$ Sub.	$\Delta$ Del.	$\Delta$ Ins.
Parakeet-HD-Prosody	Control	-0.26	-0.05	+0.16	-0.37
Parakeet-HD-Prosody	Pre-HD	-0.22	-0.11	+0.00	-0.11
Parakeet-HD-Prosody	Prodromal	-0.24	+0.00	-0.24	+0.00
Parakeet-HD-Prosody	Manifest	+3.59	+0.19	+3.35	+0.05
Parakeet-HD-Phonation	Control	-0.05	-0.16	+0.37	-0.26
Parakeet-HD-Phonation	Pre-HD	-0.11	+0.00	+0.00	-0.11

Table 4: Cohort-level change relative to Parakeet-HD for biomarker-auxiliary variants. Values are percentage-point differences; negative is better. Best

\Delta

WER in each cohort is bolded.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems
Jun 22, 2026Mateusz Barański, Jan Jasiński, Julitta Bartolewska +2ASR Evaluation Hallucination Detection
Signal Processing Group, Institute of Electronics, AGH University of Krakow, Poland
S-DiverSe: Spanish Diverse Speech
Jul 3, 2026Fernando López, Fernando Ibañez, Ana Martínez +4ASR Evaluation Speech Processing
Scientific Research, Telefónica Innovación Digital, Spain · Universidad Autónoma de Madrid, Spain · Brno University of Technology, Czech Republic
Adapting Foundation ASR Models to Dysarthric Speech: A Case Study
Jun 30, 2026Christian Huber, Laura Kernahan, Alexander WaibelPersonalized Speech Recognition Automatic Speech Recognition
Interactive Systems Lab Karlsruhe Institute of Technology Karlsruhe, Germany · Interactive Systems Lab Carnegie Mellon University Pittsburgh PA, USA

Jun 22, 2026J/K move · Enter open · S save

HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems

Mateusz Barański, Jan Jasiński, Julitta Bartolewska, Marcin Witkowski+1

Signal Processing Group, Institute of Electronics, AGH University of Krakow, Poland

ASR Evaluation Hallucination Detection Automatic Speech Recognition

Open paper AskPDF arXiv

End-to-end Automatic Speech Recognition (ASR) systems hallucinate on natural speech, yet existing mitigation methods are typically evaluated on non-speech or artificially corrupted audio. We introduce HALAS, the first human-annotated dataset of naturally occurring hallucinations from seven state-of-the-art ASR models on real unprocessed earnings call recordings. HALAS provides span-level labels, enabling analysis of hallucination patterns and their severity. Our analysis reveals strong cross-model vocabulary overlap and confirms that hallucinations also occur for almost correctly transcribed speech (characterized by a low Word Error Rate). The proposed benchmark with HALAS shows that the character and semantic-level metrics used as a proxy for hallucination detection reach 81% ROC-AUC, while state-of-the-art detection methods achieve an F1 score of only 53.1%. As such, HALAS establishes the first rigorous non-artificial benchmark for the detection and mitigation of ASR hallucinations.

Abstract

Automatic Speech Recognition

Explore similar work

HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems

S-DiverSe: Spanish Diverse Speech

Adapting Foundation ASR Models to Dysarthric Speech: A Case Study