FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech

Wei Chu; Yuanzhe Dong; Ke Tan; Dong Han; Yichao Zhou; Ruchao Fan; Bingshen Mu; Jingbei Li

cs.CLSep 26, 2026

FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech

Authors: Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Yichao Zhou, Ruchao Fan, Bingshen Mu, Jingbei Li,

Organizations: Olewave, USA · Stanford University · Cartesia · Microsoft ·

Abstract

Forced alignment estimates the timestamps of each word, phone or character in speech given its transcript. Published comparisons normalize transcripts, split the data and match boundaries differently, so their numbers cannot be read together. We present FA-Bench, an open framework that fixes those choices once and releases the code, splits, phone mapping, text normalization and scoring script, with results published periodically. Track 1 gives every aligner the reference transcript and Track 2 gives it a recognizer's output, on the same audio, clean and degraded four ways, with 21 open models and 9 commercial APIs under a unified protocol. We score every boundary of an utterance and check the two labels beside it, so a word the recognizer missed or invented is charged. Using a tolerance-based F1 as our primary metric eliminates the 9% to 14% score inflation that standard MAE causes on recognition-dependent systems in conversational speech. We then group boundaries by position and how many adjacent words were recognized correctly, which shows where a system lost the score. We discovered systematic bias in how current systems time words, with Whisper about 150 ms early and several commercial ASR APIs over 50 ms late. Code and results are at https://github.com/olewave/fa-bench

Figures & tables

Panel of Figure 1 — Figure 1: Mean signed word-boundary error on TIMIT test, hypothesis minus reference, over the words the label alignment matched. Track 2 only. Every two-step point aligns Qwen3-ASR’s transcript, so the recognizer is fixed and only the aligner varies. Both axes are warped by $\mathrm{asinh}(v/4)$ . The distance from the dashed line $y=x$ is the mean error in word duration.

Family		TIMIT test						Buckeye test
	System	clean			noisy			clean			noisy
	Track 1 , given the reference transcript
		MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	$\boldsymbol{F_{1}^{\scriptscriptstyle 50}}$	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	$\boldsymbol{F_{1}^{\scriptscriptstyle 50}}$	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	$\boldsymbol{F_{1}^{\scriptscriptstyle 50}}$	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	$\boldsymbol{F_{1}^{\scriptscriptstyle 50}}$
Open	NeMo-FA [ 14 ]	0 47.4	.17	0 .41	0 47.1	.18	0 .43	0 62.8	.21	0 .49	0 65.1	.21	0 .50
	stable-ts [ 29 ]	0 96.8	.18	0 .39	0 94.5	.19	0 .41	0 71.2	.29	0 .55	0 75.2	.30	0 .55
	WhisperX [ 10 ]	0 37.2	.18	0 .43	0 39.5	.17	0 .42	0 41.7	.16	0 .49	0 52.0	.17	0 .48

Table 1: Word-tier boundary error on the test split of each corpus, on clean audio and under degradation, noisy being the mean of the four. Every number here comes from our own runs. MAE is over word boundaries in ms.

F_{1}

is over every word boundary, the two utterance edges included. A boundary is a hit only when the words on each side of it match the reference and the time falls within the tolerance of the reference boundary. The superscript on

F_{1}

is that tolerance in ms. The step size of a word boundary of each system is shown in Table 3 . Open means the weights are published, and NeMo-FA, WhisperX, TorchAudio, BFA, MMS-FA, UnitY2, Charsiu, MFA and MAPS also publish the code that trained them. The NeuFA model is trained with its open-source code on GitHub and is not released. W is Whisper large-v3, P and Parakeet are Parakeet-TDT, Q and Qwen3 are Qwen3-ASR, and G is Google Chirp 2. Whisper-ts is Whisper-timestamped and NeMo-FA uses its Conformer checkpoint. The APIs are Speechmatics Enhanced, Deepgram Nova-3, IBM Large US English, Azure en-US, AssemblyAI Universal 3.5 Pro, Amazon Transcribe en-US, ElevenLabs Scribe v2, Google Chirp 2 and Olign 0.9. MFA is Montreal Forced Aligner 3.4. MAPS and NeuFA trained on 7 of 8 speakers in both Buckeye splits.

Family		TIMIT test						Buckeye test
	System	clean			noisy			clean			noisy
	Track 1 , reference words, except FALCON †
		MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	PER	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	PER	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	PER	MAE	$\boldsymbol{F_{1}^{\scriptscriptstyle 20}}$	PER
Open	BFA [ 7 ]	45.1	.18	36.2	48.9	.18	36.2	50.2	.25	32.1	0 54.8	.25	32.2
	TorchAudio [ 30 ]	33.8	.21	35.9	33.3	.21	35.9	34.5	.27	28.5	0 41.3	.26	28.5
	NeuFA [ 8 ]	20.5	.33	36.0	56.1	.28	39.1	22.6	.46	28.0	0 46.5	.40	30.7

Table 2: Phone-tier boundary error on the test split of each corpus, on clean audio and under degradation, noisy being the mean of the four, laid out as Table 1 . Every Track 1 aligner is given the reference words and derives its own phones, and PER is the phone error rate of what it derived against the gold phones, per split and condition. FALCON, daggered, is handed the gold phones, and overlaps our Buckeye split in its training data. We use the FALCON released model and did not retrain it. No one-step ASR in Table 1 emits phone timestamps, so Track 2 has only the two-step rows whose aligner does, and their PER carries the recognizer’s word errors too.

		Grid	Buckeye test clean					Buckeye test noisy
		(ms)	$\mathrm{M2}_{\scriptscriptstyle I}$	$\mathrm{M1}_{\scriptscriptstyle I}$	$\mathrm{M1}_{\scriptscriptstyle B}$	$\mathrm{M1}_{\scriptscriptstyle E}$	$\mathrm{M0}$	$\mathrm{M2}_{\scriptscriptstyle I}$	$\mathrm{M1}_{\scriptscriptstyle I}$	$\mathrm{M1}_{\scriptscriptstyle B}$	$\mathrm{M1}_{\scriptscriptstyle E}$	$\mathrm{M0}$
	% of boundaries		69.2	10.4	7.2	7.6	5.5	64.4	12.2	6.7	6.9	9.7
Open	Whisper-ts	20	.17	.08	.58	.03	.14	.17	.09	.57	.02	.14
	Whisper	20	.12	.06	.57	.05	.13	.12	.07	.57	.04	.14
	Parakeet	80	.23	.15	.22	.04	.08	.23	.16	.23	.03	.09
	W $\to$ WhisperX	20	.13	.07	.10	.28	.06	.13	.08	.08	.25	.08

Table 3: The

F_{1}

s at 20 ms of different types of word-tier boundaries of Track 2 on Buckeye test, on clean audio and under degradation, noisy being the mean of the four. A boundary inside (

I

) an utterance has two adjacent words. We use

\mathrm{M2}_{I}

to denote both words are matched and

\mathrm{M1}_{I}

to denote one label is matched, where M denotes matched.

\mathrm{M1}_{B}

and

\mathrm{M1}_{E}

denote the beginning (

B

) and the ending (

E

) boundary of an utterance, each with only one adjacent word.

\mathrm{M0}

denotes the remaining boundaries. The first row is the share of reference boundaries in each category, averaged over the rows.

Forced Alignment

See all 9 papers

Where Words Begin and End
Oct 13, 2026Husein ZolkepliForced Alignment Audio-Text Alignment
Scicom (MSC) Berhad, Malaysia
FuseAlign: Forced Alignment in the Wild
Sep 27, 2026Mithilesh Vaidya, Stephen Bailey, Sumukh Badam +2Forced Alignment Audio-Text Alignment
Descript, Inc.
Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation
Aug 28, 2026V. S. D. S. Mahesh Akavarapu, Michael Daniel, Gerhard JägerReference-Free Evaluation
University of Tübingen · University of Jena

Oct 13, 2026J/K move · Enter open · S save

Where Words Begin and End

Husein Zolkepli

Scicom (MSC) Berhad, Malaysia

Forced Alignment Audio-Text Alignment Speech Processing Speech Language Models

Open paper AskPDF arXiv

Forced aligners locate each word of a known transcript in the audio, and many speech pipelines depend on them. In our own data, a widely used CTC aligner inserts a pause at nearly every word boundary, and we found almost no aligner evaluated on telephone audio. We compare nine aligner configurations against human word boundaries on read, conversational and telephone English. The Montreal Forced Aligner (MFA) is the most precise, while CTC aligners consistently start words late, because most frames inside a word go to the blank token. We then fine-tune Whisper with LoRA to predict each word's start and end as timestamp tokens, training only on boundaries placed by people: 47 hours in 87 languages. The resulting aligner is more precise than MFA on every English test condition, and than MFA's own Korean and Dutch models. With FA-Bench's protocol and code on TIMIT, its word-boundary error is 11.6 ms, against 18.2 ms for the best published system. Two choices matter most on telephone conversations: placing the timestamp slots after the word they time, and training on noisy transcripts, which also makes the aligner nearly insensitive to errors in the transcript. The released model reaches 16.0 ms on real landline calls, against 36.5 ms for MFA. The models are available at https://huggingface.co/collections/Scicom-intl/force-alignment

Abstract

Forced Alignment

Explore similar work

Where Words Begin and End

FuseAlign: Forced Alignment in the Wild

Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation