LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Robust Voice Activity Detection

Ioannis Stylianou; Achintya kr. Sarkar; Nauman Dawalatabad; James Glass; Zheng-Hua Tan

cs.SDDec 19, 2025

LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Robust Voice Activity Detection

Authors: Ioannis Stylianou, Achintya kr. Sarkar, Nauman Dawalatabad, James Glass, Zheng-Hua Tan

Organizations: Department of Electronic Systems, Aalborg University, Denmark · Pioneer Centre for AI, Denmark · ECE Group, IIIT SriCity, Chittoor, India · Zoom Communications Inc., USA ·

Abstract

Robust Voice Activity Detection (VAD) remains a challenging task, especially under noisy, diverse, and unseen acoustic conditions. Beyond algorithmic development, a key limitation in advancing VAD research is the lack of large-scale, systematically controlled, and publicly available datasets. To address this, we introduce LibriVAD -- a scalable open-source dataset derived from LibriSpeech and augmented with diverse real-world and synthetic noise sources. LibriVAD enables systematic control over speech-to-noise ratio, silence-to-speech ratio (SSR), and noise diversity, and is released in three sizes (15 GB, 150 GB, and 1.5 TB) with two variants (LibriVAD-NonConcat and LibriVAD-Concat) to support different experimental setups. We benchmark multiple feature-model combinations, including waveform, Mel-Frequency Cepstral Coefficients (MFCC), and Gammatone filter bank cepstral coefficients, and introduce the Vision Transformer (ViT) architecture for VAD. Our experiments show that ViT with MFCC features consistently outperforms established VAD models such as boosted deep neural network and convolutional long short-term memory deep neural network across seen, unseen, and out-of-distribution (OOD) conditions, including evaluation on the real-world VOiCES dataset. We further analyze the impact of dataset size and SSR on model generalization. Experimental results show that scaling the dataset size from small-scale to medium- or large-scale noticeably and consistently improves VAD performance under OOD conditions, while appropriately balancing SSR enhances OOD performance in the small-scale setting. All datasets, trained models, and code are publicly available to foster reproducibility and accelerate progress in VAD research.

Figures & tables

Figure 1: An example of forced alignments on a clip from the LibriSpeech dataset. The bottom plot illustrates the phoneme boundaries, the top plot shows the corresponding word boundaries generated by the Montreal Forced Aligner, and the middle plot presents the spectrogram.

		Datasets
		LibriSpeech			LibriSpeech-Concat
Category	Characteristics	train-clean-100	test-clean	dev-clean	train-clean-100	test-clean	dev-clean
Signal	Average length	12.71s	7.42s	7.18s	31.72s	18.56s	17.94s
	Standard Deviation	3.56s	5.15s	4.69s	6.80s	10.31s	9.29s
	Maximum length	24.53s	34.96s	32.65s	45.70s	69.29s	69.76s
	Minimum length	1.41s	1.29s	1.45s	5.32s	4.84s	5.05s

Table 1: Characteristics of LibriSpeech and LibriSpeech-Concat datasets.

Average signal length	10s
Maximum signal length	47.7s
Minimum signal length	3.5s
Number of signals	28000

Table 2: Signal length statistics of the WHAM! dataset.

Environments	Conditions
	Domestic: Washroom (DWASHING), Kitchen (DKITCHEN), Living Room (DLIVING)
Indoor	Office: Office (OOFFICE), Hallway (OHALLWAY), Meeting Room (OMEETING)
	Public: Subway Station (PSTATION), Cafeteria (PCAFETER), Restaurant (PRESTO)
	Transportation: Subway (TMETRO), Bus (TBUS),Car (TCAR)
Outdoor	Nature: Field (NFIELD), River (NRIVER), Park (NPARK)
	Street: Traffic Intersection (STRAFFIC), Town Square (SPSQUARE), Cafe Terrace (SCAFE)

Table 3: Overview of the different recording environments and conditions available in the DEMAND noise dataset.

Noise Specifics
Noise type	Source	Duration (Train,Val,Test)	Unseen	Seen
Babble	LibriLight-medium	(3h, 30m, 30m)	✓	✗
Speech Shaped	LibriLight-small	(1h, 10m, 10m)	✓	✗
Domestic	DEMAND	(3h, 30m, 30m)	✓	✗
Nature	DEMAND	(3h, 30m, 30m)	✗	✓
Office	DEMAND	(3h, 30m, 30m)	✗	✓

Table 4: Overview of the noise sources used in LibriVAD, including noise type, source, duration of each split, and whether the noise is used for generating the training set (seen) or not (unseen). Training splits for unseen noises are still included, so researchers can experiment with different seen-unseen combinations.

LibriVAD Dataset Variants
Dataset Variant	Size Variant	Size (Train, Val, Test)	Duration (Train, Val, Test)
LibriVAD-NonConcat	Small	5.9GB, 0.34GB, 0.33GB	54.6, 3.1, 3.1 hours
	Medium	59GB, 3GB, 3.3GB	545.9, 27.4, 30.2 hours
	Large	584GB, 32GB, 32GB	5440, 291, 292 hours
LibriVAD-Concat	Small	7.4GB, 0.39GB, 0.45GB	68.7, 3.5, 4.1 hours
	Medium	73GB, 3.8GB, 3.9GB	679.8, 35.2, 35.9 hours

Table 5: Overview of LibriVAD dataset variants, including their respective sizes and durations.

Dataset	Source Duration (hrs)	Full Duration (hrs)	Noise types and data characteristics	SNR (dB)	Availability
AVA-Speech [ 3 ]	Unspecified	$\sim 45$	Speech with background music/noise from movie audio; speaker count unspecified	Unspecified and unchangeable	Open source
RATs [ 52 ]	350	3000	Channel distortion; 16kHz; speaker count unspecified	Unspecified	Purchased license
Apollo [ 17 ]	-	80	Channel distortion and extended periods of non-speech; 8kHz; speaker count unspecified	0-20 (approx.)	Open source
Aurora-2 [ 20 ]	$\sim 6$	$\sim 39$	Airport, babble, car, exhibition, restaurant, street, subway, train; 8kHz; 110 speakers	-5, 0, 5, 10, 15, 20	Purchased license
N-Timit [ 12 ]	5	$\sim 55$	Telephone - channel distortion with narrowband characteristics; stored at 16kHz; 630 speakers.	Unspecified	Purchased license
LibriVAD-NonConcat	100	$55$ / $546$ / $5440$	Nature, office, public, street, transport, city, babble, SSN, domestic; 16kHz; 330 speakers	-5, 0, 5, 10, 15, 20	Open source (MIT)

Table 6: Comparison of LibriVAD and existing VAD datasets across duration, noise characteristics, and availability.

Model	SNR	Seen						Unseen
		Nature	Office	Public	Street	Transport	City	Babble	SSN	Domestic
raw-CLDNN	-5	.8229	.8501	.7104	.8122	.9049	.7402	.5254	.6635	.773
	0	.8738	.8947	.7983	.872	.931	.8164	.6008	.7719	.8265
	5	.9079	.9218	.8624	.9094	.9481	.8683	.6969	.8479	.8718
	10	.9306	.9386	.9019	.9345	.9577	.9044	.7913	.896	.9105
	15	.9479	.9505	.9281	.9502	.9631	.9309	.866	.9257	.939

Table 7: Comparison of VAD performance across different feature-model combinations (AUC) using small-scale training and test sets on the LibriVAD-NonConcat dataset.

Model	SNR	Seen						Unseen
		Nature	Office	Public	Street	Transport	City	Babble	SSN	Domestic
raw-CLDNN	-5	.906	.9068	.7253	.7889	.9156	.7693	.524	.6815	.5259
	0	.9377	.939	.8239	.8705	.9437	.8482	.5996	.7946	.6586
	5	.9566	.9558	.8896	.915	.961	.899	.7011	.8696	.7826
	10	.9686	.9675	.9267	.9403	.9721	.9282	.7986	.9103	.8785
	15	.9762	.9752	.9479	.9548	.9785	.9468	.8742	.9367	.9332

Table 8: Comparison of VAD performance across different feature-model combinations (AUC) using small-scale training and test sets on the LibriVAD-Concat dataset.

	raw-	bDNN	#of encoder layers in ViT
	CLDNN		1	2	4	8	10	12	14	16
Model Parameters	157k	406.5k	471.9k	916.2k	1.81M	3.58M	4.47M	5.36M	6.25M	7.14M
LibriVAD-NonConcat	.8728	.8785	.9565	.9573	.9635	.9574	.9636	.9574	.9621	.9588
LibriVAD-Concat	.8807	.9026	.9697	.9710	.9732	.9733	.9725	.9723	.9693	.9735

Table 9: Effect of varying number of encoder layers in the ViT compared to bDNN and CLDNN for MFCC based system on the LibriVAD-NonConcat and LibriVAD-Concat datasets. All evaluations are performed under the respective small-scale training and test conditions, and the average AUC across all noise conditions is reported.

Panel of Figure 2 — Figure 2: Illustration of the t-SNE [ 51 ] scatter plots of the unseen evaluation noisy data based on the last layer hidden representations in the respective systems for the LibriVAD-NonConcat and LibriVAD-Concat datasets under the small-scale condition. Each system uses the same data points for the respective datasets. $NS$ (blue) and $S$ (orange) indicate the non-speech and speech classes, respectively. (a) LibriVAD-NonConcat. (b) LibriVAD-Concat.

Model	Parameters	Training time (Seconds per Epoch)		Evaluation time (Seconds/RTF)
		LibriVAD-NonConcat	LibriVAD-Concat	LibriVAD-NonConcat	LibriVAD-Concat
raw-CLDNN	157k	305.48	383.23	9.58/8.58 $\times 10^{-4}$	11.96/8.10 $\times 10^{-4}$
MFCC-bDNN	406.5k	93.03	125.34	2.92/2.62 $\times 10^{-4}$	3.32/2.24 $\times 10^{-4}$
MFCC-ViT:
- 1 Encoder layer	471.9k	18.90	21.68	6.64/5.94 $\times 10^{-4}$	6.75/4.57 $\times 10^{-4}$
- 12 Encoder layers	5.36M	147.68	190.55	36.47/32.67 $\times 10^{-4}$	57.29/38.81 $\times 10^{-4}$

Table 10: Comparison of model parameters, training time, and inference time for the different systems on the LibriVAD-NonConcat and LibriVAD-Concat datasets under the small-scale training and test conditions.

Table 11: Impact of training data size on VAD performance (AUC) using the ViT model with MFCC features on the LibriVAD-NonConcat and LibriVAD-Concat datasets. All evaluations are conducted on the respective small test sets to ensure consistency.

Train data	Train	Test	AUC
Size	Database	Database
Small	LibriVAD-NonConcat	LibriVAD-NonConcat	.9865
Small	LibriVAD-Concat	LibriVAD-Concat	.9934
Medium	LibriVAD-NonConcat	LibriVAD-NonConcat	.9947
	LibriVAD-Concat	LibriVAD-Concat	.9977
Large	LibriVAD-NonConcat	LibriVAD-NonConcat	.9951

Table 12: Comparison of VAD performance (AUC) across different training data sizes and datasets on clean test data under the small-scale evaluation setup using the ViT-MFCC model.

Table 13: Effect of training-test data mismatch on VAD performance (AUC) using the MFCC-ViT technique on the LibriVAD-NonConcat and LibriVAD-Concat datasets. Evaluations are conducted under the small-scale setup across models trained with small, medium, and large data sizes within each respective dataset.

Train data	Train data	Test data	System	EER (%)	MinDCF
Size	Concat /NonConcat	Concat /NonConcat
	LibriVAD-NonConcat	LibriVAD-NonConcat	raw-CLDNN	19.29	.0756
Small	✓	✓	MFCC-bDNN	19.52	.0919
	✓	✓	GFCC-bDNN	22.13	.0953
-	-	✓	Silero-VAD	26.06	.0593
	✓	✓	MFCC-ViT	10.40	.0528

Table 14: Comparison of VAD system performance based on EER and MinDCF under the small-scale evaluation setup.

Panel of Figure 3 — Figure 3: DET curves for different VAD models across a wide operating range, with scores aggregated over various noise types and SNR levels. (a) Model trained and evaluated on the small set of LibriVAD-NonConcat. (b) Model trained and evaluated on the small set of LibriVAD-Concat.

Table 15: Comparison of VAD performance on the out-of-distribution VOiCES devset using the ViT-MFCC model. Models are trained on the LibriVAD-NonConcat and LibriVAD-Concat datasets, with VAD labels generated by the Montreal Forced Aligner.

Train data	Microphone	Recording	Distractors				Avg.	Overall
Size	Id (position)	Room	Babble	Music	Television	None	AUC	Avg. AUC
Baseline	clo	rm1	.9733	.9613	.9675	.9683	.9676
(Silero)		rm2	.9565	.9709	.9722	.9771	.9692
		rm3	.8454	.8204	.8753	.9246	.8664
		rm4	.7599	.9667	.9733	.9772	.9193	.9306
	far	rm1	.8449	.8857	.8965	.9428	.8925

Table 16: Silero-VAD performance on the out-of-distribution VOiCES devset.

Table 17: Comparison of VAD system performance based on EER and MinDCF on the VOiCES dataset.

Table 18: Effect of the SSR on the VAD performance of MFCC-ViT, trained under the LibriVAD-Concat small-scale setup using different SSR values and evaluated on both the LibriVAD-Concat small-scale test set and the VOiCES dataset. SSR values of 0.2 and 0.5 correspond to LibriVAD-NonConcat and LibriVAD-Concat, respectively; the associated results are reproduced from previous tables for comparison.

Model	Silero	LibriVAD
Training Dataset		Small-NonConcat	Small-Concat	Large-NonConcat	Large-Concat
Silero		0.124	0.005	$<0.001$	$<0.001$
Small-NonConcat	0.124		$<0.001$	$<0.001$	$<0.001$
Small-Concat	0.005	$<0.001$		$<0.001$	$<0.001$
Large-NonConcat	$<0.001$	$<0.001$	$<0.001$		$<0.001$
Large-Concat	$<0.001$	$<0.001$	$<0.001$	$<0.001$

Table 19: Pairwise Wilcoxon signed-rank test

p

-values comparing the out-of-domain generalization of five models: Silero and the four ViT-MFCC models trained on all combinations of small-scale vs. large-scale training and LibriVAD-NonConcat vs. LibriVAD-Concat datasets across the 32 acoustic conditions in the VOiCES dataset. Models are ordered by ascending overall performance. To account for multiple comparisons,

p

-values are adjusted using the Holm-Bonferroni procedure. Values where

p<0.05

indicate a statistically significant difference in performance.

Voice Activity Detection

See all 12 papers

Oct 10, 2026

Window Size Matters: Benchmarking Voice Activity Detectors in Noise

Voice activity detection (VAD) sits in front of almost every streaming speech recognizer, yet the choice of detector is often made by feel rather than by measurement. We report a frame-level offline benchmark of three detectors on the test-clean split of LibriVAD-small: the neural TEN VAD, the neural Silero VAD and a simple RMS energy baseline. The evaluation covers 2,214 noisy utterances (about 7.1 hours) built from LibriSpeech, nine noise families and six SNR levels from -5 to 20 dB. At a 16 ms hop, TEN VAD reaches a ROC AUC of 0.917, an average precision of 0.968 and an F1 of 0.913. A sweep over 20, 30, 40 and 50 ms windows shows TEN to be almost insensitive to frame length (AUC 0.916 to 0.919), while RMS stays near 0.75 to 0.77. Silero matches TEN only near its native 32 ms input size and drops to 0.63 at 20 ms. We hypothesize that this is likely caused by our fixed-frame protocol rather than by the model itself. Zero padding, sub-chunk averaging and unreset model state are the likely factors, but we have not ablated them. In a per-condition breakdown of TEN, babble noise at low SNR stays close to chance. We also show that F1 at a fixed 0.5 threshold is a weak metric on this data, because a trivial all-speech detector already scores 0.854. All settings needed to reproduce the results are given in the paper.

Alok Kumar, Mohit Srivastava, Anupam Purwar +4

OneByZero AI

Voice Activity Detection Speech Processing

Ask PDF

Jul 28, 2026

VAD to the Bone: Ultra-Tiny Speech Activity Detection for Edge Deployment

Voice activity detection (VAD) triggers downstream speech processing in always-on systems under strict memory, latency, and compute constraints. Recent compact models report strong accuracy but rely on components that are not widely supported: learnable filterbanks, recurrent layers, or non-causal post-processing. We propose kiloVAD, designed for embedded inference using standard Mel features, CNN-only layers, and tunable context/spectral parameters. We introduce per-layer structured pruning with self-distillation and angle-based quantization-aware training (QAT) that outperforms standard QAT by 1-4%. Evaluated per-frame under causal conditions, kiloVAD achieves 0.850 AUC on AVA-Speech with 2.1 k parameters and 200 ms context, establishing a new state of the art for causal, deployment-ready VAD.

Stephen Bauer, Sheila Seidel, Shanza Iftikhar +2

Analog Devices, Inc., USA · University of California, Los Angeles (UCLA), USA

Model Compression Speech Processing Voice Activity Detection

Ask PDF

Aug 12, 2019

Personal VAD: Speaker-Conditioned Voice Activity Detection

In this paper, we propose "personal VAD", a system to detect the voice activity of a target speaker at the frame level. This system is useful for gating the inputs to a streaming on-device speech recognition system, such that it only triggers for the target user, which helps reduce the computational cost and battery consumption, especially in scenarios where a keyword detector is unpreferable. We achieve this by training a VAD-alike neural network that is conditioned on the target speaker embedding or the speaker verification score. For each frame, personal VAD outputs the probabilities for three classes: non-speech, target speaker speech, and non-target speaker speech. Under our optimal setup, we are able to train a model with only 130K parameters that outperforms a baseline system where individually trained standard VAD and speaker recognition networks are combined to perform the same task.

Shaojin Ding, Quan Wang, Shuo-yiin Chang +2

Texas A&M University, USA · Google Inc., USA

Speech Processing Voice Activity Detection

Ask PDF

Abstract

Voice Activity Detection

Explore similar work

Window Size Matters: Benchmarking Voice Activity Detectors in Noise

VAD to the Bone: Ultra-Tiny Speech Activity Detection for Edge Deployment

Personal VAD: Speaker-Conditioned Voice Activity Detection