findsylls: A Language-Agnostic Toolkit for Syllable-Level Speech Tokenization and Embedding

Héctor Javier Vázquez Martínez

cs.CLMar 27, 2026

findsylls: A Language-Agnostic Toolkit for Syllable-Level Speech Tokenization and Embedding

Authors: Héctor Javier Vázquez Martínez

Organizations: Department of Linguistics, University of Pennsylvania, USA

Abstract

Syllable-level units offer compact and linguistically meaningful representations for spoken language modeling and unsupervised word discovery, but research on syllabification remains fragmented across disparate implementations, datasets, and evaluation protocols. We introduce findsylls, a modular, language-agnostic toolkit that unifies classical syllable detectors and end-to-end syllabifiers under a common interface for syllable segmentation, embedding extraction, and multi-granular evaluation. The toolkit implements and standardizes widely used methods (e.g., Sylber, VG-HuBERT) and allows their components to be recombined, enabling controlled comparisons of representations, algorithms, and token rates. We demonstrate findsylls on English and Spanish corpora and on new hand-annotated data from Kono, an underdocumented Central Mande language, illustrating how a single framework can support reproducible syllable-level experiments across both high-resource and under-resourced settings.

Figures & tables

Name	Type	Hours	# Syllables
LS-100h	Adult read (English)	100	1374476
WikiSpanish	Adult read (Spanish)	25	530816
TIMIT	Adult read (English)	5	63265
PHC	Child-directed (English)	0.84	10631
Ornat-Swingley	Child-directed (Spanish)	0.24	2755
Brent	Child-directed (English)	0.11	1487

Table 1: Evaluation corpora.

Features	Envelope	Segmentation	Nucleus $\uparrow$			Boundary $\uparrow$			Span $\uparrow$			Tok/s $\downarrow$	RTFx $\uparrow$
Features	Envelope	Segmentation	Prec	Rec	F1	Prec	Rec	F1	Prec	Rec	F1	Tok/s $\downarrow$	RTFx $\uparrow$
Audio (raw)	SBS	peakdetect	90.0	92.4	91.2	65.1	56.8	60.6	37.9	36.3	37.1	3.5	684x
Audio (raw)	Theta	peakdetect	86.5	80.1	83.1	52.6	41.5	46.3	19.7	17.0	18.2	3.1	27x
Sylber	Cos. Sim.	Cos. thresh.	89.9	97.0	93.3	51.9	80.6	63.1	45.8	45.6	45.7	3.4	36x
VG-HuBERT	featSSM	Mincut	78.2	89.3	83.0	66.1	64.8	65.0	36.3	39.5	37.6	3.8	6x
VG-HuBERT	CLS attn.	CLS thresh.	37.4	48.2	41.9	40.3	44.4	42.1	12.0	14.7	13.2	5.8	72x

Table 2: Segmentation performance averaged over seven corpora (Brent, Kono, LibriSpeech, Ornat, PhiladelphiaHomeCorpus, TIMIT, WikiSpanish). Metrics are syllable-weighted Precision (Prec), Recall (Rec), and F1. Nucleus = syllable peak detection; Boundary = inter-syllabic boundary detection; Span = full syllable interval recovery.

\uparrow

indicates higher is better;

\downarrow

lower is better. Tok/s = predicted syllable nuclei per second of audio (lower = more compression). RTFx = inverse real-time factor (total_audio_duration / elapsed_time) reported to illustrate relative throughput under a fixed hardware/software setup.

Figure 1: Syllable boundary segmentation across datasets. (a) Boundary F1 by method and dataset for envelope baselines and default published neural configurations (higher is better; Ornat-Swingley excluded due to missing syllable intervals). (b) Sylber component effect: $\Delta$ F1 when swapping the segmenter (CosThresh $\rightarrow$ peakdetect ) with the cosine-similarity cue held fixed. (c) VG-HuBERT component effect: $\Delta$ F1 when swapping the cue/envelope (SSM $\rightarrow$ GreedyCosine) with the peakdetect segmenter held fixed; positive values indicate improvement from the substituted component.

Audio Tokenization

Audio tokenization converts speech, music, and other sounds into compact units for machine learning, generation, and understanding.

See all 42 papers

ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modeling
Feb 17, 2026Nicol Visser, Simon Malan, Danel Slabbert +1Audio Tokenization Speech Language Models
Electrical and Electronic Engineering, Stellenbosch University, South Africa
Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization
Jul 5, 2026Ryota Komatsu, Kota Kawakita, Takuma Okamoto +1Audio Tokenization Self-Supervised Speech Representation Learning
Institute of Science Tokyo, Meguro, Tokyo 152-8550, Japan · National Institute of Information and Communications Technology, Kyoto 619-0289, Japan
Beyond Atomic Tokens: Factorizing Syllables for Language Model Pretraining
Sep 18, 2026Nghia Hieu Nguyen, Thai Bao Huynh, Binh-An Dinh-Le +4Tokenization Language Model Pretraining
University of Information Technology, Vietnam National University, Ho Chi Minh city, Viet Nam · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE, Abu Dhabi, UEA

Feb 17, 2026J/K move · Enter open · S save

ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modeling

Nicol Visser, Simon Malan, Danel Slabbert, Herman Kamper

Electrical and Electronic Engineering, Stellenbosch University, South Africa

Audio Tokenization Speech Language Models Discrete Speech Representation Learning Subword Tokenization

Open paper AskPDF arXiv

Pure speech language models aim to learn language directly from raw audio without textual resources. A key challenge is that discrete tokens from self-supervised speech encoders result in excessively long sequences, motivating recent work on syllable-like units. However, methods like Sylber and SyllableLM rely on intricate multi-stage training pipelines. We propose ZeroSyl, a simple training-free method to extract syllable boundaries and embeddings directly from a frozen WavLM model. Using L2 norms of features in WavLM's intermediate layers, ZeroSyl achieves competitive syllable segmentation performance. The resulting segments are mean-pooled, discretized using K-means, and used to train a language model. ZeroSyl outperforms prior syllabic tokenizers across lexical, syntactic, and narrative benchmarks. Scaling experiments show that while finer-grained units are beneficial for lexical tasks, our discovered syllabic units exhibit better scaling behavior for syntactic modeling.

Abstract

Audio Tokenization

Explore similar work

ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modeling

Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

Beyond Atomic Tokens: Factorizing Syllables for Language Model Pretraining