cs.CLNov 10, 2026

On the information rate of phones, syllables and words

Authors: Herman Kamper

Organizations: Electrical and Electronic Engineering, Stellenbosch University, South Africa

Abstract

Which linguistic units minimise the bitrate required to encode speech content: phones, syllables, words, maybe larger units, or something in between? This question has implications for the development of speech and language systems and theories of human language processing. Using a transcribed corpus of 960 hours of English audiobooks, I compare the information rates of phones, syllables, words, multi-word combinations and intermediate tokenisations. While all units are equivalent under a perfect language model, it makes sense to compare these under limited modelling budgets; I use nn-gram and transformer language models. The lowest bitrates are achieved using words, with slightly longer units sometimes performing best. The results show that, when modelling capacity is limited, units larger than syllables but smaller than long multi-word phrases provide the most efficient representation in terms of information rate.

Figures & tables

Explore similar work

CardsList
  1. findsylls: A Language-Agnostic Toolkit for Syllable-Level Speech Tokenization and Embedding

    Mar 27, 2026Héctor Javier Vázquez MartínezAudio TokenizationDiscrete Speech Representation Learning