Subword Tokenization

Follow this topic Ask about this topic

Recent momentum

-50%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Subword Tokenization.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in Subword Tokenization.

31 papers

Latest in Subword Tokenization

Open your feed →

Do speech foundation models really learn words?
Sep 9, 2026Robin Huo, Ewan DunbarSelf-Supervised Speech Models Wav2Vec
The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100
Sep 3, 2026Francesco Mantegna, Gereon Elvers, Dulhan Jayalath +18Brain-Computer Interface Brain
Subword Segmental BabyLMs: Learning to Tokenise for Sample-Efficient Pretraining
Sep 1, 2026Francois MeyerSubword Tokenization Large Language Model Post-Training
Explicit Boundary Markers for Subword Vocabularies
Aug 9, 2026Sander Land, Clara MeisterSubword Tokenization Repetition
Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining
Aug 4, 2026Hai Wang, Chenhao Wang, Qifeng Cai +6Semantic Deduplication Repetition
Morphology Aware Reversible Semantic Tokenization and Hierarchical Word Composition for Tamil Language Models
Aug 2, 2026Anand MuruganSubword Tokenization Tokenizer
Pruned BPE: Post-training Visibility Pruning and Token Reallocation for Byte Pair Encoding
Aug 1, 2026Kenny ShaoByte-Pair Encoding Subword Tokenization
The Tokenizer Tax: Quantifying and Explaining the Cross-Lingual Cost of Subword Tokenization for Indian Languages
Jul 27, 2026Priyansh SrivastavaSubword Tokenization Cross-Lingual Consistency
BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis
Jul 25, 2026Poornima Kumaresan, Pavithra Muruganantham, Lakshmi Rajendran +1Subword Tokenization Byte-Pair Encoding
Workload-Driven Optimization for On-Device Real-Time Subtitle Translation
Jul 10, 2026Tsz-To WongSubword Tokenization Translation
Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES
Jul 6, 2026Hunter HeidenreichSubword Tokenization Byte-Pair Encoding
ROMEVA: Geometry-Preserving Vocabulary Expansion for Roman Urdu Language Models
Jun 21, 2026Mahnoor Khan, Afsheen Asif, Milhan Afzal Khan +2Urdu Multilingual Language Models
Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet
Jun 18, 2026Milan Miletić, Julie Kallini, Ekaterina ShutovaSubword Tokenization Multilingual Language Models
Examining the Limits of Word2Vec with Toki Pona
Jun 15, 2026Daniel Zhenhan Huang, Hongchen WuSubword Tokenization Linguistics
CraBERT: Efficient Phoneme Encoder Pre-Training via Cascade Fusion of Subword Representations for Text-to-Speech
Jun 15, 2026Dong Yang, Yuki Saito, Wataru Nakata +1Text-To-Speech Subword Tokenization
Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language
Jun 15, 2026Rabindra Nath NandiBangla Small Large Language Models
PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino
Jun 13, 2026Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3Phonological Understanding Morphology
S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP
Jun 11, 2026Mohammed Bouri, Mohammed Erradi, Adnane SaoudSubword Tokenization Natural Language Processing
Explaining Black-Box Language Models: Learning to Optimize Linguistically-Structured Word Subsets
Jun 7, 2026Minyoung Hwang, Seokhyun Lee, Changhee LeeModel Interpretability Methods Protein Language Model
The Tokenizer Tax Across 25 European Languages: Domain Invariance, Cross-Lingual Few-Shot Effects, and the Ukrainian Penalty
May 23, 2026Volodymyr OvcharovSubword Tokenization Tokenizer
Tokenization with Split Trees
May 21, 2026Craig W. Schmidt, Michael Krumdick, Adam Wiemerslage +4Subword Tokenization N$-Grams Features
Effective Context in Transformers: An Analysis of Fragmentation and Tokenization
May 13, 2026Amirmehdi Jafari Fesharaki, Mohammadamin Rami, Aslan TchamkertenSubword Tokenization Contextual
Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLP
May 13, 2026Ruan Visser, Trienko Grobler, Marcel DunaiskiSubword Tokenization Model Pretraining
Does language matter for spoken word classification? A multilingual generative meta-learning approach
May 13, 2026Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der MerweMultilingual Automatic Speech Recognition Multilingual Language Models
Scaling few-shot spoken word classification with generative meta-continual learning
May 13, 2026Louise Beyers, Batsirayi Mupamhi Ziki, Ruan van der MerweFew-Shot Learning Subword Tokenization
Dolphin-CN-Dialect: Where Chinese Dialects Matter
May 9, 2026Yangyang Meng, Huihang Zhong, Guodong Lin +6Dialects Automatic Speech Recognition Model
A Comprehensive Analysis of Tokenization and Self-Supervised Learning in End-to-End Automatic Speech Recognition applied on French Language
May 5, 2026Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa +1Automatic Speech Recognition Subword Tokenization
Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation
Apr 29, 2026Théo Gigant, Bowen Peng, Jeffrey QuesnelleSubword Tokenization Decoupling
How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
Apr 18, 2026Disen Liao, Freda ShiSubword Tokenization Phonological Understanding
Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion
Apr 17, 2026Maitrey Mehta, Nishant Subramani, Zhichao Xu +2Subword Tokenization Vocabulary
Less Is More: Reducing Token Counts Without Compromising Performance
Jun 18, 2025Gyeongje Cho, Yeonkyoung So, Sangmin Lee +1Subword Tokenization Tokenizer

Sep 9, 2026 · cs.CLJ/K move · Enter open · S save

Do speech foundation models really learn words?

Robin Huo, Ewan Dunbar

Self-supervised speech foundation models are now used in a wide array of downstream applications, including traditional speech recognition and as the basis for tokens in speech-aware language models. Attempts to understand their usefulness have largely focused on probing their representations' ability to discriminate phonemes and words. However, discriminative ability for words need not imply specialized representation of words per se. Good discrimination of words may be explained by good encoding of word form (phonemes) rather than form-independent word representations encoding identity or syntactic/semantic properties. By partialling out phoneme information using residualization, we show that, in later layers, HuBERT and wav2vec 2.0 do in general learn representations which encode words with reasonable fidelity independently of local phonetic content. We show that this simple approach to disentanglement can enhance higher-order linguistic information in word discovery tasks.

Self-Supervised Speech Models Wav2Vec Wireless Foundation Models Subword Tokenization Large Audio Language Models Foundation Model

Open paper Ask about it PDF