Subword Tokenization

Momentum

0 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 26

All topics
CardsList
  1. The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100

    Sep 3, 2026Francesco Mantegna, Gereon Elvers, Dulhan Jayalath +18Brain-Computer InterfacePretraining

  2. Pruned BPE: Post-training Visibility Pruning and Token Reallocation for Byte Pair Encoding

    Aug 1, 2026Kenny ShaoByte-Pair EncodingSubword Tokenization

  3. BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis

    Jul 25, 2026Poornima Kumaresan, Pavithra Muruganantham, Lakshmi Rajendran +1Subword TokenizationIndian Languages

  4. ROMEVA: Geometry-Preserving Vocabulary Expansion for Roman Urdu Language Models

    Jun 21, 2026Mahnoor Khan, Afsheen Asif, Milhan Afzal Khan +2Multilingual Language ModelsSubword Tokenization

  5. Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet

    Jun 18, 2026Milan Miletić, Julie Kallini, Ekaterina ShutovaSubword TokenizationMultilingual Language Models

  6. PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

    Jun 13, 2026Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3MorphologySubword Tokenization

  7. Tokenization with Split Trees

    May 21, 2026Craig W. Schmidt, Michael Krumdick, Adam Wiemerslage +4Subword TokenizationUnigram Tokenizer

  8. Effective Context in Transformers: An Analysis of Fragmentation and Tokenization

    May 13, 2026Amirmehdi Jafari Fesharaki, Mohammadamin Rami, Aslan TchamkertenTransformer ArchitecturesSubword Tokenization

  9. Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLP

    May 13, 2026Ruan Visser, Trienko Grobler, Marcel DunaiskiSubword TokenizationPretraining

  10. Dolphin-CN-Dialect: Where Chinese Dialects Matter

    May 9, 2026Yangyang Meng, Huihang Zhong, Guodong Lin +6Subword Tokenization

  11. A Comprehensive Analysis of Tokenization and Self-Supervised Learning in End-to-End Automatic Speech Recognition applied on French Language

    May 5, 2026Thibault Bañeras-Roux, Mickael Rouvier, Jane Wottawa +1Automatic Speech RecognitionSubword Tokenization

  12. Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion

    Apr 17, 2026Maitrey Mehta, Nishant Subramani, Zhichao Xu +2Subword TokenizationInterpretability

  13. Less Is More: Reducing Token Counts Without Compromising Performance

    Jun 18, 2025Gyeongje Cho, Yeonkyoung So, Sangmin Lee +1Subword Tokenization