Tokenizer

Follow this topic Ask about this topic

Recent momentum

-37%

10 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Tokenizer.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Tokenizer.

105 papers

Latest in Tokenizer

Open your feed →

On the role of the tokenizer in ECG transformer models
Sep 14, 2026Jiawei Li, Fabio Bonassi, Johan Sundström +2Tokenizer
StreamAlign: Streaming Text-Aligned Speech Tokenization
Sep 9, 2026Kang-wook Kim, Jinyoung Park, Jinsoo Kim +3Automatic Speech Recognition Tokenizer
BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models
Sep 9, 2026Shivam Singh, Aditya Yadavalli, Catherine Arnett +1Automatic Speech Recognition Model Whisper-Large-V3
Studying Image Tokenizers as Visual Languages in Unified Multimodal Models
Sep 8, 2026Siting Li, Zhengyang Wang, Simon Shaolei Du +2Multimodal Learning Tokenizer
Compressing Streaming Neural Audio Encoders via Latent-Space Distillation
Sep 3, 2026Prasanth Yadla, Mohammad Samragh, Dongseong Hwang +7Neural Audio Codecs Tokenizer
Subword Segmental BabyLMs: Learning to Tokenise for Sample-Efficient Pretraining
Sep 1, 2026Francois MeyerSubword Tokenization Large Language Model Post-Training
Beat-Synchronous Tokenization for ECG Transformers
Aug 31, 2026Ahmed Sameh, Nolan Wilson, Max Enderlein +1Electrocardiogram Tokenizer
Manacá-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation
Aug 31, 2026Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andre Machado PortoPortuguese Protein Language Model
Game-Agnostic Value Functions through Automatic JSON Feature Extraction
Aug 30, 2026Dien Nguyen, Diego Perez-LiebanaValue Functions Value
PRQ-KMeans: Projection Residual Quantization for Semantic ID Tokenization
Aug 25, 2026Yunxiao Luo, Siyuan Wang, Ben Chen +2Residual Quantization Tokenizer
TokEval: A Tokenizer Evaluation Suite
Aug 18, 2026Clara MeisterTokenizer Protein Language Model
Phoenix TTS: High-Fidelity Synthesis and Voice Conversion via Flow-Matching-Driven Speech Tokenization
Aug 12, 2026Peijie Chen, Zhuanling Zha, Zhipeng Nie +8Autoregressive Text-To-Speech Text-To-Speech
MedUP: Awakening Unified Understanding and Perception in Medical Vision-Language Models
Aug 11, 2026Yuan Wang, Hualiang Wang, Yixin Chen +6Medical Vision-Language Models Perception
Tokenizer Generator Coupling in Medical Image Generation
Aug 7, 2026Liam ChalcroftMedical Image Generation Tokenizer
KVAE: Family of Tokenizers for Multimodal Generative Models
Aug 6, 2026Andrey Shutkin, Denis Parkhomenko, Ivan Kirillov +11Discrete Image Tokenizers Variational Autoencoder Latent
Agogic: Performance-Timed Music Tokens for LLM-Native Text-to-Symbolic-Music Generation
Aug 4, 2026Junhao Chen, Mingjin Chen, Jingjia Mao +12Generative Music Models Tokenizer
Token-Native Storage: Read and Write in your Agent's Language
Aug 3, 2026Kumar ShivenduByte-Pair Encoding Tokenizer
Morphology Aware Reversible Semantic Tokenization and Hierarchical Word Composition for Tamil Language Models
Aug 2, 2026Anand MuruganSubword Tokenization Tokenizer
Writing-System-Level Tokenizer Adaptation for Byte-Level BPE
Aug 1, 2026Bohdan DidenkoByte-Pair Encoding Tokenizer
Agentic Graph Token Reasoning
Aug 1, 2026Zhuoyi Peng, Yi YangNeural Tokens Agentic Reasoning
Explicit Note-Event Tokenization and Pitch-Validity Constrained Decoding for MIDI-to-Tablature Transcription
Jul 29, 2026Ting-Kai Hsu, Wei-Chin Wang, Kai-Xi Hong +1Music Transcription Transcription
Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization
Jul 24, 2026Hao Wang, Kun Yuan, Wenlin Zhong +4On-Policy Distillation Tokenizer
Ordered Action Tokens for Visuomotor Policy Learning
Jul 23, 2026Chaoqi Liu, Yue Zhao, Haonan Chen +4Tokenizer Visuomotor Policy
A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities
Jul 22, 2026Stefanos Gkikas, Christian Arzate Cruz, Valentina Becchetti +3Tokenizer Infrared
Content is What Remains: Invariant Speech Tokenization from Parallel Utterances
Jul 21, 2026Laurin Wagner, Bernhard Thallinger, Miroslav Stankovic +1Audio Tokens Tokenizer
Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation
Jul 21, 2026Bo Liang, Chen Gong, Wei Gao +13D Human Pose Estimation Millimeter Wave
Attacking Graph Foundation Models Through Their Shared Representation
Jul 20, 2026Pankaj Kumar, Subhankar MishraMultimodal Graph Foundation Learning Novel Latent-Class Attack
When Close Enough Is Not Enough: Autoregressive Drift in Quantum Circuit Synthesis
Jul 14, 2026Mehdi Saeedi, Eddie Richter, Paul HartkeQuantum Circuits Circuits
Qwen-Music Technical Report
Jul 13, 2026Jin Xu, Kangdi Wang, Ruibin Yuan +24Music Generation Singing Voice Conversion
Tokenizing Numerical and Embedding Features for LLM RecSys
Jul 10, 2026Zhe Xu, Ankit Peshin, Chiyu Zhang +7Neural Tokens Tokenizer
Event-Based Token Sequences for Audio-Conditioned Music-Game Level Modeling
Jul 10, 2026Ke Zhang, Chu-Hsuan Hsueh, Kokolo IkedaMusical Structure Audio Tokens
FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention
Jul 8, 2026Athanasios ZerisTransformer Attention Fourier
NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
Jul 7, 2026Ziye Wang, Modi Shi, Chaojun Ni +5Vision-Language-Action Memory-Augmented Large Language Models
ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
Jul 7, 2026Tianjiao Yu, Xinzhuo Li, Yifan Shen +4Text-To-3D Generation 3D
BlueMagpie-TTS: A Token-Efficient Tokenizer, Language Model, and TTS for Taiwanese-Accent Code-Switching Speech
Jul 7, 2026Ho Lam Chung, Bo-Xuan Zheng, Cheng-Chieh Huang +8Tts Application Text-To-Speech
Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization
Jul 5, 2026Ryota Komatsu, Kota Kawakita, Takuma Okamoto +1Self-Supervised Speech Models Audio Tokens
SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference
Jul 1, 2026Kyan Mahajan, Mohammad SaqlainParameter-Efficient Fine-Tuning Methods Wireless Foundation Models
GEAR: Guided End-to-End AutoRegression for Image Synthesis
Jun 30, 2026Bin Lin, Zheyuan Liu, Chenguo Lin +8Autoregressive Image Generation Generative Models
BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language
Jun 29, 2026Haitao Wu, Qirui Zhang, Zhouheng Yao +8Brain Tokenizer
SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
Jun 29, 2026Tengyue Jiang, Chunpu Xu, Jiayue Kang +1Vision-Language-Action Policies Vision-Language-Action
Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling
Jun 26, 2026Qinfeng Zhu, Lei FanPanoramic Images Panoptic Segmentation
Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling
Jun 25, 2026Yiwen Yan, Wanning He, Yu-Wing TaiHuman Motion Generation Human Motion Understanding
ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
Jun 23, 2026Lei Xu, Haowei Wang, Shen Chen +3Digital Forensics Large Multimodal Models
JupOtter: Cell-Level Bug Detection in Jupyter Notebooks
Jun 22, 2026Lukas Ottenhof, Thibaud LutellierBug Tokenizer
LangMAP: A Language-Adaptive Approach to Tokenization
Jun 22, 2026Clara Meister, Suchir Salhan, Andrzej Szablewski +3Tokenizer Multilingual Language Models
Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformers
Jun 18, 2026Dong Hoon Lee, Seunghoon HongDiffusion Transformers Diffusion Models
BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation
Jun 17, 2026Max Van Puyvelde, Ibrahim Gulluk, Wim Van Criekinge +13D Latent Diffusion Model 3D Masked Autoencoders
Morpheus: A Morphology-Aware Neural Tokenizer and Word Embedder for Turkish
Jun 17, 2026Tolga ŞakarTokenizer Morphology
PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino
Jun 13, 2026Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3Phonological Understanding Morphology
MotionVLA: Vision-Language-Action Model for Humanoid Motion
Jun 13, 2026Nonghai Zhang, Siyu Zhai, Yanjun Li +5Humanml3D Video Latents
Equity with Efficiency: An Empirical Study of Tokenizers for Multilingual Large Language Models
Jun 13, 2026Kieron Seven Jun Wei Lee, Muhammad Reza Qorib, Andrew Ivan Soegeng +1Multilingual Language Models Tokenizer
From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion
Jun 10, 2026Yuchen Xian, Yunqiu Xu, Yang He +1Visible Image Fusion Tokenizer
Augmenting Molecular Language Models with Local $n$ -gram Memory
Jun 10, 2026Xinni Zhang, Zijing Liu, He Cao +2N$-Grams Features Protein Language Model
SARA: A Dual-Stream VAE for High-Fidelity Speech Generation via Integrating Semantic and Acoustic Representations
Jun 10, 2026Peijie Chen, Wenhao Guan, Weijie Wu +7Autoregressive Text-To-Speech Text-To-Speech
What the Eyes See, the LLMs Miss: Exploiting Human Perception for Adversarial Text Attacks
Jun 8, 2026Qin Yang, Lu Malloy, Joshua Lee +4Language Model Evasion Attacks Content Moderation
Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
Jun 8, 2026Yifan Niu, Han Xiao, Dongyi Liu +4On-Policy Distillation Tokenizer
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
Jun 8, 2026Yejin Lee, Junwon Moon, Hyoeun Kim +3Autoregressive Text-To-Speech Audio Tokens
X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining
Jun 7, 2026Miracle Kang, Lights Shi, Lucy Liang +10Discrete Image Tokenizers Vision-Language-Action
AdaTok: Self-Budgeting Image Tokenization with Quality-Preserving Dynamic Tokens
Jun 5, 2026Xiaocheng Lu, Yuxi Chen, Jie Zhang +5Discrete Image Tokenizers Tokenizer
TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management
Jun 4, 2026Shweta MishraPersistence Contextual Reasoning

Sep 14, 2026 · cs.LGJ/K move · Enter open · S save

On the role of the tokenizer in ECG transformer models

Jiawei Li, Fabio Bonassi, Johan Sundström, Thomas B. Schön+1

Tokenization determines both the physiological content presented to an ECG Transformer and the sequence over which attention operates. We compare eight tokenization strategies across Transformer, Informer, Reformer, and FEDformer on the nine-label CPSC2018 classification task. The input projection and principal backbone capacity are controlled to isolate the effect of token construction. Median-beat and HeartLang tokenization achieve mean macro-AUCs of 0.893 and 0.889 across the four backbones, compared with 0.822 and 0.824 for point-wise and patch-wise tokenization. Pooling the two physiology-aware representations yields an 8.2% relative improvement in macro-AUC. They also reduce mean sequence length from 1,250 to 158 tokens and mean peak training memory from 5.21 to 0.27 GB. The results show that aligning tokens with ECG morphology can improve both predictive performance and memory efficiency without increasing backbone capacity. The source code is available on https://github.com/LeeJarvis996/ecg_tokenizer.