Discrete Tokens

Follow this topic Ask about this topic

Recent momentum

+25%

5 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

1 new paper

A weekly snapshot of new work published in Discrete Tokens.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Discrete Tokens.

49 papers

Latest in Discrete Tokens

Browse all topics →

Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Sep 14, 2026Albert Ge, Chandan Singh, Yufan Zhuang +3Diffusion Language Models Masked Diffusion Language Models
TEFM: Token-Efficient Faithful Modeling for Structured Data
Sep 9, 2026Zhichao Hou, Lingdao Sha, Xueyu Mao +3Discrete Tokens
The Dynamics of Continuous Mixture Collapse in Language Models
Sep 2, 2026Ali BackourLatent States Mixtures
Prediction-Assisted Pricing and Admission for LLM APIs with Stochastic Token Consumption
Sep 1, 2026Patrick WongPrices Stochastic Delays
Beyond Token Positions: Safety Alignment Across Denoising Steps in Diffusion Language Models
Aug 31, 2026Guoli Wang, Haonan Shi, Tu Ouyang +1Diffusion Language Models Large Language Model Safety
StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems
Aug 13, 2026Yanwen Peng, Delvin Ce Zhang, Xi Wang +1Multi-Agent Large Language Model Large Language Model Backbones
DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
Aug 5, 2026Amirmohammad Karimi, Chao Gao, Negar HassanpourSpeculative Decoding Drafter
TokTier: Exact Stateful CPU+GPU Tokenization for Agentic LLM Serving
Jul 31, 2026Zhenyu Zhang, Zhichao CaoPer-Token Latency Large Language Model Serving
DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
Jul 29, 2026Yilei Wang, Jiaxin Gan, Kexuan Zhang +3Sequence Modeling Direct Preference Optimization
Pixels for Programs? A Cross-Provider Case Study of Input-Token Accounting for Source Code as Text and Images
Jul 23, 2026Ronak BhalgamiDiscrete Tokens Openai
Multi-Mask Diffusion Language Models for Few-Step Generation
Jul 22, 2026Sijin Chen, Yinuo Ren, Heyang Zhao +3Masked Diffusion Language Models Diffusion Language Models
Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
Jul 21, 2026Maohua Li, Qirui Li, Yanke Zhou +10Text-To-Image Diffusion Models Latent Reasoning Tokens
The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation
Jul 19, 2026Shayell Aharon Salomon, Noam Israel, Ido Safruti +1Persona Code Generation
Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026
Jul 16, 2026Anthony Miyaguchi, Murilo Gustineli, Adrian CheungSelf-Supervised Representations Neural Audio Codecs
The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese
Jul 12, 2026Siyuan Song, Zhiheng Qian, Yunhao Zhang +11Training Language Models Discrete Tokens
Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
Jul 3, 2026Kaley Brauer, Claudio Mayrink Verdun, Samuel MarksDiscrete Tokens Deepseek
Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
Jun 29, 2026Wenlong Wang, Fergal ReidChain-of-Thought Reasoning Large Language Model Reasoning
Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
Jun 29, 2026Shuochen Chang, Qingyang Liu, Shaobo Wang +8Efficient Latent Reasoning Intermediate Latent States
Position Spaces and Graphs
Jun 24, 2026Rita-Nathalia Assaf, Tom Davot, Frédéric Lardeux +1Position Structural Constraints
Dual Dimensionality for Local and Global Attention
Jun 17, 2026Zhiyuan Wang, Xuan Luo, Sirui Zeng +1Transformer Encoder Representational Capacity
When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis
Jun 9, 2026Orion Reblitz-RichardsonFragility Large Language Model Pretraining
A Unifying View of Attention Sinks: Two Algorithms, Two Solutions
Jun 6, 2026Lukas Fesser, Mozes Jacobs, Thomas Fel +2Discrete Tokens
Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories
Jun 3, 2026Kyungmin Park, Taesup KimInference-Time Defense Large Language Model Safety
LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling
Jun 3, 2026Daria Ledneva, Denis KuznetsovGenomics Wireless Foundation Models
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
Jun 2, 2026Ding Zhang, Runtao Zhou, Wenqing Zheng +3Graph Language Models Discrete Tokens
Wavelet as Tokenizer: Preliminary Results on a Shared Wavelet Token Schema for Natural Signals
May 30, 2026Shenghao DingAudio Tokens Discrete Wavelet Transform
Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
May 28, 2026Soumyadeep Jana, Sagar Nishad, Sanasam Ranbir SinghKey-Value Cache Compression Efficient Long-Context Inference
Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection
May 27, 2026Zhixing Zuo, Huilin He, Jiasheng Wu +1Fraud Detection Deep Semantic Representations
Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio
May 25, 2026Georgios Milis, Yubin Qin, Yihan Wu +1Watermarking Watermarks
CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer
May 23, 2026Piotr WilamCircuits Discrete Tokens
Vision Transformers Need Better Token Interaction
May 22, 2026Linxiang SuVision Transformer Dense Prediction
Learnability-Informed Fine-Tuning of Diffusion Language Models
May 21, 2026Shubham Parashar, Atharv Chagi, Jacob Helwig +5Diffusion Language Models Discrete Tokens
Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
May 19, 2026Peter El Hachem, Ahmed Nassar, A. Said Gurbuz +2Visual Document Retrieval Discrete Tokens
Learning Faster with Better Tokens: Parameter-Efficient Vocabulary Adaptation for Specialized Text Summarization
May 17, 2026Gunjan Balde, Soumyadeep Roy, Mainack Mondal +1Large Language Model Adaptation Summarization
ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both
May 14, 2026Ziyu Guo, Rain Liu, Xinyan Chen +1Latent Visual Reasoning Visual Reasoning
TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints
May 13, 2026Yoshio Kato, Shuhei TarashimaLarge Language Model Generation Discrete Tokens
A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoning
May 7, 2026Ilan Doron-Arad, Idan Mehalel, Elchanan MosselAutoregressive Generation Online Learning
Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
May 7, 2026Zecheng Tang, Jiaye Fu, Qiankun Gao +5Feed-Forward 3D Reconstruction 3D Reconstruction
Breaking Safety at the Token Boundary: How BPE Tokenization Creates Exploitable Gaps in LLM Alignment
May 1, 2026Tung-Ling Li, Hongliang Liu, Yuhao WuLarge Language Model Alignment Discrete Tokens
Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference
May 1, 2026Yuxuan Gao, Megan Wang, Yi Ling YuOnline Inference Artificial Intelligence Benchmarks
To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
Apr 23, 2026Nevena Lazić, Liam Fowl, András György +1Transformer Encoder Transformer Architectures
Slot Machines: How LLMs Keep Track of Multiple Entities
Apr 22, 2026Paul C. Bogdan, Jack LindseySentence-Level Classification Entities
(1D) Ordered Tokens Enable Efficient Test-Time Search
Apr 16, 2026Zhitong Gao, Parham Rezaei, Ali Cy +7Autoregressive Generation Generative Models
Executing as You Generate: Hiding Execution Latency in LLM Code Interpreters
Apr 1, 2026Zhensu Sun, Zhihao Lin, Zhi Chen +4Large Language Low-Latency Systems
Accelerating Masked Image Generation by Learning Controlled Latent Dynamics
Feb 27, 2026Kaiwen Zhu, Quansheng Zeng, Yuandong Pu +9Large-Scale Image Generation Masked Diffusion Language Models
You Can Learn Tokenization End-to-End with Reinforcement Learning
Feb 15, 2026Sam Dauncey, Roger WattenhoferTokenization Discrete Tokens
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Oct 11, 2025Wentian Zhu, Zhen Xiang, Wei Niu +1Large Language Model Jailbreaks Prompt-Level Mitigations
EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation
Sep 15, 2025Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan +3Token Compression Code Generation
Fine PT-PT Web: A High-Quality 41 Billion Tokens Data Collection of the European Portuguese Web
Date pendingGonçalo Vinagre, Rui Pedro Guerra, Pedro Gomes +7Text Corpora Portuguese

Sep 14, 2026 · cs.CLJ/K move · Enter open · S save

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Albert Ge, Chandan Singh, Yufan Zhuang, Xiaodong Liu+2

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked tokens with bidirectional attention. Extending reasoning across generation chunks normally requires keeping earlier generated text in context. We ask whether a dLLM can instead continue reasoning after that text is cleared, using only a fixed-size carried state. We implement this state as a small number of register tokens: dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks. We post-train dLLMs to decode a chunk of text, clear it while preserving the register values, and continue decoding from the prompt and carried state. In our main comparisons on LLaDA and Dream, registers outperform discrete-text carry on every benchmark, with gains of up to 8.5 points on math and 19.5 points on code. Registers are especially effective for bounded code generation, where correct programs usually span several chunks. Finally, registers can be further refined with reinforcement learning on long-horizon reasoning tasks.

Diffusion Language Models Masked Diffusion Language Models Discrete Tokens Hidden States

Open paper Ask about it PDF