Object Tokens

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 20

All topics

Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Sep 14, 2026Albert Ge, Chandan Singh, Yufan Zhuang +3Diffusion Language Models Masked Diffusion Language Models
University of Wisconsin–Madison · Microsoft Research · UC San Diego
Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Sep 1, 2026Matteo He, William F. Shen, Xinchi Qiu +1Readout Logit Lens
Department of Computer Science and Technology, University of Cambridge
Beyond Token Positions: Safety Alignment Across Denoising Steps in Diffusion Language Models
Aug 31, 2026Guoli Wang, Haonan Shi, Tu Ouyang +1Diffusion Language Models Large Language Model Safety
Case Western Reserve University 10900 Euclid Avenue, Cleveland, OH 44106, USA
DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
Aug 5, 2026Amirmohammad Karimi, Chao Gao, Negar HassanpourSpeculative Decoding Block-Diffusion Drafter
Huawei Technologies Canada Co., Ltd.
Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding
Aug 4, 2026Xiuyuan Zhu, Ke Lu, Kun Dong +6Long Visual-Token Sequences Recent Vision-Language Models
University of Chinese Academy of Sciences, Beijing, China · 2State Key Laboratory of Communication Content Cognition, Beijing, China · 3Peng Cheng Laboratory, Shenzhen, Guangdong, China +1
DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
Jul 29, 2026Yilei Wang, Jiaxin Gan, Kexuan Zhang +3Sequence Modeling Object Tokens
College of Computer and Data Science, Fuzhou University, Fuzhou, 350108, China · School of Computer Science, Peking University, Beijing, 100871, China
Labeled Incidence Structures for Native Transformer Modeling of Text, Knowledge Graphs, and Hypergraphs
Jul 29, 2026Mahesh GodavartiTransformer Attention Transformer Architectures
A Carrot, Inc
Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
Jul 21, 2026Maohua Li, Qirui Li, Yanke Zhou +10Diffusion Transformers Text-To-Image Diffusion Models
Nanjing University · Alibaba Group · Zhejiang University
Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation
Jul 20, 2026Peng Ren, Haoyang Ge, Jiang Zhao +4Humanoid Loco-Manipulation Goal-Conditioned Dynamic Manipulation
1BUAA · 2BZA · 4DeepCybo +2
Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment
Jun 26, 2026Zhuo Zuo, Li Yue, Wenhao Zheng +2Maximum Mean Discrepancy Latent Representation Alignment
College of Computer Science, Sichuan University, Chengdu, China · Dongfang Electric (Chengdu) Academy of Science and Technology Co., Ltd., Chengdu, China · Institute of Medical Information & Library, CAMS & PUMC, Beijing, China
Object Tokens as a Bridge Between Segmentation and Visual Question Answering in Robotic Surgery
Jun 14, 2026Yiping Li, Ronald de Jong, Romy van Jaarsveld +5Surgical Videos Recent Vision-Language Models
Department of Biomedical Engineering, Eindhoven University of Technology, The Netherlands · Department of Surgery, University Medical Center Utrecht, The Netherlands · Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands
Characterizing Cultural Localization in AI-Generated Stories
Jun 12, 2026Shaily Bhatt, Supriti Vijay, Jeremiah Milbauer +1Narrative Generation Narratives
Carnegie Mellon University
Wavelet as Tokenizer: Preliminary Results on a Shared Wavelet Token Schema for Natural Signals
May 30, 2026Shenghao DingAudio Tokenizers Wavelets
Yet Another AI
Visual-Advantage On-Policy Distillation for Vision-Language Models
May 21, 2026Ruiqi Liu, Xiaolei Lv, Gengsheng Li +8Recent Vision-Language Models Object Tokens
Institute of Automation, CAS · School of Advanced Interdisciplinary Sciences, UCAS · 3Hello +1
Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
May 19, 2026Peter El Hachem, Ahmed Nassar, A. Said Gurbuz +2Visual Document Retrieval Document Parsing
ETH Zurich Rämistrasse, Switzerland · IBM Research Rüschlikon, Switzerland
Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
May 7, 2026Zecheng Tang, Jiaye Fu, Qiankun Gao +5Feed-Forward 3D Reconstruction 3D Reconstruction
School of Electronic and Computer Engineering, Peking University, Shenzhen 518055, China
(1D) Ordered Tokens Enable Efficient Test-Time Search
Apr 16, 2026Zhitong Gao, Parham Rezaei, Ali Cy +7Autoregressive Image Generation Autoregressive Generation
1Swiss Federal Institute of Technology Lausanne (EPFL) · 2Apple
TokenLight: Precise Lighting Control in Images using Attribute Tokens
Apr 16, 2026Sumit Chaturvedi, Yannick Hold-Geoffroy, Mengwei Ren +5Image Relighting Lighting
Yale University · Adobe
Poly-attention: a general scheme for higher-order self-attention
Feb 2, 2026Sayak Chakrabarti, Toniann Pitassi, Josh AlmanLinear Attention Transformer Architectures
Computer Science, Columbia University, New York, NY 10027.
Fine PT-PT Web: A High-Quality 41 Billion Tokens Data Collection of the European Portuguese Web
Date pendingGonçalo Vinagre, Rui Pedro Guerra, Pedro Gomes +7Text Corpora Portuguese
NOVA School of Science and Technology · NOVA LINCS · Fundação para a Ciência e Tecnologia +2

Sep 14, 2026, cs.CLJ/K move · Enter open · S save

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Albert Ge, Chandan Singh, Yufan Zhuang, Xiaodong Liu+2

University of Wisconsin–Madison · Microsoft Research · UC San Diego

Diffusion Language Models Masked Diffusion Language Models Object Tokens Hidden States Long-Horizon Task Planning

Open paper AskPDF arXiv

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked tokens with bidirectional attention. Extending reasoning across generation chunks normally requires keeping earlier generated text in context. We ask whether a dLLM can instead continue reasoning after that text is cleared, using only a fixed-size carried state. We implement this state as a small number of register tokens: dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks. We post-train dLLMs to decode a chunk of text, clear it while preserving the register values, and continue decoding from the prompt and carried state. In our main comparisons on LLaDA and Dream, registers outperform discrete-text carry on every benchmark, with gains of up to 8.5 points on math and 19.5 points on code. Registers are especially effective for bounded code generation, where correct programs usually span several chunks. Finally, registers can be further refined with reinforcement learning on long-horizon reasoning tasks.

Momentum

Related topics

Latest papers 20