Multi-Token Prediction

Also known as MTP

Multi-token prediction studies how language and other sequence models can learn or generate several future tokens in one step rather than predicting only the next token. Approaches include auxiliary prediction heads, latent-space and blockwise generation, adaptive prediction horizons, and speculative decoding, where a target model verifies proposed tokens to preserve output quality. The goal is to improve training signals, reasoning or planning, and inference throughput across settings such as text generation, speech, document parsing, and recommendation. A central tension is that deeper or more parallel predictions can accelerate generation but are harder to get right, so current work focuses on balancing acceptance, accuracy, training cost, and memory use.

Latest papers

42 papers11 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
May 12, 2026Shaobin Zhuang, Yuang Ai, Jiaming Han +7Multi-Token Prediction Autoregressive Language Modeling
Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences +1
Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models
Apr 20, 2026Lin YaoLLM Self-Correction Multi-Token Prediction
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
LLM-Codec: Neural Audio Codec Meets Language Model Objectives
Apr 20, 2026Ho-Lam Chung, Yiming Chen, Hung-yi LeeNeural Audio Codecs Multi-Token Prediction
Graduate Institute of Communication Engineering, National Taiwan University · ASUS Intelligent Cloud Services · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
TableSeq: Unified Generation of Structure, Content, and Layout
Apr 17, 2026Laziz Hamdi, Amine Tamasna, Pascal Boisson +1Table Structure Recognition Multi-Token Prediction
LITIS, Rouen, Normandy, France · Malakoff Humanis, Paris, France
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
Apr 16, 2026Xinyu Liu, Xin Liu, Bo Jin +8Multi-Token Prediction KV Caching
School of Computer Science and Engineering, Northeastern University, China · Meituan Inc. · NiuTrans Research, Shenyang, China
How Transformers Learn to Plan via Multi-Token Prediction
Apr 13, 2026Jianhao Huang, Zhanpeng Zhou, Renqiu Xia +3Multi-Token Prediction LLM Planning
University of California, Los Angeles · Shanghai Jiao Tong University · University of Pennsylvania +2
MARS: Enabling Autoregressive Models Multi-Token Generation
Apr 8, 2026Ziqi Jin, Lei Wang, Ziwei Luo +1Multi-Token Prediction LLM Inference Acceleration
Nanyang Technological University · Singapore Management University · Uppsala University
Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement
Apr 7, 2026Qimin Zhong, Hao Liao, Haiming Qin +4Multi-Token Prediction Latent World Models
Shenzhen University · Microsoft Research Asia
Self-Distillation for Multi-Token Prediction
Mar 25, 2026Guoliang Zhao, Ruobing Xie, An Wang +3Multi-Token Prediction LLM Inference Acceleration
Large Language Model Department, Tencent
Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing
Mar 18, 2026Raghavv Goel, Mukul Gagrani, Mingu Lee +1Multi-Token Prediction LLM Inference Acceleration
Qualcomm AI Research
Efficient Document Parsing via Parallel Token Prediction
Mar 16, 2026Lei Li, Ze Zhao, Meng Li +6Efficient VLM Inference Document Parsing
Platform and Content Group, Tencent · Renmin University of China
Distribution-Aware Companding Quantization of Large Language Models
Feb 27, 2026Athul Radhakrishnan, Siddhant Mohan, Mahima SachdevaLLM Training Multi-Token Prediction

May 12, 2026J/K move · Enter open · S save

BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion

Shaobin Zhuang, Yuang Ai, Jiaming Han, Xiaohui Li+6

Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences+1

Multi-Token Prediction Autoregressive Language Modeling LLM Inference Acceleration Diffusion Language Model Decoding Diffusion Language Models

Open paper AskPDF arXiv

Autoregressive language models generate text one token at a time, yet natural language is inherently structured in multi-token units, including phrases, n-grams, and collocations that carry meaning jointly. This one-token bottleneck limits both the expressiveness of the model during pre-training and its throughput at inference time. Existing remedies such as speculative decoding or diffusion-based language models either leave the underlying bottleneck intact or sacrifice the causal structure essential to language modeling. We propose BitLM, a language model that represents each token as a fixed-length binary code and employs a lightweight diffusion head to denoise multiple tokens in parallel within each block. Crucially, BitLM preserves left-to-right causal attention across blocks while making joint lexical decisions within each block, combining the reliability of autoregressive modeling with the parallelism of iterative refinement. By replacing the large-vocabulary softmax with bitwise denoising, BitLM reframes token generation as iterative commitment in a compact binary space, enabling more efficient pre-training and substantially faster inference without altering the causal foundation that makes language models effective. Our results demonstrate that the one-token-at-a-time paradigm is not a fundamental requirement but an interface choice, and that changing it can yield a stronger and faster language model. We hope BitLM points toward a promising direction for next-generation language model architectures.