Autoregressive Token Prediction

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 18

All topics

NesTok: Nested Self-Aligned 1D Tokenizer for Autoregressive Image Generation
Sep 29, 2026Jiawei Zhang, Shuhao Liu, Rong Huang +4Autoregressive Image Generation Visual Tokenizers
North China Electric Power University · University of Science and Technology of China · Stanford University
Variational Quantum Transformer Architecture for Synthetic Language Generation
Sep 16, 2026Julian Hager, Michael Kölle, Gerhard Stenzel +3Transformer Architectures Autoregressive Token Prediction
Institute of Informatics, LMU Munich, Oettingenstraße 67, 80538 Munich, Germany
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Aug 31, 2026Zihan Qiu, Zekun Wang, Xiao Li +33Autoregressive Token Prediction Unigram Tokenizer
Qwen Team
ReLMCodec: Designing Predictable Speech Tokens from Pre-Quantization Phoneme Structure
Aug 8, 2026Zixiang Wan, Xusheng Yang, Zheng Wang +1Neural Speech Codecs Self-Supervised Speech Models
1Peking University, Shenzhen, China · 2Tencent, Shenzhen, China
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Jul 28, 2026Hong Liu, Rui Cen, Junhan Shi +10Speculative Decoding Block-Diffusion Drafter
Tencent Inc.
DeepGaze3.5-VL: Modeling Scanpaths via Autoregressive Token Prediction
Jul 2, 2026Susmit Agrawal, Matthias Bethge, Matthias KümmererAutoregressive Token Prediction Generative Models
University of Tübingen · Tübingen AI Center · IMPRS-IS
From Tokens to States: LLMs as a Special Case of World Models and the Continuous Path Beyond
Jun 26, 2026Paul DuboisAutoregressive Token Prediction Latent World Models
DREAM: Dense Retrieval Embeddings via Autoregressive Modeling
Jun 23, 2026Yixuan Tang, Yi YangRetrieval Layer Autoregressive Token Prediction
The Hong Kong University of Science and Technology
Test-Time Training with Next-Token Prediction
Jun 19, 2026Xuan Ouyang, Zefan Cai, Junjie HuTest-Time Training Next-Token Prediction
University of Wisconsin–Madison
DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis
May 31, 2026Shu Long, Yanglei Gan, Xuchuan ZhouSentiment Autoregressive Token Prediction
University of Electronic Science and Technology of China · Southwest Petroleum University · Southwest Minzu University
Neural Scaling Laws for Jet Generation
May 27, 2026Oz Amram, Darius A. Faroughy, Tjarko Gerdes +5Scaling Laws Scaling
1Fermi National Accelerator Laboratory, Batavia, IL 60510, USA · 2NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, USA · Institute for Experimental Physics, Universität Hamburg, Luruper Chaussee 149, 22761 Hamburg, Germany +1
CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models
May 21, 2026Zhi LiuDiffusion-Based Vision-Language-Actions Post-Training
Tianjin University
Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework
May 17, 2026Yumeng Yao, Jingzhi Dong, Haowen Gu +4Point Clouds 3D Foundation Models
Nanjing University of Science and Technology · Hangzhou City University · East China Normal University +1
Second-Order Multi-Level Variance Correction for Modality Competition in Multimodal Models
May 15, 2026Yishun Lu, Wes ArmourMultimodal Alignment Multimodal Model
University of Oxford, Oxford, United Kingdom
PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization
May 7, 2026Adhiraj Banerjee, Vipul AroraAudio Tokenizers Autoregressive Token Prediction
Department of Electrical Engineering Indian Institute of Technology, Kanpur
Perturbation is All You Need for Extrapolating Language Models
May 5, 2026Zetai Cen, Jin Zhu, Xinwei Shen +1Autoregressive Token Prediction Extrapolation
School of Mathematics, University of Bristol · School of Mathematics, University of Birmingham · Department of Statistics, University of Washington +1
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
Mar 6, 2026Kai Luo, Xu Wang, Rui Fan +1Multi-Object Tracking Autonomous Driving
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China · College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410082, China · State Key Laboratory of Intelligent Autonomous Systems, Tongji University, Shanghai 201804, China
Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking
Sep 15, 2025Zirui Zheng, Takashi Isobe, Tong Shen +12Autoregressive Image Generation Layouts
Advanced Micro Devices Inc. · Dalian University of Technology · S-Lab, Nanyang Technological University

Sep 29, 2026, cs.CVJ/K move · Enter open · S save

NesTok: Nested Self-Aligned 1D Tokenizer for Autoregressive Image Generation

Jiawei Zhang, Shuhao Liu, Rong Huang, Yuancheng Li+3

North China Electric Power University · University of Science and Technology of China · Stanford University

Autoregressive Image Generation Visual Tokenizers Autoregressive Token Prediction Tokenizer

Open paper AskPDF arXiv

One-dimensional (1D) variable-length visual tokenizers enable adaptive compression by varying the number of tokens, allowing downstream autoregressive (AR) models to flexibly trade off generation quality against computational cost using a single tokenizer. However, existing approaches based on nested dropout often fail to fully exploit the representational capacity of the tokenizer, resulting in suboptimal performance in both image reconstruction and generation. In this work, we introduce NesTok, a nested self-alignment framework tailored to dynamic visual tokenizers. NesTok introduces cross-length training, which jointly optimizes reconstruction across token lengths while using the full-length sequence to guide shorter counterparts, enabling shorter token sequences to approach the reconstruction quality of full-length sequences. On ImageNet, NesTok improves substantially over standard training and achieves an rFID score of 0.98. On downstream image generation, it achieves the state-of-the-art gFID score of 1.46 on ImageNet 256

\times

256 among existing variable-length autoregressive image generation methods. Code will be available at https://github.com/Jiawei804/NesTok.

Momentum

Related topics

Latest papers 18