Vector-Quantized Variational Autoencoder

Also known as VQ-VAE

A VQ-VAE encodes data into vectors, replaces them with entries from a learned codebook, and trains a decoder to reconstruct the input. The resulting discrete tokens support compression and generative modeling across images, audio, video, motion, and other structured signals, and can also aid interpretation or control. Common approaches include hierarchical or modality-specific tokenizers, improved codebook updates and priors, and continuous pretraining to reduce codebook or dimensional collapse. The main tension is balancing compact, well-used token vocabularies with faithful reconstruction and reliable downstream performance, especially at low bitrates or on unfamiliar data.

Latest papers

34 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

ORBIT: Autoregressive Time Series Generation via Next Frequency-Band Prediction
Oct 19, 2026Chunyi Hou, Xiangfei Qiu, Hanyin Cheng +5Time Series Generation Frequency-Domain Time Series Modeling
School of Data Science and Engineering, East China Normal University, Shanghai, China · Chang’an University, Xi’an, China
A Locally Tokenized Generative Model for Robust Time-Series Watermarking
Aug 20, 2026Dongbin Kim, Geonwoo Shin, Yujin Choi +2Robust Watermarking Vector-Quantized Variational Autoencoder
Seoul National University · Nanyang Technological University
Atrial Fibrillation Detection with Arbitrary Leads via a Codebook-Based Reconstruction-Classification Framework
Aug 19, 2026Hongtao Li, Jia Wei, Guoyao Li +6Atrial Fibrillation Detection Electrocardiogram Classification
Information Center, The People’s Hospital of Baoan Shenzhen, Shenzhen, 518101, Guangdong, China · School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, Guangdong, China · Department of Cardiology, The People’s Hospital of Baoan Shenzhen, Shenzhen, 518101, Guangdong, China +1
Pitch Contour Tokenization using VQ-VAE and Its Application on Korean Traditional Music Analysis
Aug 11, 2026Seonguk Ju, Seola Cho, Sooin Chung +2Audio Representation Learning Vector-Quantized Variational Autoencoder
Music & Arts Learning (MALer) Lab, Sogang University, South Korea · Graduate School of Culture Technology, KAIST, South Korea
FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latents
Jul 31, 2026Nitish Nagesh, Mahdi Bagheri, Amir M. RahmaniAlgorithmic Fairness Diffusion Sampling
University of California, Irvine
Where Quality Breaks in Compressed Short-Text Generation: Staged Bottleneck Localization
Jul 27, 2026Alexey Gavrilov, Alan-Barsag Gazzaev, Sergey MuravyovVector-Quantized Variational Autoencoder Diffusion Language Model Inference
ITMO University Saint-Petersburg, Russia
Codebook Capacity Governs Perceptual Quality Across Resolutions in Hierarchical Discrete Video Compression
Jul 25, 2026Manikanta Kotthapalli, Banafsheh RekabdarVector-Quantized Variational Autoencoder Vector Quantization
Department of Computer Science Portland State University Portland, OR 97201
VQ-Touch: A Data-Efficient Tactile Generation Framework Across Sensors and Scenarios
Jul 16, 2026Kailin Lyu, Long Xiao, Jianing Zeng +3Tactile Sensing Vector-Quantized Variational Autoencoder
Institute of Automation, Chinese Academy of Sciences
Entropy-Coded MS-VQ-VAE with Learned Priors for Ultra-Low Bitrate Video Compression
Jun 28, 2026Manikanta Kotthapalli, Banafsheh RekabdarLearned Image Compression Vector-Quantized Variational Autoencoder
Portland State University Department of Computer Science Portland, OR, USA
Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling
Jun 25, 2026Yiwen Yan, Wanning He, Yu-Wing TaiSynthetic Data Generation Human Motion Generation
Dartmouth College
MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations
Jun 22, 2026Yuhua Luo, Junsheng Zhang, Mengyin Liu +73D Reconstruction Vector-Quantized Variational Autoencoder
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, School of Informatics, Xiamen University · National Institute for Data Science in Health and Medicine, Xiamen University +1
Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment
Jun 11, 2026Xiang Li, Yixuan Zhou, Jingran Xie +2Neural Audio Codecs Vector-Quantized Variational Autoencoder
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
Using Seismic Statistical Features and VQ-VAE to Improve Spatiotemporal Seismicity Predictability
Jun 8, 2026Wei Quan, Denise GorseVector-Quantized Variational Autoencoder Seismology
Department of Computer Science University College London
Stochastic weather generators for high-frequency wind vector time series
Jun 8, 2026Mingshi Cui, Kevin Eng, Justin T. Greene +5Time Series Generation Vector-Quantized Variational Autoencoder
Department of Statistics, Rutgers University, Piscataway, NJ, USA
Planning-aligned Token Compression for Long-Context Autonomous Driving
Jun 5, 2026Zhixuan Liang, Yuxiao Chen, Yurong You +12End-to-End Autonomous Driving Vector-Quantized Variational Autoencoder
NVIDIA Research · School of Computing and Data Science, The University of Hong Kong
BandVQ: Band-Wise Vector-Quantized EEG Foundation Model
May 24, 2026Jamiyan Sukhbaatar, Satoshi Imamura, Toshihisa TanakaCross-Subject EEG Decoding Masked Autoencoders
National University of Mongolia, Ulaanbaatar, Mongolia · Tokyo University of Agriculture and Technology, Tokyo, Japan
Composable Crystals: Controllable Materials Discovery via Concept Learning
May 14, 2026Nian Liu, Yuwei Zeng, Ryoji Kubo +7Vector-Quantized Variational Autoencoder Concept Learning
National University of Singapore · Loyola Marymount University
SimPersona: Learning Discrete Buyer Personas from Raw Clickstreams for Grounded E-Commerce Agents
May 14, 2026Zahra Zanjani Foumani, Alberto Castelo, Shuang Xie +3LLM Personalization Vector-Quantized Variational Autoencoder
Shopify · Bellevue, Washington, USA
ArcVQ-VAE: A Spherical Vector Quantization Framework with ArcCosine Additive Margin
May 13, 2026Jaeyung Kim, YoungJoon YooVariational Autoencoders Image Generation
Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea · SNUAILAB, Seoul, Republic of Korea
BeeVe: Unsupervised Acoustic State Discovery in Honey Bee Buzzing
May 8, 2026Hamze Hammami, Nidhal AbdulazizAudio Representation Learning Unsupervised Clustering
Continuous First, Discrete Later: VQ-VAEs Without Dimensional Collapse
May 7, 2026Xinyu Zhao, Nikita Karagodin, Hamed Hassani +3Variational Autoencoders Rate-Distortion Theory
MIT
VQ-SAD: Vector Quantized Structure Aware Diffusion For Molecule Generation
May 1, 2026Farshad Noravesh, Reza Haffari, Layki Soon +1Latent Diffusion Models Vector-Quantized Variational Autoencoder
LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Image
Apr 22, 2026Dimitrije Antić, Alvaro Budria, George Paschalidis +23D Reconstruction Vector-Quantized Variational Autoencoder
University of Amsterdam, The Netherlands · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Aristotle University of Thessaloniki, Greece
Modality-Aware and Anatomical Vector-Quantized Autoencoding for Multimodal Brain MRI
Apr 6, 2026Mingjie Li, Edward Kim, Yue Zhao +2Vector-Quantized Variational Autoencoder Medical Image Representation Learning
Stanford University
Collapse-Free Prototype Readout Layer for Transformer Encoders
Apr 4, 2026Giansalvo Cirrincione, Rahul Ranjeev KumarToken Compression Prototype Learning
Laboratory LTI, Université de Picardie Jules Verne, Amiens, France · Institute of Energy and Resources, Charles Darwin University, Darwin, NT, Australia
Investigating Permutation-Invariant Discrete Representation Learning for Spatially Aligned Images
Apr 2, 2026Jamie S. J. Stirling, Noura Al-Moubayed, Hubert P. H. ShumVector Quantization Image Generation
Durham University, United Kingdom
HUydra: Full-Range Lung CT Synthesis via Multiple HU Interval Generative Modelling
Mar 24, 2026António Cardoso, Pedro Sousa, Tania Pereira +1Medical Image Synthesis Computed Tomography
INESC TEC, Portugal · Faculty of Engineering, University of Porto, Portugal · Faculty of Sciences, University of Porto, Portugal
Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks
Mar 10, 2026Honghui Wang, Zhi Jing, Jicong Ao +4Long-Horizon Robotic Manipulation Robot Manipulation Benchmarks
Tsinghua University · Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom
SOM-VQ: Topology-Aware Tokenization for Interactive Generative Models
Feb 24, 2026Alessandro Londei, Denise Lanzieri, Matteo BenatiVector-Quantized Variational Autoencoder Topological Representation Learning
Sony Computer Science Laboratories - Rome. Joint Initiative CREF-SONY, Centro Ricerche Enrico Fermi. Via Panisperna 89/A, 00184, Rome, Italy · Department of Computer, Automatic and Management Engineering. Sapienza University, Via Ariosto 25, Rome, Italy
PCA-VAE: Differentiable Subspace Quantization without Codebook Collapse
Feb 21, 2026Hao Lu, Onur C. Koyun, Yongxin Guo +3Vector Quantization Vector-Quantized Variational Autoencoder
Wake Forest University School of Medicine, Winston-Salem, NC, USA

Oct 19, 2026J/K move · Enter open · S save

ORBIT: Autoregressive Time Series Generation via Next Frequency-Band Prediction

Chunyi Hou, Xiangfei Qiu, Hanyin Cheng, Chenchen Sun+4

School of Data Science and Engineering, East China Normal University, Shanghai, China · Chang’an University, Xi’an, China

Time Series Generation Frequency-Domain Time Series Modeling Autoregressive Transformer Time Series Representation Learning Vector-Quantized Variational Autoencoder

Open paper AskPDF arXiv

Deep generative models provide an effective modern solution to alleviating the problem of data scarcity by synthesizing realistic time series. However, despite the considerable progress achieved by existing deep time series generators, two key challenges remain: (1) learning hierarchical representations for heterogeneous temporal patterns; and (2) coordinating the generation process of heterogeneous temporal dynamics. To address these key bottlenecks, in this work, we propose ORBIT, a unified deep generative framework for time series. Specifically, we develop two key components: the Frequency-Hierarchical VQ-VAE (FH-VQ-VAE) and the Frequency-Band Autoregressive Transformer (FBAR). The former learns hierarchical representations by decomposing latent temporal representations into different frequency bands and applying residual quantization with increasing token budgets. The latter establishes a novel low-to-high frequency autoregressive direction, yielding a time series generation process that first constructs global temporal structures and then incrementally incorporates fine-grained temporal details. These compact designs not only disentangle heterogeneous temporal patterns for more effective representation learning but also mitigate potential interference among different temporal dynamics during generation. Extensive experiments on multiple challenging real-world benchmarks demonstrate that ORBIT achieves state-of-the-art overall performance while exhibiting superior inference efficiency compared with advanced baselines.