Vector-Quantized Variational Autoencoder

Also known as VQ-VAE

Overview

A VQ-VAE encodes data into vectors, replaces them with entries from a learned codebook, and trains a decoder to reconstruct the input. The resulting discrete tokens support compression and generative modeling across images, audio, video, motion, and other structured signals, and can also aid interpretation or control. Common approaches include hierarchical or modality-specific tokenizers, improved codebook updates and priors, and continuous pretraining to reduce codebook or dimensional collapse. The main tension is balancing compact, well-used token vocabularies with faithful reconstruction and reliable downstream performance, especially at low bitrates or on unfamiliar data.

Latest papers

CardsList

37 papers3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

  1. Adaptive Residual Quantization for Memory-Efficient Temporal Action Segmentation

    Nov 9, 2026Gerard Donahue, Guven Gergerli, Ayush Gupta +3Temporal Action SegmentationVector-Quantized Variational Autoencoder

  2. Does Pose Encoding Matter? Analyzing Geometric Representations for Discrete Motion Vocabulary Learning

    Nov 8, 2026Amir Kimiyaie, Nahal Mafi, Hamed TabkhiTokenizationMotion Representation Learning

  3. ORBIT: Autoregressive Time Series Generation via Next Frequency-Band Prediction

    Oct 19, 2026Chunyi Hou, Xiangfei Qiu, Hanyin Cheng +5Time Series GenerationFrequency-Domain Time Series Modeling

  4. A Locally Tokenized Generative Model for Robust Time-Series Watermarking

    Aug 20, 2026Dongbin Kim, Geonwoo Shin, Yujin Choi +2Robust WatermarkingVector-Quantized Variational Autoencoder

  5. Atrial Fibrillation Detection with Arbitrary Leads via a Codebook-Based Reconstruction-Classification Framework

    Aug 19, 2026Hongtao Li, Jia Wei, Guoyao Li +6Atrial Fibrillation DetectionElectrocardiogram Classification

  6. Pitch Contour Tokenization using VQ-VAE and Its Application on Korean Traditional Music Analysis

    Aug 11, 2026Seonguk Ju, Seola Cho, Sooin Chung +2Audio Representation LearningVector-Quantized Variational Autoencoder

  7. Codebook Capacity Governs Perceptual Quality Across Resolutions in Hierarchical Discrete Video Compression

    Jul 25, 2026Manikanta Kotthapalli, Banafsheh RekabdarVector-Quantized Variational AutoencoderVector Quantization

  8. VQ-Touch: A Data-Efficient Tactile Generation Framework Across Sensors and Scenarios

    Jul 16, 2026Kailin Lyu, Long Xiao, Jianing Zeng +3Tactile SensingVector-Quantized Variational Autoencoder

  9. Entropy-Coded MS-VQ-VAE with Learned Priors for Ultra-Low Bitrate Video Compression

    Jun 28, 2026Manikanta Kotthapalli, Banafsheh RekabdarLearned Image CompressionVector-Quantized Variational Autoencoder

  10. MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations

    Jun 22, 2026Yuhua Luo, Junsheng Zhang, Mengyin Liu +73D ReconstructionVector-Quantized Variational Autoencoder

  11. Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment

    Jun 11, 2026Xiang Li, Yixuan Zhou, Jingran Xie +2Neural Audio CodecsVector-Quantized Variational Autoencoder

  12. Stochastic weather generators for high-frequency wind vector time series

    Jun 8, 2026Mingshi Cui, Kevin Eng, Justin T. Greene +5Time Series GenerationVector-Quantized Variational Autoencoder

  13. Planning-aligned Token Compression for Long-Context Autonomous Driving

    Jun 5, 2026Zhixuan Liang, Yuxiao Chen, Yurong You +12End-to-End Autonomous DrivingVector-Quantized Variational Autoencoder

  14. BandVQ: Band-Wise Vector-Quantized EEG Foundation Model

    May 24, 2026Jamiyan Sukhbaatar, Satoshi Imamura, Toshihisa TanakaCross-Subject EEG DecodingMasked Autoencoders

  15. Composable Crystals: Controllable Materials Discovery via Concept Learning

    May 14, 2026Nian Liu, Yuwei Zeng, Ryoji Kubo +7Vector-Quantized Variational AutoencoderConcept Learning

  16. ArcVQ-VAE: A Spherical Vector Quantization Framework with ArcCosine Additive Margin

    May 13, 2026Jaeyung Kim, YoungJoon YooVariational AutoencodersImage Generation

  17. LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Image

    Apr 22, 2026Dimitrije Antić, Alvaro Budria, George Paschalidis +23D ReconstructionVector-Quantized Variational Autoencoder

  18. Collapse-Free Prototype Readout Layer for Transformer Encoders

    Apr 4, 2026Giansalvo Cirrincione, Rahul Ranjeev KumarToken CompressionPrototype Learning

  19. Investigating Permutation-Invariant Discrete Representation Learning for Spatially Aligned Images

    Apr 2, 2026Jamie S. J. Stirling, Noura Al-Moubayed, Hubert P. H. ShumVector QuantizationImage Generation

  20. HUydra: Full-Range Lung CT Synthesis via Multiple HU Interval Generative Modelling

    Mar 24, 2026António Cardoso, Pedro Sousa, Tania Pereira +1Medical Image SynthesisComputed Tomography