Image Tokenization

Follow this topic Ask about this topic

Image tokenization turns pixels into visual units that models can store, generate, or pass to language and vision systems with less computation. Common approaches include vector and residual quantization, continuous latent encoders, and semantic alignment with pretrained vision models; tokens may be arranged as 2D grids, ordered 1D sequences, or adaptive-length representations. Methods are evaluated on reconstruction and class-conditional generation, often using ImageNet, as well as downstream tasks such as visual question answering and domain-specific medical or scientific imaging. A central tension is that pixel-level reconstruction quality does not always predict how well a tokenizer supports generation or understanding, and performance can depend on the generator and sampler paired with it.

Latest papers

57 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.0% of all new papers.

MedVTok: A General-Purpose Medical Visual Tokenizer
Oct 26, 2026Chenglong Ma, Yuanfeng Ji, Junzhi Ning +18Image Tokenization Visual Tokenization
Fudan University, China · Shanghai Innovation Institute, China · Stanford University School of Medicine, United States +4
Transporting Quantiles to Codebook: Scalable Vector Quantization without Codebook Collapse
Oct 26, 2026Yuwei Zeng, Zekun ShiVector Quantization Vector-Quantized Variational Autoencoder
National University of Singapore · SEA AI Lab
Patch-Derived or Register Tokens? Comparing Variable-Length Image Tokenizers
Oct 24, 2026Philipp Schenkel, Tobias Schürmann, Omer Sahin TasImage Tokenization Visual Token Compression
FZI Research Center for Information Technology · Karlsruhe Institute of Technology
RoAR-VAE: Improving Latent Diffusability through Robust Prediction
Oct 19, 2026Yang Yue, Fangyun Wei, Yitong Wang +4Image Tokenization Variational Autoencoders
Tsinghua University · Microsoft Research
FeaTok: Feature-Loss Tokenization for Image Generation
Oct 15, 2026Zhang Xiaofeng, Samuel Lavoie, Nicolas Beltran-Velez +6Image Tokenization Image Generation
Mila - Québec AI Institute · Université de Montréal · FAIR at Meta +4
Generative Representation Learning for Scalable Latent Tokenization
Oct 14, 2026Alan Baade, Kyle Sargent, Vincent Sitzmann +4Representation Learning Image Tokenization
Stanford University · Massachusetts Institute of Technology · University of Michigan
Image Tokenizers Should Be Trained Between Images
Oct 12, 2026Ole Beißwenger, Jan-Niklas Dihlmann, Hendrik LenschImage Tokenization Latent Diffusion Models
University of Tübingen
Making Language the Low-Rate Code of Vision: Successive Refinement from Words to Pixels
Oct 12, 2026Ruilin Gong, Hao GongT2I Generation Image Tokenization
Henan University of Technology · University of Chinese Academy of Sciences
Semantically Steerable Image Tokenization
Oct 11, 2026Danilo de Goede, Mohammad Mahdi Derakhshani, Yuki M. Asano +1Image Tokenization Visual Token Compression
QUVA Lab, University of Amsterdam · VIS Lab, University of Amsterdam · Fundamental AI Lab, University of Technology Nuremberg
BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image Diffusion
Oct 8, 2026Ozgur Kara, Yujia Chen, Daniel Watson +4Image Tokenization Diffusion Model Inference Acceleration
University of Illinois Urbana-Champaign · Google
Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding
Sep 25, 2026Enzhi Zhang, Du Wu, Rui Zhong +18Masked Autoencoders Masked Image Modeling
Hokkaido University, Japan · RIKEN Center for Computational Science (R-CCS), Japan · Oak Ridge National Laboratory, USA +4
What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization
Sep 21, 2026Niklas Bubeck, Yundi Zhang, Vasiliki Sideri-Lampretsa +4Image Tokenization Medical Image Synthesis
School of Computation, Information and Technology, Technical University of Munich, Germany · Munich Center for Machine Learning, Technical University of Munich, Germany · School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany +3
Studying Image Tokenizers as Visual Languages in Unified Multimodal Models
Sep 8, 2026Siting Li, Zhengyang Wang, Simon Shaolei Du +2Image Tokenization Unified Multimodal Models
University of Washington · Amazon FAR
Tokenizer-Generator Coupling in Medical Image Generation
Aug 7, 2026Liam ChalcroftImage Tokenization Diffusion Sampling
University College London London, United Kingdom
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Aug 2, 2026Guiqiu Liao, Matjaz Jogan, Daniel A. HashimotoImage Tokenization Healthcare
GRASP Laboratory, University of Pennsylvania · PCASO Laboratory, Department of Surgery, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania
Pixels for Programs? A Cross-Provider Case Study of Input-Token Accounting for Source Code as Text and Images
Jul 23, 2026Ronak BhalgamiImage Tokenization Token Efficiency
Test-Time Registers as Global Priors for Tokenized Image Generation
Jul 18, 2026Cheng-Yao Hong, Yifan Wang, Yuewei Lin +1Image Tokenization Vision Transformer
Stony Brook University · Brookhaven National Laboratory
SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference
Jul 1, 2026Kyan Mahajan, Mohammad SaqlainImage Tokenization Efficient ViTs
Indian Institute of Information Technology, Allahabad, Uttar Pradesh, India.
From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion
Jun 10, 2026Yuchen Xian, Yunqiu Xu, Yang He +1Multi-Source Image Fusion Image Tokenization
ReLER, The State Key Lab of Brain Machine Intelligence, Zhejiang University · College of Artificial Intelligence, Zhejiang University · National University of Singapore +2
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Jun 9, 2026Yitong Chen, Zijie Diao, Junke Wang +5Visual Representation Learning Image Tokenization
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · University of Maryland, College Park
AdaTok: Self-Budgeting Image Tokenization with Quality-Preserving Dynamic Tokens
Jun 5, 2026Xiaocheng Lu, Yuxi Chen, Jie Zhang +5Image Tokenization Visual Tokenization
The Hong Kong University of Science and Technology · The Hong Kong Polytechnic University
Balancing Image Compression and Generation with Bootstrapped Tokenization
Jun 4, 2026Haozhe Chi, Jinghan Li, Hao Jiang +4Learned Image Compression Image Tokenization
Peking University · Central Media Technology Institute, Huawei
ChannelTok: Efficient Flexible-Length Vision Tokenization
Jun 3, 2026Sukriti Paul, Arpit Bansal, Tom GoldsteinImage Tokenization Visual Tokenization
University of Maryland, College Park
Diffusing in the Right Space: A Systematic Study of Latent Diffusability
Jun 2, 2026Tianxiong Zhong, Xingye Tian, Xuebo Wang +2Latent Diffusion Models Image Tokenization
Kling Team, Kuaishou Technology
Unified Driving Tokens: Representation- and Geometry-Guided Discrete Tokenizer for Driving World Models and Planning
Jun 1, 2026Ziyang Yao, Zeyu Zhu, YunCheng Jiang +2Image Tokenization World Models
Peking University · Xiaomi EV
Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering
Jun 1, 2026Dongxing Mao, Jinpeng Wang, Jiahao Tang +6Image Tokenization Visual Tokenization
Central South University · University of Oxford · Microsoft Research
HiTokSR: A Coarse-to-Fine Tokenizer with Hierarchical Codebooks for High-Fidelity Real-World Image Super-Resolution
May 31, 2026Mingxi LiImage Tokenization Vector Quantization
MergeTok: Unified Continuous and Discrete Visual Tokenization via Token Merging
May 29, 2026Luyuan Zhang, Siyuan Li, Zedong Wang +7Variational Autoencoders Token Merging
1Tsinghua University · 2Westlake University · 3Zhejiang University +3
PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding
May 28, 2026Selim Kuzucu, Alessio Tonioni, Vasile Lup +3Image Tokenization Efficient VLM Inference
Max Planck Institute for Informatics, SIC · Google · Technical University of Munich
Structure over Pixels: Learning Variable-Length Visual Programs
May 26, 2026Piotr Wyrwiński, Kacper Dobek, Krzysztof KrawiecVisual Representation Learning Image Tokenization
Institute of Computing Science Poznan University of Technology, Poznan, Poland

Oct 26, 2026J/K move · Enter open · S save

MedVTok: A General-Purpose Medical Visual Tokenizer

Chenglong Ma, Yuanfeng Ji, Junzhi Ning, Jiyao Liu+17

Fudan University, China · Shanghai Innovation Institute, China · Stanford University School of Medicine, United States+4

Image Tokenization Visual Tokenization Medical Image Representation Learning Medical Imaging Foundation Models Multimodal Pretraining

Open paper AskPDF arXiv

Multimodal medical AI requires a common visual interface for encoding heterogeneous imaging evidence for patient condition modeling. However, existing medical visual tokenizers are often tied to a specific dimension and imaging modality, forcing multimodal systems to rely on fragmented and poorly reusable representations. We introduce MedVTok, a general-purpose visual tokenizer that maps 2D images and 3D volumes from diverse imaging modalities into a unified token space. Building such a tokenizer poses two key challenges: preserving anatomical structure across dimensions and capturing clinical semantics across modalities. To this end, MedVTok introduces (i) slice differential regularization, which explicitly models inter-slice anatomical coherence missing in previous per-pixel/per-voxel supervision, and (ii) multi-expert representation alignment, which integrates knowledge from multiple medical expert encoders while retaining modality-specific complementary knowledge. Trained on 28M images and 70K volumes across 9 imaging modalities, MedVTok supports classification, retrieval, segmentation, synthesis, visual question answering, and medical report generation through a single tokenizer. Across 24 benchmarks, MedVTok achieves state-of-the-art performance, demonstrating a scalable token interface that unifies medical image dimension, modality, and task usage.