Visual Tokenization

Follow this topic Ask about this topic

Visual tokenization studies how to represent images and video as sequences or sets of units that models can process efficiently. Approaches range from fixed patch grids and discrete or continuous autoencoder codes to semantic, object-centric, hierarchical, and variable-length tokens; video methods also encode motion and reuse information across frames. These representations support image generation, video understanding, multimodal language models, and robotic control, where they can reduce computation while retaining useful visual detail. The main challenge is balancing compactness and reconstruction fidelity with semantic and spatial information that remains useful across tasks and downstream models.

Latest papers

70 papers8 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

MedVTok: A General-Purpose Medical Visual Tokenizer
Oct 26, 2026Chenglong Ma, Yuanfeng Ji, Junzhi Ning +18Image Tokenization Visual Tokenization
Fudan University, China · Shanghai Innovation Institute, China · Stanford University School of Medicine, United States +4
TrackTok: Object-Centric Video Tokenization with Semantically Persistent Tokens
Oct 24, 2026Nhi Pham, Christopher Wewer, Bernt Schiele +2Visual Tokenization Video Object-Centric Learning
Max Planck Institute for Informatics, Saarland Informatics Campus, Germany · Saarland University, Saarland Informatics Campus, Germany
Query-adaptive Recurrent Tokenization
Oct 16, 2026Bavesh Balaji, Pranav Gopinath, Pouya Sadeghi +1Visual Tokenization Efficient VLM Inference
Critical ML Lab, University of Waterloo
Thinking from Above: Efficient Spatial Reasoning with Bird's-Eye-View Visual Tokens
Oct 15, 2026Faizan Siddiqui, Nirav Savaliya, Sahith Reddy Chada +1Visual Spatial Reasoning Bird's-Eye-View Representation
Honda Research Institute USA
VITAL: From Visual Motion to Precise Action
Oct 14, 2026Ao Liang, Youquan Liu, Mingxiao Li +7Visuomotor Policy Learning Visual Tokenization
Horizon Robotics · Shanghai Innovation Institute · Fudan University +2
Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning
Oct 13, 2026Tianhao Fu, Austin Wang, Charles Chen +5Visual Token Compression Efficient VLM Inference
University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · McGill University, Montreal, QC, Canada +2
MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement
Oct 9, 2026Chenge Li, Yefei He, Xinyu Liu +3Visual Tokenization Visual Token Compression
Zhejiang University · The Hong Kong University of Science and Technology · University of Adelaide
QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation
Oct 7, 2026Yucheng Mao, Zeyuan Chen, Xiaojun Shan +4Visual Tokenization Autoregressive Image Generation
University of California, San Diego
SemanTok: Predictable Semantic Tokens for Efficient Autoregressive Video Generation
Sep 30, 2026Mikhail Dereviannykh, Vikram Voleti, Simon Donne +3Visual Tokenization Autoregressive Video Generation
Stability AI · Karlsruhe Institut für Technologie
Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Level
Sep 20, 2026Alexandra Schild, Gerard de MeloVLM Robustness VLM Evaluation
HPI/University of Potsdam Potsdam, Germany
LoopVAE: Recurrent Depth Across Scales for Visual Tokenization
Sep 10, 2026Zhiying LuVariational Autoencoders Autoencoders
University of Science and Technology of China
Variable-Granularity Tokenization for High-Resolution Object Detection
Aug 27, 2026Khayrul IslamVisual Tokenization Efficient Object Detection
Lawrence Livermore National Laboratory 7000 East Ave, Livermore, CA 94550
VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision
Aug 25, 2026Zhehan Kan, Yubo Zhu, Xinghua Jiang +10Vision-Language Pretraining Fine-Grained Visual Perception
Tsinghua University · Tencent Youtu Lab · Nanjing University
UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
Aug 25, 2026Zhehan Kan, Xinghua Jiang, Yubo Zhu +10Vision-Language Models Unified Multimodal Models
Tsinghua University · Tencent Youtu Lab · Nanjing University +1
AffineTok: Semantic Affine Consistency for Diffusion-Friendly Visual Tokenizer
Aug 24, 2026Junqiu Yu, Pandeng Li, Yikai Wang +11Visual Tokenization Diffusion-Based Image Generation
Fudan University · Alibaba Token Hub, Alibaba Group
HeatTok: Enhancing Remote Sensing Image Understanding via Thermodiffusion-based Tokenization
Aug 23, 2026Yingying Yan, Jiaqi Tang, Wei Wei +6Visual Tokenization Remote Sensing Image Understanding
Hong Kong University of Science and Technology Hong Kong, China
FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning
Aug 17, 2026Tristan Gottwald, Maximilian Schier, Melanie Schaller +1Representation Learning for RL Event-Based Vision
Institute of Information Processing, L3S, Leibniz University Hannover, Germany
Persistent Object Narratives for Token-Efficient Video Language Models
Aug 5, 2026Junzhe Chen, Siyuan Meng, Xiaojie GuoVisual Tokenization Video-Language Models
Tianjin University
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Aug 2, 2026Guiqiu Liao, Matjaz Jogan, Daniel A. HashimotoImage Tokenization Healthcare
GRASP Laboratory, University of Pennsylvania · PCASO Laboratory, Department of Surgery, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania
One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting
Jul 30, 2026Rui Tang, Wentao Yang, Peirong Zhang +4Visual Tokenization Scene Text Recognition
South China University of Technology · Guangzhou, China · HiThink Research +1
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
Jul 28, 2026Weiming Zhuang, Jiabo Huang, Jingtao Li +4Unified Multimodal Models Visual Tokenization
Sony AI
Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
Jul 27, 2026Senqiao Yang, Kaichen Zhang, Zhaoyang Jia +20Vision-Language Models Efficient VLM Inference
Microsoft Mage Team
ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment
Jul 22, 2026Stefanos Gkikas, Yu Fang, Christian Arzate Cruz +2Visual Tokenization
Honda Research Institute Japan Wako City, Japan · BioSIS (Biosensing & Intelligent Systems) Lab Centre for Intelligent Computing and Systems University of Canberra Canberra, Australia
VQ-Transplant: Efficient VQ-Module Integration for Pre-trained Visual Tokenizers
Jul 21, 2026Xianghong Fang, Yuan Yuan, Dehan Kong +1Visual Tokenization Image Reconstruction
University of Toronto · 2Boston College · 3Vijil
Subtoken Vision Transformer for Fine-grained Recognition
Jul 10, 2026Jie Zhu, Ivy Zhang, Minchul Kim +1Vision Transformer Visual Tokenization
Michigan State University · Cranbrook Kingswood School · University of North Carolina at Chapel Hill
Native Video-Action Pretraining for Generalizable Robot Control
Jul 9, 2026Qihang Zhang, Lin Li, Luyao Zhang +26Robotic Control Language-Conditioned Robot Manipulation
VocaDet: Sample-Driven Open-Vocabulary Object Detection and Segmentation via Visual Tokenization and Vector Database Retrieval
Jul 9, 2026ZhiXin SunOpen-Vocabulary Object Detection Visual Tokenization
PowerChina Zhongnan Engineering Corporation Limited
TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization
Jun 16, 2026Weiliang Chen, Yuanhui Huang, Xuebo Wang +1Visual Tokenization Video Token Compression
Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China · Department of Automation, Tsinghua University, Beijing, 100084, China · Kuaishou Technology, Beijing, China
Explainable Task-Oriented Token Communication for AI-Native 6G Networks
Jun 12, 2026Feibo Jiang, Lei Mao, Li Dong +36G Networks Visual Tokenization
RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Jun 11, 2026Junke Wang, Qihang Zhang, Shuai Yang +5Latent Action Learning World Models for Robotics
Institute of Trustworthy Embodied AI, Fudan University · 2Robbyant, Ant Group · 3Hongkong University of Science and Technology

Oct 26, 2026J/K move · Enter open · S save

MedVTok: A General-Purpose Medical Visual Tokenizer

Chenglong Ma, Yuanfeng Ji, Junzhi Ning, Jiyao Liu+17

Fudan University, China · Shanghai Innovation Institute, China · Stanford University School of Medicine, United States+4

Image Tokenization Visual Tokenization Medical Image Representation Learning Medical Imaging Foundation Models Multimodal Pretraining

Open paper AskPDF arXiv

Multimodal medical AI requires a common visual interface for encoding heterogeneous imaging evidence for patient condition modeling. However, existing medical visual tokenizers are often tied to a specific dimension and imaging modality, forcing multimodal systems to rely on fragmented and poorly reusable representations. We introduce MedVTok, a general-purpose visual tokenizer that maps 2D images and 3D volumes from diverse imaging modalities into a unified token space. Building such a tokenizer poses two key challenges: preserving anatomical structure across dimensions and capturing clinical semantics across modalities. To this end, MedVTok introduces (i) slice differential regularization, which explicitly models inter-slice anatomical coherence missing in previous per-pixel/per-voxel supervision, and (ii) multi-expert representation alignment, which integrates knowledge from multiple medical expert encoders while retaining modality-specific complementary knowledge. Trained on 28M images and 70K volumes across 9 imaging modalities, MedVTok supports classification, retrieval, segmentation, synthesis, visual question answering, and medical report generation through a single tokenizer. Across 24 benchmarks, MedVTok achieves state-of-the-art performance, demonstrating a scalable token interface that unifies medical image dimension, modality, and task usage.