Visual Tokenization

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

A More Word-like Image Tokenization for MLLMs
May 18, 2026Hyun Lee, Hyemin Jeong, Yejin Kim +4Image Tokenization Efficient VLM Inference
Seoul National University · Ewha Womans University
Unified Pix Token And Word Token Generative Language Model
May 13, 2026Haun Leung, ZiNan WangImage Tokenization Self-Supervised Pre-Training
Spectral Vision Transformer for Efficient Tokenization with Limited Data
May 12, 2026Alexandra G. Roberts, Maneesh John, Jinwei Zhang +10Medical Imaging Image Tokenization
Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenization
May 11, 2026Xuanyu Zhu, Yan Bai, Yang Shi +4Autoencoders Visual Tokenization
Peking University · Meituan Inc · Tsinghua University +1
Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs
May 10, 2026Yigui Feng, Qinglin Wang, Yang Liu +1Efficient VLM Inference Visual Tokenization
The College of Computer Science, National University of Defense Technology, Changsha, Hunan, China · College of Computer Science, National University of Defense Technology, Deya Road 109, Changsha, Hunan 410073, China · The Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, Guangdong, China +1
Learning Discrete Autoregressive Priors with Wasserstein Gradient Flow
May 7, 2026Bowen Zheng, Yihong Luo, Tianyang HuImage Tokenization Visual Tokenization
The Chinese University of Hong Kong, Shenzhen · Hong Kong University of Science and Technology
MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonality
May 7, 2026Panqi Yang, Haodong Jing, Jiahao Chao +5Visual Representation Learning Image Tokenization
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence,National Engineering Research Center of Visual Information and Applications,and Institute of Artificial Intelligence and Robotics, Xi’an Jiao Tong University · Xiaohongshu Inc.
Visual Text Compression as Measure Transport
May 6, 2026Lv Tang, Tianyi Zheng, Yang Liu +2Efficient VLM Inference Visual Tokenization
University of Alberta1 · vivo Mobile Communication Co., Ltd2 · Tsinghua University3
ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parameters
May 6, 2026Philippe Hansen-Estruch, Jiahui Chen, Vivek Ramanujan +9Image Tokenization Autoencoders
University of Texas, Austin · University of Washington · 3Stanford University +3
End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer
May 1, 2026Wenda Chu, Bingliang Zhang, Jiaqi Han +4Image Tokenization Visual Tokenization
ByteDance Seed · California Institute of Technology · Stanford University
VibeToken: Scaling 1D Image Tokenizers and Autoregressive Models for Dynamic Resolution Generations
Apr 27, 2026Maitreya Patel, Jingtao Li, Weiming Zhuang +2Image Tokenization Visual Tokenization
Arizona State University · SonyAI · 3Work partially done while at SonyAI. Maitreya is now at Adobe.
COMiT: Learning Structured Visual Tokens through Sequential Communication
Feb 24, 2026Aram Davtyan, Yusuf Sahin, Yasaman Haghighi +4Image Tokenization Visual Tokenization
Computer Vision Group, University of Bern, Switzerland · VITA Lab, EPFL, Switzerland
Spiking Patches: Asynchronous, Sparse, and Efficient Tokens for Event Cameras
Oct 30, 2025Christoffer Koo Øhrstrøm, Ronja Güldenring, Lazaros NalpantidisEvent-Based Vision Visual Tokenization
Technical University of Denmark
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Date pendingGuiqiu Liao, Matjaz Jogan, Daniel A. HashimotoImage Tokenization Healthcare
GRASP Laboratory, University of Pennsylvania · PCASO Laboratory, Department of Surgery, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania

May 18, 2026, cs.CVJ/K move · Enter open · S save

A More Word-like Image Tokenization for MLLMs

Hyun Lee, Hyemin Jeong, Yejin Kim, Hyungwook Choi+3

Seoul National University · Ewha Womans University

Image Tokenization Efficient VLM Inference Visual Tokenization Multimodal Large Language Models Multimodal Foundation Models Visual Token Compression+1

Open paper AskPDF arXiv

Modern multimodal large language models (MLLMs) typically keep the language model fixed and train a visual projector that maps the pixels into a sequence of tokens in its embedding space, so that images can be presented in essentially the same form as text. However, the language model has been optimized to operate on discrete, semantically meaningful tokens, while prevailing visual projectors transform an image into a long stream of continuous and highly correlated embeddings. This causes the visual tokens to behave differently from the word-like units that LLMs are originally trained to understand. We propose a novel Disentangled Visual Tokenization (DiVT) that clusters patch embeddings into coherent semantic units, so each token corresponds to a distinct visual concept instead of a rigid grid cell. DiVT further adapts its token budget to image complexity, providing an explicit accuracy-compute trade-off modifying neither the vision encoder nor the language model. Across diverse multimodal benchmarks, DiVT matches or surpasses baselines with significantly fewer visual tokens, demonstrating robustness under limited token budgets, significantly reducing memory cost and latency while making visual inputs more compatible with LLMs. Our code is available at https://github.com/snuviplab/DiVT.

Momentum

Related topics

Latest papers 44