cs.CVNov 15, 2026

TIVS: Mapping the Sensitivity Landscape of Visual Token Transformations in MLLMs

Authors: Yikang Qiao, Jianye Zeng, Duan Huang, Ziying Song

Organizations: Central South University · University of Science and Technology of China · Yanshan University

Abstract

Recent efforts to improve the efficiency of multimodal large language models (MLLMs) have explored visual token pruning, quantization, and feature compression. However, the properties that determine whether transformed visual tokens remain usable by a frozen decoder are not well characterized. We introduce TIVS (Token Identity Versus Statistics), a training-free intervention framework that separately studies sequence permutation, token retention, numerical precision, and statistical replacement under controlled conditions. On 192 POPE questions, three token permutations change TinyLLaVA accuracy by only −0.17-0.17 percentage points on average, while changing 4.69% of individual predictions. Diagonal and rank-64 Gaussian replacements decrease accuracy by 25.0 and 28.1 points on the same subset. Further experiments with exact moment constraints, quantization error, token retention, and additional visual tasks reveal how strongly the effects depend on the original feature values and the evaluation setting. These findings provide an empirical basis for assessing visual token transformations in frozen MLLMs and designing more reliable multimodal interfaces.

Figures & tables

Explore similar work

CardsList
  1. TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference

    Jun 25, 2026Tinghao Wang, Yichen Guo, Rui Huang +9Efficient VLM InferenceMultimodal Large Language Models

  2. Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models

    Sep 28, 2026Jingdi lei, Junxian Li, Di Zhang +3Efficient Multimodal InferenceMultimodal Large Language Models

  3. A More Word-like Image Tokenization for MLLMs

    May 18, 2026Hyun Lee, Hyemin Jeong, Yejin Kim +4Multimodal UnderstandingEfficient VLM Inference