cs.CVNov 11, 2026

Keep Images in Sight: Visual Position Re-Anchoring for Long-Context MLLMs

Authors: Juntao Xu, Qichai Zhao, Rui Sun, Zhouyang Jiang, Shiwei Wu, Zhi Li, Kaisheng Ma, Yu Liu

Organizations: Tsinghua University

Abstract

Long-context multimodal intelligence requires revisiting earlier images as conversations and documents accumulate, yet retaining an image does not guarantee reliable access to its details. By varying context length, redistributing a fixed text passage around the image, and changing positions without adding text, we identify the positional relation between images and questions as a direct influence on visual access. To address this gap, we propose Visual Position Re-Anchoring (VPR), a training-free method that shifts cached visual-key rotary phases toward a short-context reference before question processing. It operates directly on the existing KV cache, preserving standard attention computation without extracting attention maps or adding model forward passes. We develop static and continuous compensation schedules, using the static schedule by default. Across ten checkpoints and ten benchmarks, static VPR improves 97 of 100 model--task scores. Further analysis shows greater attention to regions containing answer evidence across six checkpoints, with minimal runtime overhead under fixed rotary frequencies. These findings demonstrate an efficient way to sustain visual access through positional adjustment within the cache.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Adaptive Visual KV Cache Reuse for Efficient Multi-Turn Iterative Multimodal Reasoning

    Jun 25, 2026Mengzhao Wang, Yanli Ji, Wangmeng Zuo +1Efficient Multimodal InferenceKV Caching

  2. VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference

    Aug 25, 2026Lyuke Wang, Zhuo Li, Guangxu ZhuEfficient VLM InferenceLong-Context VLMs

  3. ReToken: Improving Long-Context VLMs with Visual Retrieval Token

    Jul 30, 2026Yao Xiao, Reuben Tan, Zhen Zhu +3Vision-Language ModelsVision-Language Retrieval