cs.AINov 9, 2026

Disentangling Multimodal Rotary Positional Embedding in Vision-Language Models

Authors: Zhaoyi Liu, Haichuan Zhang, Xiaochuan Gong, Yancheng Zhu, Bowei Tian, Wanghao Ye, Yufan Zhang, Ang Li

Organizations: University of Maryland, College Park · University of Utah

Abstract

Vision-language models widely adopt Multimodal Rotary Position Embedding (MRoPE), an extension of Rotary Position Embedding (RoPE) that incorporates temporal, height, and width axes to encode the spatial structure of visual tokens. However, MRoPE also assigns height and width coordinates to text tokens that lack inherent spatial structure, introducing spatial positional effects into text-vision attention. Existing MRoPE variants explore different frequency allocations and coordinate layouts, with their designs suggesting different intuitions about these cross-modal effects: they may provide useful spatial references for associating language with visual regions, or introduce unnecessary positional preferences that hinder text-vision interactions. In this work, we disentangle intra-visual spatial structure from cross-modal spatial effects to investigate this question. We identify a constraint of shared coordinate layouts that prevents eliminating cross-modal spatial effects while preserving intra-visual spatial encoding, and develop controlled experiments to isolate their respective contributions. Our results across multiple model scales, MRoPE variants, and benchmarks show that cross-modal spatial effects provide little additional benefit as spatial references and can even impair model performance, while intra-visual spatial encoding remains useful for fine-grained grounding.

Figures & tables

Explore similar work

CardsList
  1. Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encoding

    Mar 11, 2026Lin Chen, Bolin Ni, Qi Yang +5Multimodal Large Language ModelsLong-Context Modeling

  2. Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability

    Apr 19, 2026Lijie ZhouCross-Modal LearningVLM Robustness

  3. Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs

    Apr 17, 2026Sai Srinivas Kancheti, Aditya Sanjiv Kanade, Vineeth N. Balasubramanian +1Multimodal Model EvaluationVisual Spatial Reasoning