cs.CVOct 30, 2026

SIEVE: Preserving Cross-Image Evidence via Depth-Aware Token Compression

Authors: Nitish Shukla, Surgan Jandial, Nhat-Tan Bui, Arun Ross

Organizations: Michigan State University · DatologyAI · Carnegie Mellon University

Abstract

Multi-image vision-language models (VLMs) incur substantial computation when processing long visual sequences. Compressing these sequences must preserve complementary evidence across images and keep tokens from different images separate. Global selection can underrepresent question-relevant images, and pruning in early decoder layers risks removing relevant content before sufficient multimodal processing, yet existing methods spend most of the budget on these early layers. We propose SIEVE, a training-free, single-pass framework that jointly allocates visual tokens across images and decoder depth. SIEVE first pools each image into a compact spatial grid to reduce early processing cost. It then progressively selects representatives using prompt attention under equal per-image quotas and aggregates unselected tokens into similar representatives from the same image. These operations keep every image represented and its tokens separate from other images, while refining the retained representations. A closed-form schedule sets retention counts to target a prescribed average token budget across decoder layers. Across four VLM backbones, SIEVE achieves the highest average at a 6.25% budget on five multi-image benchmarks and four single-image benchmarks, and the highest mean on two video benchmarks. On LLaVA-OneVision-7B, it outperforms the strongest baseline by 9.96 and 9.03 points at 6.25% and 12.5% budgets, and at 12.5% it exceeds the uncompressed model by 2.01 points with 6.6x fewer prefill FLOPs.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs

    Aug 11, 2026Congyang Ou, Ruike Song, Yang Zhou +3Efficient VLM InferenceLarge Vision-Language Models

  2. Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity

    Mar 10, 2026Zhengyao Fang, Pengyuan Lyu, Chengquan Zhang +3Visual Token PruningVisual Token Compression

  3. Multi-Image Visual Token Pruning in Large Visual Language Models

    Aug 27, 2026Rongyang Zhang, Chengqiang Lu, Cong Li +9Efficient VLM InferenceLarge Vision-Language Models