cs.CVNov 18, 2026

APS: Adaptive Partitioning-Based Frame Selection for Long Video Understanding

Authors: Junxiang Wen, Yinxi Zhao, Jiawei Chen, Jingying Hu, Lijun Wang, Zhen Qin, Shuiguang Deng

Organizations: Zhejiang University · Tsinghua University · University of Science and Technology Beijing · University of Chinese Academy of Sciences

Abstract

Recent advances in large vision-language models (LVLMs) have significantly improved video understanding capabilities, yet processing long videos remains challenging due to the substantial computational cost of extensive visual inputs. Frame selection offers an efficient alternative, but many existing region-based methods rely on fixed or predefined temporal partitions, which may result in suboptimal temporal regions, compromising subsequent budget allocation and frame selection. In this work, we propose Adaptive Partitioning-Based Frame Selection (APS), a training-free framework that adapts temporal partitions to local variations in query-frame relevance. Specifically, APS formulates temporal partitioning as a complexity-regularized fitting problem, jointly optimizing partition boundaries and local models through dynamic programming. The resulting adaptive temporal partitions guide frame budget allocation based on query relevance and temporal extent, enabling informative frame selection under limited visual budgets. Extensive experiments on Video-MME, MLVU, and LongVideoBench demonstrate that APS consistently improves performance across diverse LVLM backbones, achieving the highest average gains over uniform sampling among competing methods. Further analyses validate the effectiveness of adaptive partitioning and demonstrate its generalizability across model scales, similarity encoders, and advanced LVLMs.

Figures & tables

Explore similar work

CardsList
  1. Adaptive Greedy Frame Selection for Long Video Understanding

    Mar 20, 2026Yuning Huang, Xiaoyu Ji, Joseph Huang +2Query-Aware Video Frame SelectionAdaptive Video Frame Sampling

  2. When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understanding

    Aug 4, 2026Ke Li, Jiayu Chen, Maoliang Li +5Efficient VLM InferenceLarge Vision-Language Models

  3. RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding

    Aug 30, 2026Shanqing Xu, Meng Luo, Mengchen Qian +7Large Vision-Language ModelsLong-Video Understanding