cs.CVNov 3, 2026

Beyond Token Representations: Explicit Visual Object Grounding for Video Reasoning Segmentation

Authors: Jun Huang, Junyang Zhang, Zijie Yue, Bingkun Wang, Yong Luo, Miaojing Shi

Organizations: College of Electronic and Information Engineering, Tongji University · Shanghai Zhenhua Heavy Industries Co., Ltd. · College of Computer Science, Wuhan University · Shanghai Institute of Intelligent Science and Technology, Tongji University

Abstract

Video Reasoning Segmentation (VRS) aims to infer and segment target objects specified by reasoning queries in videos. Existing methods typically first infer the target and generate a few segmentation tokens for it, which are then fed into a mask decoder for mask prediction. However, such segmentation tokens tend to be both semantically monotonous and spatially ambiguous, making them largely insufficient to represent the target to guide mask prediction. In this paper, we propose an explicit visual object grounding framework for VRS. Specifically, we first propose a spatial-aware prompt generation and refinement scheme, in which we query a multimodal large language model to generate explicit spatial prompts (i.e., bounding box and point set) for the target and then refine them through multi-step dialogue. Furthermore, we introduce a query diversification and alignment module to generate auxiliary queries that describe the same target from different perspectives. We then enforce cross-query consistency between their predicted spatial prompts to alleviate the training bias caused by query limitations. Finally, we select several reliable frames using a prompt-based keyframe selection strategy to support mask decoding and propagation. Extensive experiments on eight datasets demonstrate that our method significantly outperforms state-of-the-art methods. The code is available at https://github.com/viscom-tongji/EVOG.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation

    Oct 18, 2026Junwei Wen, Deshui Miao, Guangming Lu +3Temporal Reasoning in VLMsVideo Object Segmentation

  2. SteerSeg: Attention Steering for Reasoning Video Segmentation

    May 14, 2026Ali Cheraghian, Hamidreza Dastmalchi, Abdelwahed Khamis +3Video Object SegmentationReferring Video Object Segmentation

  3. RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation

    May 8, 2026Junwei Wen, Deshui Miao, Guangming Lu +2Video Object SegmentationVideo Reasoning