cs.CVNov 4, 2026

Sekai: Incremental Event Memory for Generalized Reasoning Video Segmentation

Authors: Zhengtao Liu, Jinrong Xu, Shiu-Hong Kao, Huixin Sun, Yicong Li

Organizations: University of Science and Technology of China · National University of Singapore

Abstract

Online reasoning video segmentation (Online RVS) requires a model to segment query-referred objects as a video streams in, using only the frames observed so far. Existing formulations largely assume a fixed target set throughout the video, overlooking a fundamental property of streaming reasoning: as new events occur, the objects that satisfy a query may change over time. To address this, we propose a new task, Generalized Reasoning Video Segmentation (GRVS), where the target set depends on query-relevant events accumulated over the observed video history. As new evidence accumulates, the target set can evolve from empty to one or multiple objects, requiring a model to continuously maintain target validity while localizing the valid objects currently in view. To evaluate GRVS, we build Isekai, a diagnostic benchmark of 60 videos and 197 queries across 11 temporal query families, with 12,569 scoring points whose target sets are causally annotated over time. We further propose Sekai, an online framework with two complementary components: an Event Memory Reasoner (EMR) that maintains query-relevant events and evolving target states in structured memory, and a Memory-Grounded Decoder (MGD) that identifies valid targets from this memory and localizes them in the current frame. Experiments show that Sekai significantly outperforms existing online RVS methods, achieving 50.33 J&F and improving over the strongest baseline by 3.2 points. Code, example annotations, and evaluation scripts are available at https://github.com/zenastangbi/Sekai.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Online Reasoning Video Object Segmentation

    Apr 13, 2026Jinyuan Liu, Yang Wang, Zeyu Zhao +3Video Object SegmentationReasoning Segmentation

  2. Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation

    Oct 18, 2026Junwei Wen, Deshui Miao, Guangming Lu +3Temporal Reasoning in VLMsVideo Object Segmentation

  3. VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation

    Jun 5, 2026Ming Dai, Sen Yang, Boqiang Duan +4Video Object SegmentationReferring Video Object Segmentation