cs.CVNov 11, 2026

MemInStream: Memorizing or Merely Recognizing? Rethinking Streaming Video Memory Evaluation

Authors: Zhenqi He, Dunyuan Xu, Yang Bo, Kien T. Pham, Yanghao Wang, Shuhan Li, Xintong Guo, Yuyang He, 

Organizations: HKUST · CUHK · Huawei Research · CityU

Abstract

Streaming video understanding aims to continuously process incoming visual observations, making the retention and retrieval of relevant information increasingly important as the video history grows. Although existing benchmarks include memory-related questions, question-answering accuracy alone does not reliably measure memory capabilities. Incorrect predictions may arise from failures in visual understanding rather than forgetting, whereas correct predictions may rely on linguistic priors or recent observations without requiring historical memory. To address these ambiguities, we introduce MemInStream, a benchmark dedicated to evaluating memory capabilities in streaming video understanding. We develop an evidence-controlled validation protocol that associates each question with temporally localized evidence intervals. The protocol validates evidence solvability and memory necessity by retaining questions that models can answer from the annotated evidence but cannot resolve using recent observations, post-evidence observations, or textual input alone. This helps disentangle memory limitations from understanding failures while reducing memory-independent shortcuts. MemInStream provides a systematic evaluation of memory capabilities through four major categories, covering Episodic Retrieval, State Tracking & Updating, Multi-Evidence Integration, and Contextual Memory across diverse video scenarios. Overall, MemInStream provides a comprehensive and reliable benchmark for systematically evaluating memory capabilities in streaming video understanding.

Figures & tables

Explore similar work

CardsList
  1. A Simple Baseline for Streaming Video Understanding

    Apr 2, 2026Yujiao Shen, Shulin Tian, Jingkang Yang +1Streaming Video UnderstandingStreaming Inference

  2. StreamFlow: Dynamic Memory Flows for Streaming Video Understanding

    Aug 11, 2026Muxin Fu, Yifan Zhang, Wentao Zhang +5Memory-Augmented VLMsStreaming Video Understanding

  3. EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision

    May 29, 2026Rosario Forte, Giuseppe Lando, Antonino FurnariVLM EvaluationStreaming Video Understanding