Memory-Augmented Video Understanding

Overview

Memory-augmented video understanding studies how models can retain useful evidence from long videos or live streams when their context and compute are limited. Approaches include bounded frame caches and semantic compression, hierarchical event- or entity-based memories, and retrieval systems that can revisit relevant clips; some methods also update recurrent states or model weights over time. These techniques support tasks such as streaming question answering, long-video and egocentric reasoning, and tracking objects or events across time, which are tested in both real-time and offline benchmarks. A central challenge is balancing compact, efficient memory with reliable retrieval of fine details, especially when later questions depend on evidence that seemed unimportant when first observed.

Latest papers

CardsList

96 papers15 papers in the last four weeks, down 32% on the four weeks before. 0.1% of all new papers.

  1. From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents

    Mar 2, 2026Niu Lian, Yuting Wang, Hanshu Yao +5Multimodal MemoryMemory-Augmented Video Understanding

  2. EgoGraph: Temporal Knowledge Graph for Egocentric Video Understanding

    Feb 27, 2026Shitong Sun, Ke Han, Yukai Huang +2Egocentric Video UnderstandingEgocentric VQA

  3. WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMs

    Feb 25, 2026Yulin Zhang, Cheng Shi, Sibei YangVideo-Language ModelsStreaming Video Understanding

  4. FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding

    Feb 2, 2026Kangcong Li, Peng Ye, Lin Zhang +4Streaming Video UnderstandingMemory-Augmented Video Understanding

  5. SpatialMem: Metric-Aligned Long-Horizon Video Memory for Language Grounding and QA

    Jan 21, 2026Xinyi Zheng, Yunze Liu, Chi-Hao Wu +5Memory-Augmented Video UnderstandingRobotic Spatial Memory

  6. Action-Dynamics Modeling and Cross-Temporal Interaction for Online Action Understanding

    Oct 12, 2025Xinyu Yang, Zheheng Jiang, Feixiang Zhou +5Temporal Action DetectionNext-Action Prediction