Memory-Augmented Video Understanding

Memory-augmented video understanding studies how models can retain useful evidence from long videos or live streams when their context and compute are limited. Approaches include bounded frame caches and semantic compression, hierarchical event- or entity-based memories, and retrieval systems that can revisit relevant clips; some methods also update recurrent states or model weights over time. These techniques support tasks such as streaming question answering, long-video and egocentric reasoning, and tracking objects or events across time, which are tested in both real-time and offline benchmarks. A central challenge is balancing compact, efficient memory with reliable retrieval of fine details, especially when later questions depend on evidence that seemed unimportant when first observed.