Memory-Augmented Video Understanding

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 35

All topics

Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos
Oct 7, 2026Shravan Chaudhari, William Paul, Suchi Saria +2Memory-Augmented Video Understanding Egocentric Video Understanding
Johns Hopkins University · Johns Hopkins University Applied Physics Laboratory
VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding
Oct 7, 2026Yongchao Xu, Bowen Ye, Jiefeng Gan +5Long-Video Understanding Memory-Augmented Video Understanding
Alibaba Group, Hangzhou, China · University of Science and Technology of China, Hefei, China · Shanghai Jiao Tong University, Shanghai, China +2
VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding
Oct 5, 2026Yucheng Liu, Yufei Yin, Mingxiao Feng +3Long-Video Understanding Memory-Augmented Video Understanding
University of Science and Technology of China · Jilin University · Hangzhou Dianzi University +1
ReMem: Streaming Video Understanding With Long Context Retention
Oct 5, 2026Li Yiheng, He Xu, Wang Shaobo +2Memory-Augmented VLMs Streaming Video Understanding
Nanyang Technological University · Shanghai Jiaotong University · University of the Chinese Academy of Sciences
MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding
Sep 30, 2026Yinying Li, Yuqian Fu, Yulin Dai +3Memory-Augmented VLMs Streaming Video Understanding
East China Normal University · King Abdullah University of Science and Technology
VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents
Sep 29, 2026Jianming Xu, Jinfa Huang, Jingyang Lin +2Memory-Augmented VLMs Long-Video Understanding
University of Rochester · Microsoft
Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning
Sep 29, 2026Ziheng Huang, Yicheng Bao, Xueheng Li +8Streaming Video Understanding Video QA
East China Normal University · University of Science and Technology of China · Xiamen University
MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms
Sep 29, 2026Guohong Liu, Jialei Ye, Shanhui Zhao +2Memory-Augmented VLMs Streaming Video Understanding
Institute for AI Industry Research, Tsinghua University · Peking University
Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding
Sep 28, 2026Wei Chen, Xuanyu Zheng, Yancheng Long +6Memory-Augmented VLMs Memory-Augmented Video Understanding
HKUST · Kling AI
PREM: Prefix-Steered Recurrent Memory for Long-Video Understanding
Sep 20, 2026Siru Zhong, Qiongyan Wang, Xiaohui Lv +5Memory-Augmented VLMs Recurrent Neural Networks
The Hong Kong University of Science and Technology (Guangzhou) · Beta Infinity · Tsinghua University
VLX-VR: An Agentic-Aware Video Reasoning Model
Sep 9, 2026Sheng Li, Peng Liu, Qianqian Zhang +1Video Reasoning Memory-Augmented Video Understanding
Om AI Research
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal RAG Efficient Multimodal Inference
Zhejiang University · South China University of Technology · Lenovo Group Limited
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Aug 31, 2026Ce Zhang, Jing Bi, Jinxi He +9Adaptive Inference Streaming Video Understanding
Carnegie Mellon University · TikTok · University of Rochester +2
Dynamic Hub-and-Spoke Memory for Streaming Video Understanding
Aug 31, 2026Xinru Jiang, Lin Zhao, Xi Xiao +7Memory-Augmented VLMs Streaming Video Understanding
Northeastern University · University of Alabama at Birmingham · Tulane University +2
StreamFlow: Dynamic Memory Flows for Streaming Video Understanding
Aug 11, 2026Muxin Fu, Yifan Zhang, Wentao Zhang +5Memory-Augmented VLMs Streaming Video Understanding
Tongji University · Nanyang Technological University · University of Michigan +2
Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding
Aug 7, 2026Yeeun Choi, Youngbeom Yoo, Joon-Young Lee +2Memory-Augmented VLMs Episodic Memory
Yonsei University · Adobe Research
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Jul 30, 2026Mingkang Dong, Muxin Pu, Jie Li +8Memory-Augmented VLMs Streaming Video Understanding
Universiti Malaya · Monash University · SJTU +5
FOLIO: Focused Semantic Memory for Streaming Video Understanding
Jul 14, 2026Haoyang Fan, Dhruv Parikh, Anvitha Ramachandran +4Memory-Augmented VLMs Streaming Video Understanding
University of Southern California (USC), Los Angeles, CA, USA · Intel Labs, USA · DEVCOM Army Research Office
SLVMBench: Skill Learning from Video Memory
Jul 13, 2026Yudong Yang, Guangzhi Sun, Yixuan Li +1Memory-Augmented VLMs Video-Language Models
Tsinghua University · University of Cambridge
Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning
Jul 1, 2026Yixin Ji, Fanghua Ye, Juntao Li +5Long-Video Understanding Memory-Augmented Video Understanding
1Soochow University · 2Tencent Hunyuan Multimodal Department
ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memory
Jun 25, 2026Le Tu Ngoc Minh, Jinyeong Lim, Dongsu HanStreaming Video Understanding KV Caching
KAIST · School of Electrical Engineering, KAIST, Daejeon, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea
Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding
Jun 24, 2026Baiyang Song, Yuli Lin, Qiong Wu +5Streaming Video Understanding Multimodal Large Language Models
1Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
What Should a Streaming Video Model Remember?
Jun 15, 2026Haonan Ge, Yiwei Wang, Hang Wu +1Streaming Video Understanding Video-Language Models
University of California, Santa Barbara · University of California, Merced · The University of Queensland
MemDreamer: Decoupling Perception and Reasoning for Long Video Understanding via Hierarchical Graph Memory and Agentic Retrieval Mechanism
Jun 5, 2026Cong Chen, Guo Gan, Kaixiang Ji +7Agentic Retrieval Long-Video Understanding
Ant Group · Zhejiang University · Central South University +2
MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering
Jun 4, 2026Qing Yang, Pengcheng Huang, Xinze Li +6Video Summarization Video QA
School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science and Technology, Tsinghua University, Beijing, China · Digital China Group, Beijing, China
FlowNar: Scalable Streaming Narration for Long-Form Videos
May 30, 2026Zeyun Zhong, Manuel Martin, Chengzhi Wu +4Streaming Video Understanding Efficient VLM Inference
Karlsruhe Institute of Technology (KIT) · Fraunhofer IOSB · Lamarr Institute for Machine Learning and Artificial Intelligence +1
Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning
May 26, 2026Xianqiang Gao, Qizhi Chen, Delin Qu +53D Spatial Reasoning Video Reasoning
University of Science and Technology of China · Shanghai AI Lab · Northwestern Polytechnical University +1
O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution
May 25, 2026Tianshuo Xu, Yichen Xie, Depu Meng +5Video Diffusion Models Long-Horizon Video Generation
Applied Intuition · University of California, Berkeley
An Efficient Streaming Video Understanding Framework with Agentic Control
May 18, 2026Jinming Liu, Jianguo Huang, Zhaoyang Jia +7Streaming Video Understanding Adaptive Model Routing
1Shanghai Jiao Tong University · 2Eastern Institute of Technology, Ningbo, China · 3Microsoft Research Asia

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos

Shravan Chaudhari, William Paul, Suchi Saria, Rama Chellappa+1

Johns Hopkins University · Johns Hopkins University Applied Physics Laboratory

Memory-Augmented Video Understanding Egocentric Video Understanding 3D Spatial Reasoning Episodic Memory

Open paper AskPDF arXiv

As we move through the world and carry out everyday tasks, we encounter objects that may become relevant only later. We are capable of recalling where we left something or what was inside a container, even without knowing we would need it later. Here, we study how an embodied assistant can build a similar memory from egocentric videos, by observing a person's day-to-day activities. We present Ledger, a persistent 3D object memory that combines object locations, their histories, and contextual descriptions. It associates observations across the recording and retains objects after they leave the view, including those the person never touches. It clusters each object's observations by resting locations and records a move only after repeated evidence, reducing the effect of localization noise. Short descriptions preserve details such as an object's contents or supporting surface. It saves these records to later answer spatial questions without having to access the original images or video. Our memory raises HD-EPIC accuracy from 29.7% to 42.6%, UCS-Bench accuracy from 33.8% to 38.5% and localizes Ego4D objects with a 0.99 m median error on returned predictions. Our analyses identify complementary roles for temporal persistence, contextual descriptions, and retrieval. Our study on 100 stitched streams of multiple scenes each further exposes failures in both retrieval and construction. Per-scene construction partially recovers the performance lost across scene changes compared to that of single scene streams.

Momentum

Related topics

Latest papers 35