Multimodal Memory

Follow this topic Ask about this topic

Multimodal memory studies how AI agents preserve and use information from conversations, images, audio, and video across long periods and changing tasks. Common approaches include hierarchical episodic and semantic memories, event or entity graphs, retrieval-augmented systems, and learned policies that decide what to store, compress, or retrieve. Evaluations use long-video question answering, personal archives, multi-session conversations, and interactive agent tasks, where systems still struggle with fine visual details, cross-session identity, and tracking changing facts. A central tension is how to reduce storage and inference costs without losing source evidence, temporal context, or the ability to verify an answer.

Latest papers

71 papers12 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning
Jul 30, 2026Jingxiang Fan, Junbao Zhuo, Bochao ZouMultimodal Memory Multimodal Reasoning
University of Science and Technology Beijing 30 Xueyuan Road, Haidian District, Beijing 100083, China
ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding
Jul 29, 2026Xinkui Zhao, Enbo Chen, Yifan Zhang +4Agent Memory Multimodal Memory
Zhejiang University Hangzhou, China · Xidian University Xi’an, China
Seen, Said, or Forgotten? A Causal Audit of Visual KV Memory Across Dialog Turns
Jul 28, 2026Hong Chen, Kang Chen, Yuxuan Fan +4Memory-Augmented VLMs Visual Question Answering
The Hong Kong University of Science and Technology (Guangzhou) · Wenzhou-Kean University
Mi-Memory: A Lifecycle Memory Framework for Personal AI
Jul 21, 2026Xule Liu, Hanlin Teng, Chao Li +15Multimodal Memory LLM Personalization
Darwin Agent Team
Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents
Jul 17, 2026Halima Bouzidi, Mboutidem Ekemini Mkpong, Mohammad Abdullah Al FaruqueMultimodal Memory Agent Memory Poisoning
University of California, Irvine
LightMem-Ego: Your AI Memory for Everyday Life
Jul 13, 2026Yijun Chen, Boyi Xiao, Yixian Zhao +10Multimodal Memory Episodic Memory
1Zhejiang University · 2South China University of Technology · 3Central China Normal University +1
Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing
Jul 9, 2026Feng Wang, Canmiao Fu, Zhipeng Huang +3Multimodal Memory Episodic Memory
Peking University · WeChat Vision, Tencent Inc.
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams
Jul 6, 2026Xiaokang Ma, Yifan Sun, Zhihong Jin +6Streaming Video Understanding Multimodal Memory
1Rightly Robotics · 2Hangzhou Institute for Advanced Study, UCAS · 3Zhejiang University
MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory
Jun 29, 2026Kuan Wang, Chao ZhangAgent Memory Multimodal Memory
Georgia Institute of Technology
DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection
Jun 25, 2026Yujin Tang, Chenming Shang, Ruize Xu +1Multimodal Memory Long-Horizon Agent Evaluation
Dartmouth College
Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games
Jun 17, 2026Shengyuan Ding, Xilin Wei, Xinyu Fang +4Multimodal Memory Sequential Decision Making
1Fudan University · 2Shanghai Innovation Institute · 3Shanghai Artificial Intelligence Laboratory +2
One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA
Jun 9, 2026Zhi Zheng, Ziqiao Meng, Hao Luan +2Multimodal RAG Multimodal Memory
School of Computing, National University of Singapore
H2HMem: A Multimodal Memory Benchmark for Agents in Human-Human Interactions
Jun 8, 2026Shiping Zhu, Yibo Yang, Zhengyang Wang +3Multimodal Memory LLM Agent Memory
Jilin University · Shanghai Jiao Tong University · University of California at Merced
M $^3$ Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
Jun 5, 2026Zhengjun Huang, Wenxuan Liu, Zhoujin Tian +6Multimodal Understanding Multimodal Grounding
The Hong Kong University of Science and Technology · Peng Cheng Laboratory · Beijing University of Chemical Technology +4
M $^3$ Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks
Jun 3, 2026Jie Huang, Ruixun Liu, Sirui Sun +4Multimodal Memory Multimodal Model Evaluation
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Yuanpei College, Peking University +3
SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory
May 30, 2026Samiul Alam, Shakhrul Iman Siam, Michael J. Proulx +4Multimodal Memory Egocentric Video QA
The Ohio State University · Meta
Task-Focused Memorization for Multimodal Agents
May 29, 2026Tao Zou, Yichen He, Tian Qiu +2Continual Learning for LLM Agents Multimodal Memory
ByteDance Seed · Fudan University
WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction
May 28, 2026Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu +14Multimodal Agent Evaluation Multimodal Memory
University of California, Santa Barbara · Johns Hopkins University · Carnegie Mellon University +3
Personal Visual Memory from Explicit and Implicit Evidence
May 27, 2026Viet Nguyen, Thao Nguyen, Vishal M. Patel +1Multimodal Memory AI Agent Benchmarks
Johns Hopkins University · University of Wisconsin-Madison · Adobe Research
Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions
May 25, 2026Jeongeun Lee, Chanyoung Park, Dongha LeeMultimodal Memory Embodied AI
Yonsei University · KAIST
StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering
May 25, 2026Ming Xie, Zizheng Huang, Xudong Tan +6Audio-Visual Understanding Streaming Video Understanding
Shanghai Innovation Institute · Fudan University · Nanjing University +2
EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
May 18, 2026Ruiping Liu, Junwei Zheng, Yufan Chen +7Multimodal Memory Video QA
Karlsruhe Institute of Technology · ETH Zurich · University of Oxford +2
MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents
May 18, 2026Ziyun Zeng, Hang Hua, Bocheng Zou +3Multimodal Memory GUI Agents
University of Rochester · MIT-IBM Watson AI Lab · University of Wisconsin-Madison
PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning
May 16, 2026Sikuan Yan, Sicheng Dong, Haotong Wang +8Multimodal Memory Long-Video Understanding
1Ludwig Maximilian University of Munich · 4Technical University of Munich · 3Huawei Heisenberg Research Center (Munich) +1
Learning to Learn from Multimodal Experience
May 16, 2026Xingyu Sui, Weixiang Zhao, Yongxin Tang +4Multimodal Memory Multimodal Agents
Harbin Institute of Technology · Huawei Technologies Co., Ltd
SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory
May 15, 2026Huacan Chai, Yukai Wang, Yingxuan Yang +7Multimodal Memory Multimodal Reasoning
Shanghai Jiao Tong University, China · OPPO, China
MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
May 14, 2026Minghao Guo, Qingyue Jiao, Zeru Shi +14Multimodal Agent Evaluation Multimodal Memory
Rutgers · Notre Dame · Princeton +1
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
May 14, 2026Xiyu Ren, Zhaowei Wang, Yiming Du +11VLM Evaluation Memory-Augmented VLMs
CSE Deparment, HKUST · CUHK · OmniMemory (Shenzhen) Intelligent Technology Co., Ltd. +1
EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding
May 11, 2026Ziyang Wang, Yue Zhang, Shoubin Yu +6Multimodal Memory Egocentric Video Understanding
UNC Chapel Hill · NTU Singapore

Jul 30, 2026J/K move · Enter open · S save

RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning

Jingxiang Fan, Junbao Zhuo, Bochao Zou

University of Science and Technology Beijing 30 Xueyuan Road, Haidian District, Beijing 100083, China

Multimodal Memory Multimodal Reasoning Video Reasoning Memory-Augmented Agents

Open paper AskPDF arXiv

Existing multimodal long-term memory agents use external memory to overcome the limited context available for long videos. However, most methods emphasize what to store rather than how stored memory should be retrieved. When retrieval becomes inaccurate or repeatedly fails to obtain useful evidence, existing agents lack mechanisms to diagnose failures from previous task trajectories and adapt future search strategies.We introduce Reflective Retrieval Memory (RRM), a reflective memory framework for long-horizon multimodal reasoning. RRM augments an entity-centric multimodal memory graph with reflective experience memory, which distills transferable procedural retrieval knowledge from historical task trajectories. Unlike episodic and semantic memories that preserve factual evidence from the current video, reflective experience memory captures reusable search strategies across tasks. RRM converts retrieved experiences into query-level guidance, while answer generation remains conditioned only on factual evidence newly retrieved from the current video. A lifecycle management mechanism further regulates experience memory through usage frequency, reuse feedback, and temporal decay, thereby reducing redundancy and noise. RRM consistently outperforms previous state-of-the-art approaches on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long, demonstrating the effectiveness of reflective retrieval memory for long-horizon multimodal reasoning.