Multimodal Memory

Follow this topic Ask about this topic

Multimodal memory studies how AI agents preserve and use information from conversations, images, audio, and video across long periods and changing tasks. Common approaches include hierarchical episodic and semantic memories, event or entity graphs, retrieval-augmented systems, and learned policies that decide what to store, compress, or retrieve. Evaluations use long-video question answering, personal archives, multi-session conversations, and interactive agent tasks, where systems still struggle with fine visual details, cross-session identity, and tracking changing facts. A central tension is how to reduce storage and inference costs without losing source evidence, temporal context, or the ability to verify an answer.

Latest papers

71 papers12 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

Remembering Through Change: Provenance-Preserving Hypergraph Memory for Multimodal Agents
Oct 21, 2026Jielong Tang, Ling-An Zeng, Xujie Yuan +7Agent Memory Multimodal Memory
Sun Yat-sen University · Huawei Technologies, Noah Arks’ Lab · Shandong Normal University +1
MOSAIC: An Omni-Modal Benchmark for Long-Term Memory and Cross-Modal Reasoning in Conversational Agents
Oct 21, 2026Hong Ting Tsang, Boyi Zhang, Haoyu Huang +7Multimodal Memory Long-Term Conversational Memory
The Hong Kong University of Science and Technology · University of Edinburgh
PACE-Mem: Prefix-Aware Complementary Evidence for Multimodal Agents
Oct 17, 2026Shidu Ren, Rong-cheng Tu, Hang Zhou +1Memory-Augmented Agents Memory-Augmented Video Understanding
Shidu Ren is with the Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada · Rong-cheng Tu is with the College of Computing and Data Science, Nanyang Technological University, Singapore 639798 · Hang Zhou is with the Department of Statistics and Operations Research and the School of Data and Information Science, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA +1
Multimodal Filesystem Memory For Long-Horizon Embodied Agents
Oct 15, 2026Chloe Su, Jiahan Li, Zhenting Qi +5Agent Memory Long-Term Memory for Embodied Agents
Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory
Oct 14, 2026Minghao Guo, Qingyue Jiao, Xiujin Liu +77Multimodal Memory Memory-Augmented Agents
Learning How to Remember: Memory Modality Selection for GUI Agents
Oct 13, 2026Taeyun Roh, Suhyeong Park, Gijin Kim +2Memory-Augmented Agents GUI Agents
Korea University · Myongji University · AIGEN Sciences
DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents
Oct 6, 2026Jike Zhong, Ritwick Chaudhry, Xuanbai Chen +4Multimodal Memory Memory Agent Benchmarks
Amazon AGI · University of Southern California
Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels
Oct 6, 2026Youxing LIMemory-Augmented VLMs Efficient Multimodal Inference
The Chinese University of Hong Kong
MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories
Sep 30, 2026Guangzhi Xiong, Xinyuan Zhang, Xiao Yang +14Multimodal Memory Reinforcement Learning
Meta Reality Labs · University of Virginia · Work done at Meta +1
MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding
Sep 30, 2026Yinying Li, Yuqian Fu, Yulin Dai +3Memory-Augmented VLMs Streaming Video Understanding
East China Normal University · King Abdullah University of Science and Technology
Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies
Sep 29, 2026Hui Ren, Lei Fan, Henry Pao +5Multimodal Memory Question Answering
University of Illinois Urbana-Champaign · Amazon.com, Inc.
Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Models
Sep 28, 2026Beomsu Kim, Chieh-Hsin Lai, Bac Nguyen +3Multimodal Memory World Model Learning
KAIST · Sony Group Corporation · Imperial College London
Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agents
Sep 26, 2026Yuhang Jiang, Qingwei Liao, Kaize Yin +3Multimodal Memory Multimodal Agents
Huawei Technologies Ltd.
CUE-Mem: Benchmarking Long-Term User Memory via Implicit Cues in Multimodal Conversations
Sep 26, 2026Yulin Hu, Yanyan Zhao, Zimo Long +6Multimodal Memory Memory Agent Benchmarks
Harbin Institute of Technology · Huawei Technologies Co., Ltd.
Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversations
Sep 26, 2026Wenxu Jia, Xize Cheng, Zihan Zhang +5Conversational Memory Multimodal Memory
Zhejiang University · Meituan
C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks
Sep 24, 2026Xueshu Chen, Yan Wang, Zihao Xue +5Multimodal Memory Persistent Memory for Language Models
Huzhou Normal University · Alibaba Group · University of Waterloo
To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives
Sep 11, 2026Wenqi Zhou, Zhuorui Yu, Kaiao Wen +6Multimodal Memory User Preference Modeling
University of Bristol · Memories.ai Research
MEMO: Multimodal Evidence Memory Organization for Long-Horizon LLM Agents
Sep 7, 2026Xian Gao, Jinpeng Wang, Jiacheng Ruan +3Multimodal Memory Long-Horizon LLM Agents
Shanghai Jiao Tong University · Tsinghua University · Institute of Automation of the Chinese Academy of Sciences
A Framework for Egocentric and Exocentric Procedural Understanding via Temporal Segmentation and Semantic Abstraction
Sep 4, 2026Vivek Chavan, Jörg KrügerMultimodal Memory Egocentric Video Understanding
Fraunhofer IPK, Berlin, Germany · Technische Universität Berlin, Berlin, Germany
ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory
Sep 3, 2026Shidu Ren, Yunze Liu, Xing Liu +3Multimodal Memory Memory Agent Benchmarks
University of Toronto · Tsinghua University · Northeastern University +1
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal RAG Efficient Multimodal Inference
Zhejiang University · South China University of Technology · Lenovo Group Limited
When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents
Aug 30, 2026Jiaqi Su, Cong Pang, Jiawei Hong +4Multimodal Memory Long-Horizon LLM Agents
National University of Singapore · SenseTime Research · ShanghaiTech University
GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory
Aug 27, 2026Geng Li, Yuhao Wang, Dong Li +2Multimodal Memory Agent Memory Management
Wangxuan Institute of Computer Technology, Peking University · MemoraX AI
Wontopos Tablet 2: Measuring Multilingual and Multimodal Memory Retrieval Without Lexical Matching
Aug 24, 2026Sunwoo KimPersistent Memory for Language Models Multilingual IR
Wontopos L.L.C.
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Search over the Visual World: Persistent Visual Memory, Layered Indexes, and Source-Grounded Evidence
Aug 8, 2026Sankalp Nagaonkar, Rohit Garg, Ankit Raj +2Multimodal Memory Information Retrieval
VideoDB
DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding
Aug 7, 2026Hanshu Yao, Janfeng Zhong, Niu Lian +1Multimodal Memory Document QA
1Harbin Institute of Technology, Shenzhen · 2Tsinghua Shenzhen International Graduate School, Tsinghua University
V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memory
Aug 2, 2026Dingyi Kang, Dongming Jiang, Yi Li +2Multimodal IR Multimodal Memory
Department of Computer Science, The University of Texas at Dallas · Department of Electrical and Computer Engineering, University of Florida
PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents
Aug 2, 2026Jingyu Sun, Yan Lin, Yuyang Xue +10Memory-Augmented VLMs Multimodal Memory
The University of Manchester · The University of Newcastle · The University of Edinburgh +5
Beyond Retrieval: Analytic Memory for Multimodal Agents
Jul 31, 2026Zhoujin Tian, Hao Zhang, Yao Tian +4Multimodal Memory Long-Term Conversational Memory
HKUST · ByteDance

Oct 21, 2026J/K move · Enter open · S save

Remembering Through Change: Provenance-Preserving Hypergraph Memory for Multimodal Agents

Jielong Tang, Ling-An Zeng, Xujie Yuan, Xiao Dong+6

Sun Yat-sen University · Huawei Technologies, Noah Arks’ Lab · Shandong Normal University+1

Agent Memory Multimodal Memory Graph-Based Agent Memory Multimodal Agents

Open paper AskPDF arXiv

Long-term memory for multimodal agents remains challenging, as it must preserve fine-grained visual evidence while tracking entity evolution across sessions. Existing abstraction-based memories compress images into text, often discarding visually discriminative details, whereas pixel-preserving memories retain raw observations but struggle to reliably associate diverse observations with persistent entity identities and track their evolving states. To address these limitations, we propose MHG-Mem, a multimodal hypergraph memory that jointly organizes entity identity, state evolution, and provenance to visual evidence. Specifically, MHG-Mem represents distinct entity mentions and their observed facts as individual nodes, grouping identity-consistent mentions and facts through occurrence hyperedges. It then synthesizes aspect-specific evolution states from these grouped observations and links them to supporting entities, facts, and source evidence through evolutionary and evidential hyperedges. To navigate this structured memory, we introduce hypergraph-based agentic search, which adaptively selects an entry layer, traverses typed hyperedges across memory levels, and inspects source observations when fine-grained verification is required. These designs support both detailed visual verification and reasoning over cross-session state changes. Extensive experiments across three benchmarks demonstrate the effectiveness of MHG-Mem, improving performance by 8.20-9.98% over the strongest baseline.