Multimodal Memory

Follow this topic Ask about this topic

Multimodal memory studies how AI agents preserve and use information from conversations, images, audio, and video across long periods and changing tasks. Common approaches include hierarchical episodic and semantic memories, event or entity graphs, retrieval-augmented systems, and learned policies that decide what to store, compress, or retrieve. Evaluations use long-video question answering, personal archives, multi-session conversations, and interactive agent tasks, where systems still struggle with fine visual details, cross-session identity, and tracking changing facts. A central tension is how to reduce storage and inference costs without losing source evidence, temporal context, or the ability to verify an answer.

Latest papers

71 papers12 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning
May 8, 2026Jiazheng Li, Chi-Hao Wu, Yunze Liu +3Video Understanding Multimodal Memory
University of Connecticut · Northwestern University · University of Virginia
ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting
May 5, 2026Jiale Chang, Yuxiang RenMultimodal Memory LLM Agent Memory
Nanjing Agricultural University · Nanjing University
IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory
Apr 22, 2026Weitong Kong, Di Wen, Kunyu Peng +10Multimodal Memory Human-in-the-Loop Annotation
The authors are with Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany. · The author is with INSAIT, Sofia University “St. Kliment Ohridski”, 1784 Sofia, Bulgaria. · The author is with ETH Zurich, 8092 Zurich, Switzerland. +1
Human-Inspired Context-Selective Multimodal Memory for Social Robots
Apr 13, 2026Hangyeol Kang, Slava Voloshynovskiy, Nadia Magnenat ThalmannMultimodal Memory Social Robotics
Department of Computer Science, University of Geneva, Geneva, Switzerland · MIRALab, University of Geneva, Geneva, Switzerland
Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory
Apr 1, 2026Jiaqi Liu, Zipeng Ling, Shi Qiu +9Agent Memory Multimodal Memory
UNC-Chapel Hill · University of Pennsylvania · University of California, Santa Cruz +2
Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report
Mar 18, 2026Deliang Wen, Ke Sun, Yu WangAffective Computing Emotion Recognition
Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
Mar 16, 2026Rongjie Jiang, Jianwei Wang, Gengda Zhao +3Multimodal Memory Multimodal Agents
University of New South Wales Sydney, Australia · Zhejiang University Hangzhou, China · Shanghai Jiao Tong University Shanghai, China
PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMs
Mar 10, 2026Jinyue Li, Yuci Liang, Qiankun Li +7Memory-Augmented VLMs Medical VLMs
University of Science and Technology of China · Shenzhen University · Nanyang Technological University +2
Cognitive Prosthetic: An AI-Enabled Multimodal System for Episodic Recall in Knowledge Work
Mar 2, 2026Lawrence Obiuwevwi, Krzysztof J. Rechowicz, Vikas Ashok +2Multimodal Memory Episodic Memory
Old Dominion University Norfolk, Virginia, United States
According to Me: Long-Term Personalized Referential Memory QA
Mar 2, 2026Jingbiao Mei, Jinghong Chen, Guangyu Yang +3LLM Memory Memory Agent Benchmarks
Department of Engineering, University of Cambridge, United Kingdom · Department of Physics, University of Cambridge, United Kingdom · Independent Researcher
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
Mar 2, 2026Niu Lian, Yuting Wang, Hanshu Yao +5Multimodal Memory Memory-Augmented Video Understanding
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory

May 8, 2026J/K move · Enter open · S save

Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning

Jiazheng Li, Chi-Hao Wu, Yunze Liu, Kaize Ding+2

University of Connecticut · Northwestern University · University of Virginia

Video Understanding Multimodal Memory Long-Video Understanding Agentic RAG Memory-Augmented Video Understanding Long-Video QA

Open paper AskPDF arXiv

Understanding ultra-long videos such as egocentric recordings, live streams, or surveillance footage spanning days to weeks, remains a challenge. For current multimodal LLMs: even with million-token context windows, frame budgets cover only tens of minutes of densely sampled video, and most evidence is discarded before inference begins. Memory-augmented and agentic approaches help with scale, but their retrieval remains fragmented across modalities and lacks long-range narrative summaries that span days or weeks. We propose MAGIC-Video, a training-free framework built around a multimodal memory graph with interleaved narrative chain: the graph unifies episodic, semantic, and visual content through six typed edges and supports cross-modal retrieval, while the chain distils long-horizon entity biographies and recurring activity events. At inference time, an agentic loop interleaves graph retrieval with narrative fact injection, covering both the modality and time dimensions of ultra-long video in a single retrieval pipeline. On EgoLifeQA, Ego-R1 and MM-Lifelong, MAGIC-Video consistently outperforms strong general-purpose, long-video, and agentic baselines, with gains of 10.1, 7.4, and 5.9 points over the prior best agentic system on each benchmark. Code is available at https://github.com/lijiazheng0917/MAGIC-video.