Memory-Augmented VLMs

VLM: Vision-Language Model

Memory-augmented vision-language models study how systems can retain and reuse visual, textual, and action-related information beyond the current image or context window. Approaches include retrieval from episodic or structured memory, recurrent and latent memory tokens, compressed visual histories, and external world models for tasks such as long-video question answering, navigation, and robot manipulation. Work is evaluated on long-context video and spatial-reasoning benchmarks as well as memory-dependent robotic tasks, with designs ranging from training-free retrieval to end-to-end learned memory. A central challenge is preserving fine-grained, trustworthy evidence while controlling memory and inference costs; compressed or stale memories can omit useful details or mislead later decisions.

Latest papers

160 papers49 papers in the last four weeks, up 96% on the four weeks before. 0.3% of all new papers.

Ego: Embedding-Guided Personalization of Vision-Language Models
Mar 10, 2026Soroush Seifi, Simon Gardier, Vaggelis Dorovatas +2VLM Adaptation Memory-Augmented VLMs
Toyota Motor Europe
TempoFit: Plug-and-Play Layer-Wise Temporal KV Memory for Long-Horizon Vision-Language-Action Manipulation
Mar 8, 2026Jun Sun, Boyu Yang, Jiahao Zhang +7Temporal Modeling for VLA Models Memory-Augmented VLMs
Xi’an Jiaotong-Liverpool University
RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Mar 4, 2026Yinpei Dai, Hongze Fu, Jayjun Lee +6Memory-Augmented VLMs Robot Manipulation Benchmarks
University of Michigan · Stanford University · Figure AI
Mema: Memory-Augmented Adapter for Enhanced Vision-Language Understanding
Feb 28, 2026Ying Liu, Yudong Han, Kean Shi +1Memory-Augmented VLMs Multimodal Large Language Models
Beijing Institute of Technology · Peking University · Yangtze Delta Region Academy of Beijing Institude of Technology, Jiaxing, China
OCR-Agent: Agentic OCR with Capability and Memory Reflection
Feb 24, 2026Shimin Wen, Zeyu Zhang, Xingdou Bian +5Self-Correction in VLMs Vision-Language Models
Southwest Minzu University · AI Geeks
Notes-to-Self: Scratchpad Augmented VLAs for Memory Dependent Manipulation Tasks
Feb 24, 2026Sanjay Haresh, Daniel Dijkman, Apratim Bhattacharyya +1Memory-Augmented VLMs Vision-Language-Action Models
Qualcomm AI Research
Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation
Feb 22, 2026Zaijing Li, Bing Hu, Rui Shao +5Efficient VLA Model Inference Memory-Augmented VLMs
Harbin Institute of Technology, Shenzhen · PengCheng Laboratory · Shenzhen Loop Area Institute +1
Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory
Feb 20, 2026Vatsal Agarwal, Saksham Suri, Matthew Gwilliam +2Streaming Video Understanding Memory-Augmented VLMs
University of Maryland, College Park · TikTok
VLM-DEWM: Dynamic External World Model for Verifiable and Resilient Vision-Language Planning in Manufacturing
Feb 17, 2026Guoqin Tang, Qingxuan Jia, Gang Chen +4VLMs for Robotics Memory-Augmented VLMs
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing, 100876, China
Vision-Language Memory for Spatial Reasoning
Nov 25, 2025Zuntao Liu, Yi Du, Taimeng Fu +3Memory-Augmented VLMs Vision-Language Models
Spatial AI & Robotics (SAIR) Lab, University at Buffalo, USA · Stanford University, USA

Mar 10, 2026J/K move · Enter open · S save

Ego: Embedding-Guided Personalization of Vision-Language Models

Soroush Seifi, Simon Gardier, Vaggelis Dorovatas, Daniel Olmeda Reino+1

Toyota Motor Europe

VLM Adaptation Memory-Augmented VLMs

Open paper AskPDF arXiv

AI assistants that support humans in daily life are becoming increasingly feasible, driven by the rapid advancements in multimodal language models. A key challenge lies in overcoming the generic nature of these models to deliver personalized experiences. Existing approaches to personalizing large vision language models often rely on additional training stages, which limit generality and scalability, or on engineered pipelines with external pre-trained modules, which hinder deployment efficiency. In this work, we propose an efficient personalization method that leverages the model's inherent ability to capture personalized concepts. Specifically, we extract visual tokens that predominantly represent the target concept by utilizing the model's internal attention mechanisms. These tokens serve as a memory of that specific concept, enabling the model to recall and describe it when it appears in test images. We conduct a comprehensive and unified evaluation of our approach and SOTA methods across various personalization settings including single-concept, multi-concept, and video personalization, demonstrating strong performance gains with minimal personalization overhead.