Memory-Augmented VLMs

VLM: Vision-Language Model

Overview

Memory-augmented vision-language models study how systems can retain and reuse visual, textual, and action-related information beyond the current image or context window. Approaches include retrieval from episodic or structured memory, recurrent and latent memory tokens, compressed visual histories, and external world models for tasks such as long-video question answering, navigation, and robot manipulation. Work is evaluated on long-context video and spatial-reasoning benchmarks as well as memory-dependent robotic tasks, with designs ranging from training-free retrieval to end-to-end learned memory. A central challenge is preserving fine-grained, trustworthy evidence while controlling memory and inference costs; compressed or stale memories can omit useful details or mislead later decisions.

Latest papers

CardsList

175 papers20 papers in the last four weeks, down 52% on the four weeks before. 0.2% of all new papers.

  1. CoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training

    Nov 18, 2026Chengye Song, Rui Song, Shengqi Wang +4Tactile-Vision-Language-Action ModelsVLA Adaptation

  2. PReMind: When Personal Memory Meets Visual Hazards

    Nov 18, 2026Yeongtak Oh, Donghyeok Lee, Joonhee Oh +2Memory-Augmented VLMsVision-Language Models

  3. Visual Harness Scales General VLMs to Medical Images

    Nov 17, 2026Shaowen Wan, Hanqi Jiang, Zihao Wu +6Medical VLMsTool Use in VLMs

  4. Out of Sight, Into Memory: Compact Visual Latents Retain Local Evidence in Ultra-High-Resolution Remote Sensing

    Nov 16, 2026Meng Tang, Siyuan Du, Yuxin Liu +3Memory-Augmented VLMsRemote Sensing Visual Question Answering

  5. ReAccess: Look Once, Stay Focused with Efficient Test Time Memory

    Nov 16, 2026Tianhao Chen, Yilong Fan, Yuheng Wu +2Memory-Augmented VLMsEfficient VLM Inference

  6. SAGE-VQA: A Training-Free, Self-Evolving Agentic Framework with Grounded Evidence for Knowledge-Based Visual Question Answering

    Nov 15, 2026Ruirui Shen, Jianqi Gao, Xiangfeng Luo +5Knowledge-Based Visual Question AnsweringReasoning for VLMs

  7. Prior-as-Skill: Remember Outside and Read Inside with On-Policy Self-Distillation for Self-Evolving VLM Agents

    Nov 11, 2026Yan Lin, Jingyu Sun, Yifei Qian +6Memory-Augmented VLMsTemporal Reasoning in VLMs

  8. HiMEM: Hippocampus-Inspired Memory Consolidation, Pruning, and Composition for Vision-Language-Action Models in 3D Scenes

    Nov 6, 2026Boran Zhao, Yanbin Hu, Jun Ye +4Memory-Augmented VLMsVision-Language-Action Models

  9. RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding

    Nov 4, 2026Hengjie Zhu, Dayan Wu, Xinze Liu +3Efficient VLM InferenceSpeculative Decoding

  10. PREM: Prediction-Regulated Memory for Robot Manipulation

    Oct 31, 2026Xiao Huang, Ya Zhang, Weidi XieMemory-Augmented VLMsRobotic Manipulation

  11. What Should a Navigator Remember? Spatiotemporal Memory for Zero-Shot Vision-Language Navigation

    Oct 28, 2026Yuhang Peng, Zhikai Wang, Ziyi Wang +5Vision-Language NavigationMemory-Augmented VLMs

  12. MemVLN: Streaming Episodic Memory for Vision-Language Navigation

    Oct 26, 2026Yan Wang, Jiangyong Huang, Siyuan Huang +1Vision-Language NavigationMemory-Augmented VLMs

  13. Variational Cognitive Memory Abstraction for Long-Horizon Vision-Language-Action Models

    Oct 25, 2026Xinyu Liu, Daiying Tian, Chenwei Tang +6Memory-Augmented VLMsTemporal Modeling for VLAs

  14. LogLook: Execution-Grounded Memory for Frontier-Model Robot Planners

    Oct 23, 2026Tianyi Jiang, Tianyao Tong, Zhenshan BingMemory-Augmented VLMsLarge Language Model-Based Robot Planning

  15. Refining Open-Vocabulary Semantic Segmentation via Regional Semantics and Visual Prototypes

    Oct 22, 2026Aijing Yu, Zhengbo Wang, Lijun Sheng +2Open-Vocabulary Semantic SegmentationVision-Language Alignment

  16. Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control

    Oct 21, 2026Qinghang Liu, Conglin Wang, Zongsheng Li +6Memory-Augmented VLMsLong-Horizon Robotic Manipulation

  17. UniWALM: Unified World Action Language Model for Embodied Robot Control

    Oct 20, 2026Rahul Sajnani, Diego Romeres, Mingyuan Zhang +9Vision-Language-Action ModelsWorld Models for VLAs

  18. OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory

    Oct 19, 2026Yu Zhou, Cheng-Fu Yang, Rui Sun +8Tool Use in VLMsMultimodal Reasoning

  19. ReInsVLN: Revisiting Historical Evidence for Training-Free Vision-and-Language Navigation

    Oct 19, 2026Siwei Cheng, Hao Wu, Zitong Qi +7Vision-Language NavigationMemory-Augmented VLMs

  20. ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI Agents

    Oct 19, 2026Zhicheng Zheng, Cheng Zhang, Shengping Yu +1Memory-Augmented VLMsMemory-Augmented Language Model Agents

  21. Write Once, Query Many: Evolving Compact Reusable Memory for Long Videos

    Oct 17, 2026Kerui Chen, Jinglu Wang, Yun Wang +2Memory-Augmented Video UnderstandingMemory-Augmented VLMs

  22. From Sliding Windows to Self-evolving Lifelong Memory: A Dual-perspective Survey of Memory Paradigms for Long-Horizon Robotics VLA Models

    Oct 16, 2026Yong Wang, Boyu Shen, Xinwei Zhang +2Memory-Augmented VLMsLong-Term Memory for Embodied Agents

  23. When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models

    Oct 15, 2026Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev +6Memory-Augmented VLMsVision-Language-Action Models

  24. REKI: Registering Knowledge in VLAs

    Oct 15, 2026Juliette Marrie, Paul Pacaud, Adrien Ramanana Rahary +2Vision-Language-Action ModelsEfficient VLAs