Memory-Augmented VLMs

Memory-augmented vision-language models study how systems can retain and reuse visual, textual, and action-related information beyond the current image or context window. Approaches include retrieval from episodic or structured memory, recurrent and latent memory tokens, compressed visual histories, and external world models for tasks such as long-video question answering, navigation, and robot manipulation. Work is evaluated on long-context video and spatial-reasoning benchmarks as well as memory-dependent robotic tasks, with designs ranging from training-free retrieval to end-to-end learned memory. A central challenge is preserving fine-grained, trustworthy evidence while controlling memory and inference costs; compressed or stale memories can omit useful details or mislead later decisions.