Memory-Augmented VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 150% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 90

All topics

UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modeling
Sep 11, 2026Wei Li, Rui Shao, Jie He +3Memory-Augmented VLMs Visuomotor Policy Learning
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), China, 518055 · S-Lab, Nanyang Technological University, Singapore, 639798
Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding
Sep 11, 2026Weitong Cai, Hang Zhang, Yukai Huang +6Memory-Augmented VLMs Efficient VLM Inference
Queen Mary University of London · Independent Researcher · Durham University +2
Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving
Sep 8, 2026Baojie Chen, Zijun Jia, Jing ZhongMemory-Augmented VLMs Tool Use in VLMs
Beihang University · Tsinghua University
Whose record is this? Diagnosing and authorizing record use in personalized multimodal models
Sep 7, 2026Xinyu Mao, Junsi Li, Chenyang Liu +2Memory-Augmented VLMs Multimodal Model Evaluation
University of Electronic Science and Technology of China
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
Sep 3, 2026Hongyu Qu, Guangming Yao, Ling Xing +7Memory-Augmented VLMs Video Understanding
Nanjing University of Science and Technology · Ant Group · National University of Singapore +1
Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Sep 3, 2026Yunao Zheng, Bin Wen, Xiaojie Wang +12Memory-Augmented VLMs Representation Learning
Beijing University of Posts and Telecommunications. · Kuaishou Technology.
Candidate-Expanding Routing with Permutation-Stabilized Experts for Mixed-Format Medical VQA
Sep 1, 2026Hai-Dang Nguyen, Huy-Hieu PhamMemory-Augmented VLMs Visual Question Answering
College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam · Center for Innovations in Health Sciences, VinUniversity, Hanoi, Vietnam
SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoning
Aug 31, 2026Beidi Zhao, Gexin Huang, Ciro Zhang +6Memory-Augmented VLMs Medical VQA
University of British Columbia · Vector Institute · Harvard University +3
Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy
Aug 31, 2026Shmuel Berman, Jia DengVLM Evaluation Memory-Augmented VLMs
Department of Computer Science Princeton University
Dynamic Hub-and-Spoke Memory for Streaming Video Understanding
Aug 31, 2026Xinru Jiang, Lin Zhao, Xi Xiao +7Memory-Augmented VLMs Streaming Video Understanding
Northeastern University · University of Alabama at Birmingham · Tulane University +2
AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policies
Aug 30, 2026Hongbo Gao, Zeyu Ni, Xin Wen +4Memory-Augmented VLMs Robotic Manipulation
Harbin Institute of Technology · The University of Sydney · StellarEdge AI
Spatial Memory Agent: Experience-Grounded Procedural Memory for Spatial Intelligence
Aug 13, 2026Haokai Zhang, Yuhang Ding, Yunshu Zhou +5Memory-Augmented VLMs Visual Spatial Reasoning
Zhejiang University · Shanghai Jiao Tong University · Shanghai Innovation Institute
DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation
Aug 12, 2026Yan Deng, Fei XuMemory-Augmented VLMs Diffusion-Based Planning
School of Electronic Information Engineering, Xi’an Technological University, Xi’an, China · School of Computer Science and Engineering, Xi’an Technological University, Xi’an, China
StreamFlow: Dynamic Memory Flows for Streaming Video Understanding
Aug 11, 2026Muxin Fu, Yifan Zhang, Wentao Zhang +5Memory-Augmented VLMs Streaming Video Understanding
Tongji University · Nanyang Technological University · University of Michigan +2
Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
Aug 11, 2026Yuhang Song, Bor-Jiun Lin, Jiaxu Liu +3Memory-Augmented VLMs Vision-Language Models
Department of Computer Science, University of Liverpool · Department of Computer Science, National Tsinghua University · Imperial College London +1
DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving
Aug 11, 2026Zebin Xing, Yupeng Zheng, Qiang Chen +10Memory-Augmented VLMs End-to-End Autonomous Driving
Institute of Automation, Chinese Academy of Sciences · Chongqing Chang’an Technology Co., Ltd.
Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?
Aug 8, 2026Gabriele La Malfa, Nitay Alon, Emanuele La Malfa +2VLM Evaluation Memory-Augmented VLMs
King’s College London, UK · Massachusetts Institute of Technology, USA · University of Oxford, UK +2
Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding
Aug 7, 2026Yeeun Choi, Youngbeom Yoo, Joon-Young Lee +2Memory-Augmented VLMs Episodic Memory
Yonsei University · Adobe Research
StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding
Aug 6, 2026Xichen Zhang, Guankai Li, Yinghao Zhu +6Memory-Augmented VLMs Streaming Video Understanding
The Hong Kong University of Science and Technology · Xiaohongshu Inc. · The University of Hong Kong +1
SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentation
Aug 6, 2026Mohamad Zamini, Diksha ShuklaMemory-Augmented VLMs Open-Vocabulary Semantic Segmentation
University of Wyoming Laramie, WY
BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
Aug 5, 2026Peiyan Li, Yuze Zhu, Yixiang Chen +10Memory-Augmented VLMs Generalization in Robotic Manipulation
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · FiveAges, Beijing, China +1
Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
Aug 5, 2026Houze Xu, Jizhong Li, Ziyi YeMemory-Augmented VLMs Vision-Language-Action Models
Fudan University
When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
Aug 5, 2026Yushi Sun, Yanjie ZhangMemory-Augmented VLMs Visual Grounding
LIGHTSPEED, Shenzhen, China · The Hong Kong University of Science and Technology, Hong Kong, China
ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation
Aug 3, 2026Yuzhi Huang, Weijue Bu, Ziyi Xiong +4Memory-Augmented VLMs Long-Horizon Robotic Manipulation
Shenzhen International Graduate School, Tsinghua University · China University of Mining and Technology · Shenzhen Technology University
TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory
Aug 3, 2026Kang Liu, Zijing Wang, Yongkang Liu +5Memory-Augmented VLMs Visual Reasoning
Northeastern University, China
Linear Multi-Timescale Retention as a Memory-Efficient Vision-Language Bridge
Aug 3, 2026Ashfak Yeafi, Mehedi Hasan, Md Khairul IslamMemory-Augmented VLMs Vision-Language Models
Department of Electrical and Electronic Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh · Department of Computer Science and Engineering Brac University, Dhaka, Bangladesh · Department of Mathematics and Computer Science Hobart and William Smith Colleges Geneva, NY, USA
PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents
Aug 2, 2026Jingyu Sun, Yan Lin, Yuyang Xue +10Memory-Augmented VLMs Multimodal Memory
The University of Manchester · The University of Newcastle · The University of Edinburgh +5
HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
Jul 31, 2026An Liu, Bingxi Liu, Hongyu Ding +6Memory-Augmented VLMs Vision-Language Navigation
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Southern University of Science and Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China +3
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Jul 30, 2026Mingkang Dong, Muxin Pu, Jie Li +8Memory-Augmented VLMs Streaming Video Understanding
Universiti Malaya · Monash University · SJTU +5
Seen, Said, or Forgotten? A Causal Audit of Visual KV Memory Across Dialog Turns
Jul 28, 2026Hong Chen, Kang Chen, Yuxuan Fan +4Memory-Augmented VLMs Visual Question Answering
The Hong Kong University of Science and Technology (Guangzhou) · Wenzhou-Kean University

Sep 11, 2026, cs.ROJ/K move · Enter open · S save

UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modeling

Wei Li, Rui Shao, Jie He, Lingsen Zhang+2

School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), China, 518055 · S-Lab, Nanyang Technological University, Singapore, 639798

Memory-Augmented VLMs Visuomotor Policy Learning Robotic Manipulation Vision-Language-Action Models

Open paper AskPDF arXiv

Recent advances in Vision-Language-Action (VLA) models have improved robotic manipulation, yet observation-to-action learning remains limited by a fundamental transition realizability gap, manifested in three tightly coupled problems: (i) Transition ambiguity. Visually similar current observations may correspond to different manipulation phases and imply different subsequent transitions. (ii) Prediction--execution mismatch. A visually plausible predicted future observation does not necessarily correspond to a physically realizable transition. (iii) Experience--realization mismatch. A historically executable action pattern may not necessarily realize the intended transition in the current scene and therefore requires context-aware adaptation. Accordingly, we propose UniMPA, a Unified Memory-Prediction-Action model that addresses these problems through a shared action-grounded transition interface. (i) UniMPA introduces Persistent-Selective Future Prediction to resolve transition ambiguity by modeling the intended future state evolution. A persistent latent stream continuously tracks task-level progress, while a transition-critical pixel stream selectively resolves fine-grained interaction changes through memory-grounded prediction. (ii) To assess the physical executability of the anticipated transition, the predicted transition queries a temporal Visual-Action Memory Bank. The bank retrieves historically realized visual-action experience, grounding future prediction in executable evidence. (iii) To adapt executable experience to the current scene, an Action-Visual Memory Bank retrieves visually grounded action prototypes from historical action evolution. Prototype-Biased Flow then shifts the flow source toward a historically supported action manifold for context-aware refinement.

Momentum

Related topics

Latest papers 90