cs.CVNov 6, 2026

HiMEM: Hippocampus-Inspired Memory Consolidation, Pruning, and Composition for Vision-Language-Action Models in 3D Scenes

Authors: Boran Zhao, Yanbin Hu, Jun Ye, Zhenxian Hu, Yuxin Zhai, Jin Cui, Pengju Ren

Organizations: School of Software Engineering, Xi’an Jiaotong University, Xi’an, China. · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China. · School of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China.

Abstract

Memory plays a crucial role in long-horizon task execution and cross-task generalization for Vision-Language-Action (VLA) models operating in complex 3D scenes. However, existing VLA memory mechanisms suffer from memory noise, particularly in complex 3D scenes, as stored episodic memories contain extensive redundant details about scene layouts, object configurations, and robot-specific states, which obscure generalizable manipulation principles and hinder cross-task transfer. To address this challenge, we first introduce a hippocampus-inspired memory consolidation mechanism that synthesizes extensive episodic experiences into a compact set of essential semantic memories, enabling VLA models to derive generalizable principles from practical experience. Second, we propose an episodic memory pruning mechanism that removes redundant episodic memories whose contents are captured by consolidated semantic memories, thereby increasing effective memory capacity and improving long-horizon task execution. Finally, to handle complex task scenarios, we develop a flexible memory composition mechanism that jointly leverages multiple episodic and semantic memories to enhance cross-scenario generalization. Experiments on LIBERO, LIBERO-Plus demonstrate the effectiveness of HiMEM across different VLA backbones, improving the average success rate of pi_0 from 91.88% to 98.20% on LIBERO.

Figures & tables

Explore similar work

CardsList
  1. ECHO: Continuous Hierarchical Memory for Vision-Language-Action Models

    May 9, 2026Yanbin Hu, Jin Cui, Jiayi Lu +6Memory-Augmented VLMsLong-Horizon Robotic Manipulation

  2. Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation

    Jul 8, 2026Hongyu Qu, Jianzhe Gao, Xiaobin Hu +6Memory-Augmented VLMsRobotic Manipulation