cs.AINov 12, 2026

HeMeR: Heterogeneous Memory Reconciliation for Embodied Agents via Structured KV Reuse

Authors: Saehun Chun, Wonje Choi, Jinwoo Jang, Taeyoon Kwack, Honguk Woo

Organizations: Department of Computer Science and Engineering, Sungkyunkwan University · Department of Artificial Intelligence, Sungkyunkwan University

Abstract

Embodied agents make decisions by combining information from heterogeneous memory modules, including declarative memory for explicit knowledge, procedural memory for learned routines, and working memory for current observations. During interaction, the memory contents required for decision making change with the agent's state, goal, and task progress, while many contents remain shared across consecutive decisions. However, existing memory modules are typically accessed through separate interfaces, so each updated memory context is often processed as a new input to the foundation model. This prevents selective key-value (KV) cache reuse for unchanged memory contents and forces full KV cache recomputation even when much of the memory context remains valid. We introduce HeMeR, an inference framework that performs KV reconciliation between dynamically updated memory contexts and reusable KV states. HeMeR tracks declarative, procedural, and working memory contents across action decisions, preserves KV states for unchanged contents, and reconciles only the KV states affected by memory-content updates. This enables efficient and stable inference as memory contents evolve during interaction. We evaluate HeMeR on ALFWorld, Habitat, and real-world robotic deployments requiring coordinated use of heterogeneous memory contents. In the real-world setting, HeMeR achieves 83.20% task success and reduces per-step inference time from 2.94 to 1.87 seconds compared with MemRL, while preserving compatible KV states as task progress updates the memory context.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems

    Oct 13, 2026Zhiwei Chen, Zhongtao Rao, Waikit Xiu +4KV-Cache ReuseMulti-Agent LLM Systems

  2. Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents

    Jun 10, 2026Hao-Lun Hsu, Nikki Lijing Kuang, Boyi Liu +2Hierarchical MemoryLong-Horizon Agent Tasks

  3. HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents

    Apr 18, 2026Jinchang Zhu, Jindong Li, Cheng Zhang +2Memorization in Language ModelsHebbian Learning