Remember Where You Walk: Learning Spatial Memory for End-to-End Legged Robot Exploration

Leon Magnus; Nico Bohlinger; Jan Peters; Oleg Arenz

cs.ROOct 27, 2026

Remember Where You Walk: Learning Spatial Memory for End-to-End Legged Robot Exploration

Authors: Leon Magnus, Nico Bohlinger, Jan Peters, Oleg Arenz

Organizations: Technical University of Darmstadt, Germany. · Robotics Institute Germany (RIG). · German Research Center for AI (DFKI).

Abstract

Humanoids and other legged robots operating in complex environments must remember information that leaves their field of view and only becomes relevant much later. Reinforcement learning policies address this using memory networks such as LSTMs that are trained end-to-end with the policy to memorize such task-relevant aspects. Yet, such architectures often fail to preserve sufficient information over long horizons. In this work, we propose a network architecture based on Hadamard memory that specifically targets robot navigation. Our Local Spatial Matrix Memory (LSMM) enables RL agents to store generic task-relevant information at specific, position-based memory cells, enabling flexible and spatially grounded memorization. We evaluate LSMM on the task of autonomous maze exploration, where it matches the maze coverage of strong recurrent and finite-history baselines while being the only architecture that remembers the maze layout, reducing the reconstruction error threefold.

Figures & tables

Fig. 1 : (a) The Unitree Go2 autonomously explores a previously unseen $5\times 5\,\mathrm{m}$ outdoor maze. The red line shows the walked path. (b) The matrix memory at the end of the run. The policy has written into the cells along its path, shown with the three memory channels interpreted as RGB colors, while unwritten cells keep their initial value in gray. (c) and (d) Shows the robot navigating through the maze.

Fig. 2 : Actor architecture with the proposed LSMM memory module. Both branches share

\mathbf{z}_{t}^{\text{h}}

and

\mathbf{o}_{t}^{\text{p}}

, while the memory branch additionally uses noisy position and velocity estimates

\hat{\mathbf{p}}_{t}

and

\hat{\mathbf{v}}_{t}

to update the matrix memory

\mathbf{L}_{t}

. A CNN then extracts

\mathbf{z}_{t}^{\text{m}}

from

\mathbf{L}_{t}

and an auxiliary MLP predicts the maze wall layout

\hat{\mathbf{y}}_{t}

. The actor maps

\mathbf{z}_{t}^{\text{m}}

and its observation

\mathbf{o}_{t}

\boldsymbol{\mu}_{t}

and

\boldsymbol{\sigma}_{t}

, from which

\mathbf{a}_{t}

is sampled.

Fig. 3 : Percentage of explored cells at the end of an episode. Bars show the mean over 100 unseen evaluation mazes and three seeds per architecture, whiskers the standard deviation. All architectures reach a comparable coverage.

Fig. 4 : Average number of visits per maze cell across all evaluation episodes. $D_{\text{KL}}(p\,\|\,u)$ measures the divergence between the normalized visit distribution $p$ and a uniform distribution $u$ , where lower values mean more even exploration. LSMM spreads its visits most evenly over mazes.

Fig. 5 : Maze reconstruction error over the course of an episode. Lines show the mean over 100 unseen evaluation mazes and three seeds, shaded areas the standard deviation. LSMM is the only architecture that significantly reduces the error while exploring.

Fig. 6 : Predicted maze layout at three times during an evaluation episode. The trajectory walked up to each time is overlaid, colored from black to orange by trajectory time. The prediction of LSMM fills in wall by wall as the robot explores and closely matches the ground truth at the end of the episode, while the baselines struggle to remember the walls in the areas they have been before.

Fig. 7 : Maze reconstruction error of LSMM for memory sizes of

50\times 50\times C

with

C\in\{1,2,3,5,10\}

channels. Lines show the mean over 100 unseen evaluation mazes and three seeds, shaded areas the standard deviation. All memory sizes perform comparably.

Fig. 8 : Auxiliary prediction of spatially varying environment properties. Bars show the normalized mean absolute error over all visited cells at the final time step.

Fig. 9 : Time spent in each cell by our deployed LSMM policy. Compared to the simulation results, this deployment run distributes its visits less uniformly and follows a more repetitive pattern, which is also visible in the matrix memory in Figure 1 (b).

Fig. 10 : Maze reconstruction error over deployment time of our LSMM policy. The line shows the reconstruction error of the deployment run shown in Figure 1 . The policy reduces the error significantly within the first

30\,\mathrm{s}

Hyperparameter	Value
Training steps	$2.5\cdot 10^{9}$
Batch size (envs $\times$ steps per env)	$5\,000\times 20=100\,000$
Minibatch size / updates per batch	$25\,000$ / $5$
Discount factor $\gamma$ / GAE $\lambda$	$0.995$ / $0.9$
Clipping range	$0.2$
Entropy coef. (linearly annealed)	$4\cdot 10^{-4}\rightarrow 10^{-4}$

TABLE I : Hyperparameters used in all experiments.

Reward	Term	Coeff.
Task
Cell visit (E)	$\exp(-n_{\text{cell}}/200)$	$0.16$
Maze coverage (E)	fraction of visited cells	$0.1$
Goal direction (G)	$\tfrac{1}{2}\big(1{+}\cos(\mathbf{v}_{xy},\mathbf{g}_{t})\big)\min(\lVert\mathbf{v}_{xy}\rVert,1)$	$0.04$
Goal reached (G)	$\mathds{1}[d_{\text{goal}}{<}0.25]$	$0.0625$
Navigation shaping

TABLE II : Reward terms and coefficients, with

[x]_{+}=\max(x,0)

and

\overline{\,\cdot\,}

the mean over joints. E/G: exploration/goal episodes only;

n_{\text{cell}}

: steps in the current cell;

\delta_{i}

: offset from the cell center toward walled side

i

;

n^{\text{air}}_{\text{pairs}}

: leg pairs with both feet airborne;

t^{\text{air}}

: foot air time.

Robotic Spatial Memory

See all 30 papers

Mar 4, 2026

MEM: Multi-Scale Embodied Memory for Vision Language Action Models

Conventionally, memory in end-to-end robotic learning involves inputting a sequence of past observations into the learned policy. However, in complex multi-stage real-world tasks, the robot's memory must represent past events at multiple levels of granularity: from long-term memory that captures abstracted semantic concepts (e.g., a robot cooking dinner should remember which stages of the recipe are already done) to short-term memory that captures recent events and compensates for occlusions (e.g., a robot remembering the object it wants to pick up once its arm occludes it). In this work, our main insight is that an effective memory architecture for long-horizon robotic control should combine multiple modalities to capture these different levels of abstraction. We introduce Multi-Scale Embodied Memory (MEM), an approach for mixed-modal long-horizon memory in robot policies. MEM combines video-based short-horizon memory, compressed via a video encoder, with text-based long-horizon memory. Together, they enable robot policies to perform tasks that span up to fifteen minutes, like cleaning up a kitchen, or preparing a grilled cheese sandwich. Additionally, we find that memory enables MEM policies to intelligently adapt manipulation strategies in-context.

Marcel Torne, Karl Pertsch, Homer Walke +14

Physical Intelligence · Stanford University · UC Berkeley +1

Memory-Augmented Agents Long-Term Memory for Embodied Agents Robotic Control Long-Horizon Robotic Manipulation Vision-Language-Action Models ICL for Robotics

Ask PDF

Oct 15, 2026

Spatial Memory by Agents, for Agents

Spatial memory enables robots to reuse past observations for localization, navigation, and question answering. Designing such memory involves jointly deciding what to retain and how to retrieve it for diverse semantic, metric, and temporal needs. We investigate whether coding agents can automate this design process through iterative implementation and evaluation. Under a fixed development budget, a Designer Agent writes and revises code for memory construction and retrieval using feedback from a fixed Query Agent. The resulting implementations are frozen before building memories from held-out observation histories, without access to downstream questions, and evaluated across five complementary task tracks using the same Query Agent. Across three independent design searches, Codex / GPT-6 Astra averages 44.23 on the equally weighted five-track score, versus 29.32 for ReMEmbR, the strongest evaluated baseline supporting all five tracks, a 50.9% relative improvement. Qualitative analysis relates performance differences to evidence retention and retrieval, while identifying grounding and evidence-use failures. These results demonstrate the potential of coding agents to automate spatial memory design, while revealing task-dependent trade-offs in performance and computational cost. Project page: https://ba-fa-3344.github.io/.

Qianwei Wang, Ce Zhang, Zhanpeng Luo +3

Qianwei Wang is with Princeton University · Ce Zhang is with Carnegie Mellon University · Zhanpeng Luo is with Johns Hopkins University +2

Robotic Spatial Memory Agent Memory Agent Memory Management AI Coding Agents Memory Agent Benchmarks

Ask PDF

Sep 28, 2026

Where Memory Belongs: Ledger, an Object Ledger for Memory-Augmented VLAs

Memory is essential for long-horizon, partially observed robotic manipulation: a robot must remember which object was placed in a drawer, whose cup it moved, or how many action cycles have elapsed. Recent vision-language-action (VLA) models embed memory directly inside the policy, but benchmarks show no single in-policy mechanism covers all spatio-temporal dimensions, trailing oracle methods by a wide margin. We argue that memory type dictates where memory should reside: short-term perceptual memory (repetition, timing, retracing) belongs inside the policy, while long-term object memory (persistent spatial state, containment, event history) belongs outside as an explicit, readable record. We present Ledger, a harness that realizes this split over a single fine-tuned

π_{0.5}

policy by pairing an in-policy frame-sampling memory with an external spatio-temporal object memory, the ledger, built from a SAM3 tracker and a VLM captioner of the demonstration and read by an LLM planner that decides at step boundaries. On RoboMME, Ledger reaches the highest four-suite average among the evaluated methods, 64.3% (vs. 45.9% for the strongest prior method under identical evaluation), leading object reference (60.7% vs. 40.3%) and object permanence (86.7% vs. 56.2%) using a single set of weights. Choosing the memory source at runtime, from the instruction and the record, removes the need for a task-level router.

Tanguy Dieudonné, Jack B. Jedlicki, Heng Yang

Harvard University · ETH Zürich

Memory-Augmented VLMs Long-Horizon Robotic Manipulation Large Language Model-Based Robot Planning Robotic Manipulation Long-Term Memory for Embodied Agents Robotic Spatial Memory

Ask PDF

Abstract

Robotic Spatial Memory

Explore similar work

MEM: Multi-Scale Embodied Memory for Vision Language Action Models

Spatial Memory by Agents, for Agents

Where Memory Belongs: Ledger, an Object Ledger for Memory-Augmented VLAs