Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory. Simply enlarging the context window incurs substantial computational and memory costs and encourages overfitting to spurious correlations, leading to catastrophic failures under distribution shift and violating real-time constraints in robotic systems. By contrast, humans can compress important past experiences into long-term memories and exploit them to solve tasks throughout their lifetime. In this paper, we propose VPWEM, a non-Markovian visuomotor policy equipped with working and episodic memories. VPWEM retains a sliding window of recent observation embeddings as short-term working memory, and introduces a Transformer-based contextual memory compressor that recursively converts out-of-window observations into a fixed number of episodic memory embeddings. The compressor uses self-attention over a cache of past summary embeddings and cross-attention over a cache of historical observations, and is trained jointly with the policy. We instantiate VPWEM on diffusion policies to exploit both short-term and episode-wide information for action generation with nearly constant memory and computation per step. Experiments demonstrate that VPWEM outperforms state-of-the-art baselines including diffusion policies and vision-language-action (VLA) models by more than 20% on the memory-intensive manipulation tasks in MIKASA and achieves an average 5% improvement on the mobile manipulation benchmark MoMaRT. Code is available at https://github.com/HarryLui98/code_vpwem.
Figures & tables
Fig. 1: Comparison between VPWEM and existing methods: (a) Policies typically predict actions based on observations within a fixed-size context window. Historical observations that move out of the window are typically discarded and will not be used during action generation. (b) VPWEM augments diffusion-based policies with working and episodic memories, where the contextual memory compressor recursively consolidates historical observations into fixed-size memory embeddings.
Fig. 2: Overview of the VPWEM framework. (a) Policy architecture. Modules in green are learnable components, including the multi-modal encoder, contextual memory compressor, summary embeddings, and the Transformer-based noise predictor in the diffusion policy. These components are optimized end-to-end with a behavior cloning loss. (b) Training process. Each training sample contains the complete trajectory from the beginning of the episode. The short-term memory component follows the standard diffusion policy baselines. Observations outside the context window are subsampled with a fixed ratio and passed to the contextual memory compressor, which outputs a fixed number of memory embeddings. The combined long- and short-term memory embeddings are used to condition the noise prediction network. (c) Inference process. Each decision step consists of three steps: encoding new frames and updating the observation and summary caches; compressing out-of-window embeddings to obtain long-term memories; and predicting the action chunk.
Fig. 3: Three benchmarks in our experiments.
Hyperparameter
Value
Diffusion Policy
Diffusion Model
DDPM [ 58 ]
Sampling steps
50
Action chunk length H
8
Temperature
1.0
Number of layers
8
TABLE I: Hyperparameters of DP-VPWEM
Fig. 4: Performance on MoMaRT and Robomimic benchmark.
Fig. 5: Performance on MIKASA benchmark.
Methods
DP-PTP
DP-VPWEM
Context Length
4
8
16
32
64
128
L2S2
Model Size (M)
50.67
50.67
50.68
50.69
50.70
50.73
52.98
GPU Memory (MB)
656
684
794
942
1258
1792
734
Training Time (s/step)
0.02
0.02
0.08
0.08
0.17
0.18
0.09
Inference Time (s)
0.17
0.18
0.27
0.35
0.44
0.72
0.22
Success Rate (%)
46.1
45.0
49.1
49.6
49.6
40.4
58.3
TABLE II: Comparison of Computational and Storage Costs on the unload-suboptimal task in the MoMaRT benchmark
Fig. 6: Ablation studies on Long-Horizon Square task. (a) Fixed-context baselines remain limited even with longer observation horizons, while VPWEM outperforms them with only a few episodic memory embeddings. Increasing compressor depth and cache size improves performance until saturation, and cache management methods perform similarly except for random eviction. (b) Episodic memory preserves substantially more initial-state information than working memory, achieving 20 × lower validation MSE. Although probe performance decreases as the episode progresses, episodic memory still yields 4 × lower loss than working memory after 400 steps. More episodic memory embeddings (i.e., larger M ) slightly improve performance at later timesteps.
Many robotic tasks require short-term memory, whether it's retrieving an object that's no longer visible or turning off an appliance after a set period. Yet, most visuomotor policies trained via imitation learning rely only on immediate sensory input without using past experiences to guide decisions. We present PRISM, a transformer-based architecture for visuomotor policies to effectively use short-term memory via two key components: (i) gated attention, which filters retrieved information to suppress irrelevant details, improving performance by reducing the spurious correlations between the history and current action prediction, (ii) a hierarchical architecture that first compresses local information into compact tokens and then integrates them to capture temporally extended dependencies, improving its compute and memory footprint. Together, these mechanisms enable us to scale short-term memory in visuomotor policies for up to two minutes. To systematically evaluate memory in visuomotor control, we introduce ReMemBench -- a benchmark of eight diverse household manipulation tasks spanning four categories of short-term memory -- designed to foster general memory mechanisms rather than siloed, task-specific solutions. PRISM consistently outperforms prior works, including recurrent architectures, transformers, and their variants -- achieving an absolute improvement of 5%--12% over the strongest baseline. On the RoboCasa and LIBERO benchmarks, it achieves absolute improvements of 11%--15% over its no-memory variant and fine-tuned Vision-Language-Action baselines such as GR00T-N1-3B and OpenVLA, despite not leveraging any large-scale pretraining. Together, PRISM and ReMemBench establish a foundation for developing and evaluating short-term memory-augmented visuomotor policies that scale to long-horizon tasks. Additional materials are available at https://shahrutav.github.io/short-term-memory
Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang +5
1Robotics and AI Institute, 2The University of Texas at Austin
Robotic manipulation tasks exhibit varying memory requirements, ranging from Markovian tasks that require no memory to non-Markovian tasks that depend on historical information spanning single or multiple interaction trials. Surprisingly, simply extending observation histories of a visuomotor policy often leads to a significant performance drop due to distribution shift and overfitting. To address these issues, we propose Gated Memory Policy (GMP), a visuomotor policy that learns both when to recall memory and what to recall. To learn when to recall memory, GMP employs a learned memory gate mechanism that selectively activates history context only when necessary, improving robustness and reactivity. To learn what to recall efficiently, GMP introduces a lightweight cross-attention module that constructs effective latent memory representations. To further enhance robustness, GMP injects diffusion noise into historical actions, mitigating sensitivity to noisy or inaccurate histories during both training and inference. On our proposed non-Markovian benchmark MemMimic, GMP achieves a 30.1% average success rate improvement over long-history baselines, while maintaining competitive performance on Markovian tasks in RoboMimic. All code, data and in-the-wild deployment instructions are available on our project website https://gated-memory-policy.github.io/.
Video-world-model policies learn action-relevant representations by predicting future observations. However, they condition on only a short observation window, which renders long-horizon manipulation non-Markovian when the correct action depends on earlier events that are no longer visible. We present MemoryVAM, an episodic memory mechanism for video-world-model policies. We employ a Recap-Cue (RC) module, in which a Perceiver-based Recap Compressor maps per-frame CLIP embeddings into compact memory tokens, and a lightweight Cue Gate estimates task completion from memory and language. These tokens are injected into both the video backbone and the action decoder, aligning policy imagination with episode progress and conditioning actions on history. Our model trains the memory module with video prediction, a delta-reconstruction auxiliary loss, and episode-boundary supervision, requiring no per-frame progress labels. The same mechanism applies to UNet and Diffusion Transformer (DiT) backbones by changing only the cross-attention injection interface. On LIBERO-Mem, our model improves average success from 5% to 42.5%. On real robots, it achieves 78.3% success on counting tasks, 80.0% on spatial recall, and 75.0% on sequential tracking. Project page: https://MemoryVAM.github.io/
Yuxin Jiang, Chang Yu, Yunuo Chen +4
University of California, Los Angeles · University of California, San Diego · University of Utah +1