Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.
Figures & tables
Figure 1: Example of non-Markovian task. Due to architectural limitation, a vanilla VLA loses track after pressing the red button and presses it again. HeLM successfully executes the task by storing progress in language memory, allowing the HLP to guide the LLP correctly.
Figure 2: Overall Architecture of HeLM. The High Level Policy (HLP) integrates language memory with a pretrained VLM, which switches between a ‘detect mode’ that monitors events and an ‘update mode’ that updates these events into language memory. The Low Level Policy (LLP), a pretrained VLA, executes action commands generated by the HLP.
Figure 3: Update / Detect mode of HLP in evaluation loop. An example of transitions between Detect and Update modes and corresponding memory updates across intra- and inter-episode scenarios. Each action at is generated based on Mt−1 and the current instruction.
Figure 4: The Real-World Evaluation Suite of HeLM. Our tasks are categorized into two types based on their temporal memory requirements. (Top) Intra-episode tasks (A1–A5) evaluate working memory within a single instruction. (Bottom) Inter-episode tasks (E1–E5) assess episodic context across task boundaries.
Figure 5: Intra-episode real-world evaluation results. HeLM achieves the highest average subtask success rate on five intra-episode tasks. In contrast, Keyframe-HLP fails to detect termination in A1, A4 (denoted by ⋆ ), while the baselines PCMB- π0 and Naïve- π0 (denoted by ⋆⋆ ) fail both sequencing and termination across all trials. Under strict task success, HeLM retains 81.3%, while Keyframe-HLP drops to 29.3% and both PCMB- π0 and Naïve- π0 drop to 0%. Bars report mean subtask success rate over 15–30 trials per task (3–6 variants, 5 trials each). Error bars denote one standard deviation across variants.
Method
E1
E2
E3
E4
E5
Avg.
Keyframe-HLP
27.6⋆
0.0
33.3
0.0
22.2
16.6
HeLM (Ours)
90.5
87.7
90.0
80.0
97.7
89.2
Table 1: Inter-episode real-world evaluation. We report subtask success rates (%) on tasks E1–E5, which require retaining task-relevant information across episode boundaries. Avg. denotes the mean across the five tasks. ⋆ indicates failure to terminate.
Method
T1–T2
T3–T8
T9–T10
Avg.
π0.5
65.0
10.0
87.5
36.5
π0.5 + HLP (w/o memory)
27.5
0.0
32.5
12.0
π0.5 + HLP + memory (w/o FT)
5.0
0.0
37.5
8.5
HeLM (Ours)
67.5
45.8
87.5
58.5
Table 2: Memory and HLP fine-tuning ablations on LIBERO-Mem. We report mean task success rates (%) for three task groups and the average across all ten tasks. All configurations use π0.5 [ 18 ] as the low-level policy. The memory ablation removes explicit language memory, while the fine-tuning ablation retains memory but uses an HLP without task-specific fine-tuning. T3–T8 require persistent temporal state tracking.
Method
A1
A2
A3
A4
A5
Avg.
HeLM w/ one-stage inference
100.0
81.5
22.2
86.8
48.1
67.7
HeLM (Ours)
100.0
92.2
96.6
66.6
73.3
85.7
Table 3: Two-stage inference ablation on real-world tasks. We report subtask success rates (%) on intra-episode tasks A1–A5. The one-stage variant jointly predicts events, memory updates, and action commands, whereas HeLM separates event detection from event-triggered memory updates and command generation.
Method
A1 ′
A2 ′
A3 ′
A4 ′
A5 ′
Avg.
Keyframe-HLP
100.0⋆
63.9
77.8
38.9⋆
48.1⋆
65.7
HeLM (Ours)
100.0
95.0
67.0
67.0
61.0
78.0
Table 4: Compositional generalization on real-world tasks. We report subtask success rates (%) on held-out variants A1 ′ –A5 ′ of the intra-episode tasks. These variants evaluate unseen configurations of learned subtasks without additional HLP fine-tuning. ⋆ indicates failure to terminate.
Figure 6: Task-length robustness and memory scaling. (a) HeLM success rates on extended variants of A1 (Press N Times) and A2 (Press in Order) as the number of required subtasks increases. (b-1, b-2) Inference latency and GPU memory usage versus the number of stored events for HeLM and Keyframe-HLP.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Generation of HeLM data. Instead of recording multiple episodes of long-horizon non-Markovian tasks, we use a task configuration list to logically stitch together short subtask episodes.
Hyperparameter
Value
Base model
Qwen2.5-VL-7B-Instruct
Optimizer
AdamW
Learning rate
2×10−5
Weight decay
0.0
Scheduler
Trainer default
LR warmup
warmup_ratio = 0.03
Appendix
Table 5: Training hyperparameters used for HeLM. We fine-tune Qwen2.5-VL (7B) as the HLP for our model.
ID
Task Name
Language Instruction Template
Vars.
Variation Type
Intra-episode Tasks (Working Memory)
A1
Press N Times
Press the blue button N times.
3
N∈{1,2,3}
A2
Press in Order
Press the [Order] buttons in order.
6
Order (B, G, R)
A3
Wipe the Window
Wipe the [Order] part of the window in order. Each part of the window is wiped from left to right.
6
Order (bottom, middle, top)
A4
Find the Object
Open the drawers in the following order to find the toy hamburger, and stop when you find it in the drawer: top_left, top_right, bottom_right, bottom_left.
4
Target Location
A5
Pick, Place, Press
Pick and place the banana from the [A] plate to the [B] plate, press the orange button, and then pick and place the banana from the [B] plate to the [C] plate.
6
Transfer Path (Red, Blue, White)
Appendix
Table 6: Detailed description of the Real-World Evaluation Suite. The suite consists of five intra-episode tasks (A1–A5) for working memory and five inter-episode tasks (E1–E5) for episodic context, covering a diverse range of non-Markovian reasoning challenges. Each inter-episode task contains two language instructions.
Figure 8: Four Types of Failure Cases. Examples of low-level, detection, order, and termination failures.
HeLM (Ours)
Keyframe-HLP
Failure Type
A1
A2
A3
A4
A5
Avg
A1
A2
A3
A4
A5
Avg
1. Low-Level
0.0
4.4
0.0
0.0
17.8
4.4
6.7
4.4
0.0
0.0
0.0
2.2
2. Detection
0.0
0.0
3.3
33.3
8.9
9.1
3.3
4.4
40.0
0.0
53.3
20.2
3. Order
0.0
3.3
0.0
0.0
0.0
0.67
0.0
0.0
0
20.8
2.2
4.6
4. Termination
0.0
0.0
0.0
0.0
0.0
0.0
66.7
0.0
0.0
100
0.0
33.3
Appendix
Table 7: Failure Analysis of HeLM and Keyframe-HLP. Detailed breakdown of failure types across intra-episode tasks (A1–A5). HeLM exhibits no termination failures and only rare order violations.
Method
A1
A2
A3
A4
A5
Avg.
HeLM (Ours)
100.0
87.8
96.6
66.7
55.6
81.3
Keyframe-HLP
0.0
86.7
20.0
0.0
40.0
29.3
PCMB- π0
0.0
0.0
0.0
0.0
0.0
0.0
Naïve- π0
0.0
0.0
0.0
0.0
0.0
0.0
Appendix
Table 8: Strict task success rate on intra-episode tasks. A trial is counted as successful only if all subtasks are completed in the correct order with proper termination. Compared to partial subtask success , baselines collapse to near-zero on most tasks, revealing systematic order and termination failures masked by the diagnostic metric.
Task length (number of subtasks)
Task
1
2
3
4
5
6
7
Press N Times (A1)
100
100
100
100
100
100
100
Press in Order (A2)
100
100
93
80
80
77
88
Appendix
Table 9: Success rates across task lengths. We evaluate HeLM on extended variants of A1 (Press N Times) and A2 (Press in Order). Task length denotes the number of required subtasks. Entries report success rates (%).
Method
A1
A2
A3
A4
A5
Avg.
HeLM (base)
100
92.2
96.6
66.6
73.3
85.7
HeLM (Smol)
88.8
70.4
66.6
41.6
25.9
58.7
Appendix
Table 10: Evaluation results on different model combinations. The baseline configuration consists of Qwen2.5-VL (7B) as the HLP and π0 (3.5B) as the LLP, while the efficient variant utilizes SmolVLM (500M) and SmolVLA (450M).
Method
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
Avg.
Baselines from LIBERO-Mem paper
π0 (h=1)
50.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
5.0
SlotVLA (h=1)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
SlotVLA (h=8)
50.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
5.0
Naive E-SlotSSM
50.0
0.0
33.3
0.0
14.3
0.0
0.0
0.0
30.0
20.0
14.8
Our additional evaluation
Appendix
Table 11: Results on LIBERO-Mem benchmark. Top: baseline results from [ 10 ] . Bottom: our additional evaluation with π0.5 baseline and HeLM. The metric follows the official codebase implementation.
Figure 9: VLM System prompt used for HLP Detect mode.
Figure 10: VLM system prompt used for HLP Update mode.
Figure 11: Breakdown of non-Markovian task. Intra-tasks (A1–A5) and Inter-tasks (E1–E5).
Figure 12: Example of Taskspec. This figure shows an example of TaskSpec for task press button in human order (E2).
Figure 14: Example of HeLM HLP train data. It contains example of HLP training data of HeLM ( Press the button N times ). There are four types of train data: detect(false), detect(true), update(intra), and update(inter).
Vision-Language-Action (VLA) models fail systematically on long-horizon manipulation tasks despite strong short-horizon performance. We show that this failure is not resolved by extending context length alone in the current reactive execution setting; instead, it stems from three recurring execution-loop deficiencies: the memory gap, the verification gap, and the recovery gap. We present HELM, a model-agnostic framework that addresses these deficiencies with three components: an Episodic Memory Module (EMM) that retrieves key task history via CLIP-indexed keyframes, a learned State Verifier (SV) that predicts action failure before execution from observation, action, subgoal, and memory-conditioned context, and a Harness Controller (HC) that performs rollback and replanning. The SV is the core learning contribution: it consistently outperforms rule-based feasibility checks and ensemble uncertainty baselines, and its effectiveness depends critically on access to episodic memory. On LIBERO-LONG, HELM improves task success rate by 23.1 percentage points over OpenVLA (58.4% to 81.5%), while extending the context window to H=32 yields only a 5.4-point gain and same-budget LoRA adaptation remains 12.2 points below HELM. HELM also improves long-horizon performance on CALVIN and substantially boosts recovery success under controlled perturbations. Ablations and mechanism analyses isolate the contribution of each component, and we release LIBERO-Recovery as a perturbation-injection protocol for evaluating failure recovery in long-horizon manipulation.
Zijian Zeng, Fei Ding, Huiming Yang +1
Tsinghua University · Alibaba Group · Bengbu University
Vision-language-action (VLA) models provide a unified paradigm for connecting visual perception, language understanding, and robotic control. However, existing VLA models still face major challenges in long-horizon tasks: sparse expert demonstrations constrain cross-task compositional generalization; the non-Markovian nature of long-horizon tasks makes it difficult for policies conditioned only on current observations to maintain temporal consistency; limited closed-loop error correction allows execution errors to accumulate; and end-to-end action fine-tuning may weaken the high-level semantic representations of vision-language model (VLM) backbones. To address these issues, we propose a hierarchical long-horizon VLA architecture with an explicit language-memory module. The central idea is to convert discrete temporal observations into a coherent textual memory sequence with temporal logic. The system is decoupled into a high-level VLM and a low-level VLA: the high-level VLM performs semantic reasoning through a visual question answering training paradigm, while the low-level VLA executes precise continuous control conditioned on subtask instructions and visual observations. The high-level VLM recursively updates both language memory and subtask instructions using the previous memory as a contextual anchor, enabling persistent temporal tracking and dynamic correction during long-horizon execution. We evaluate the proposed method in multiple simulation environments and conduct sim-to-real experiments on a real robotic platform. The results demonstrate that explicit language memory improves the success rate and robustness of VLA models on complex long-horizon tasks while providing an interpretable semantic account of the decision process.
Mainstream Vision-Language-Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long-horizon, temporally dependent tasks. Existing memory-augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy-side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem-VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short-term and long-term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context-relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short-term and long-term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem-VLA.
Hongyu Qu, Jianzhe Gao, Xiaobin Hu +6
Nanjing University of Science and Technology · Zhejiang University · National University of Singapore