HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

Gyeonghun Kim; Donghoon Kim; Minsoo Choi; Suyun Lee

cs.RONov 2, 2026

HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

Authors: Gyeonghun Kim, Donghoon Kim, Minsoo Choi, Suyun Lee

Organizations: Seoul National University

Abstract

Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.

Figures & tables

Figure 1: Example of non-Markovian task. Due to architectural limitation, a vanilla VLA loses track after pressing the red button and presses it again. HeLM successfully executes the task by storing progress in language memory, allowing the HLP to guide the LLP correctly.

Figure 2: Overall Architecture of HeLM. The High Level Policy (HLP) integrates language memory with a pretrained VLM, which switches between a ‘detect mode’ that monitors events and an ‘update mode’ that updates these events into language memory. The Low Level Policy (LLP), a pretrained VLA, executes action commands generated by the HLP.

Figure 3: Update / Detect mode of HLP in evaluation loop. An example of transitions between Detect and Update modes and corresponding memory updates across intra- and inter-episode scenarios. Each action $a_{t}$ is generated based on $M_{t-1}$ and the current instruction.

Figure 4: The Real-World Evaluation Suite of HeLM. Our tasks are categorized into two types based on their temporal memory requirements. (Top) Intra-episode tasks (A1–A5) evaluate working memory within a single instruction. (Bottom) Inter-episode tasks (E1–E5) assess episodic context across task boundaries.

Figure 5: Intra-episode real-world evaluation results. HeLM achieves the highest average subtask success rate on five intra-episode tasks. In contrast, Keyframe-HLP fails to detect termination in A1, A4 (denoted by

\star

), while the baselines PCMB-

\pi_{0}

and Naïve-

\pi_{0}

(denoted by

\star\star

) fail both sequencing and termination across all trials. Under strict task success, HeLM retains 81.3%, while Keyframe-HLP drops to 29.3% and both PCMB-

\pi_{0}

and Naïve-

\pi_{0}

drop to 0%. Bars report mean subtask success rate over 15–30 trials per task (3–6 variants, 5 trials each). Error bars denote one standard deviation across variants.

Method	E1	E2	E3	E4	E5	Avg.
Keyframe-HLP	$27.6^{\star}$	0.0	33.3	0.0	22.2	16.6
HeLM (Ours)	90.5	87.7	90.0	80.0	97.7	89.2

Table 1: Inter-episode real-world evaluation. We report subtask success rates (%) on tasks E1–E5, which require retaining task-relevant information across episode boundaries. Avg. denotes the mean across the five tasks.

\star

indicates failure to terminate.

Method	T1–T2	T3–T8	T9–T10	Avg.
$\pi_{0.5}$	65.0	10.0	87.5	36.5
$\pi_{0.5}$ + HLP (w/o memory)	27.5	0.0	32.5	12.0
$\pi_{0.5}$ + HLP + memory (w/o FT)	5.0	0.0	37.5	8.5
HeLM (Ours)	67.5	45.8	87.5	58.5

Table 2: Memory and HLP fine-tuning ablations on LIBERO-Mem. We report mean task success rates (%) for three task groups and the average across all ten tasks. All configurations use

\pi_{0.5}

[ 18 ] as the low-level policy. The memory ablation removes explicit language memory, while the fine-tuning ablation retains memory but uses an HLP without task-specific fine-tuning. T3–T8 require persistent temporal state tracking.

Method	A1	A2	A3	A4	A5	Avg.
HeLM w/ one-stage inference	100.0	81.5	22.2	86.8	48.1	67.7
HeLM (Ours)	100.0	92.2	96.6	66.6	73.3	85.7

Table 3: Two-stage inference ablation on real-world tasks. We report subtask success rates (%) on intra-episode tasks A1–A5. The one-stage variant jointly predicts events, memory updates, and action commands, whereas HeLM separates event detection from event-triggered memory updates and command generation.

Method	A1 ′	A2 ′	A3 ′	A4 ′	A5 ′	Avg.
Keyframe-HLP	$\mathbf{100.0}^{\star}$	63.9	77.8	$38.9^{\star}$	$48.1^{\star}$	65.7
HeLM (Ours)	100.0	95.0	67.0	67.0	61.0	78.0

Table 4: Compositional generalization on real-world tasks. We report subtask success rates (%) on held-out variants A1 ′ –A5 ′ of the intra-episode tasks. These variants evaluate unseen configurations of learned subtasks without additional HLP fine-tuning.

\star

indicates failure to terminate.

Figure 6: Task-length robustness and memory scaling. (a) HeLM success rates on extended variants of A1 (Press N Times) and A2 (Press in Order) as the number of required subtasks increases. (b-1, b-2) Inference latency and GPU memory usage versus the number of stored events for HeLM and Keyframe-HLP.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Generation of HeLM data. Instead of recording multiple episodes of long-horizon non-Markovian tasks, we use a task configuration list to logically stitch together short subtask episodes.

Hyperparameter	Value
Base model	Qwen2.5-VL-7B-Instruct
Optimizer	AdamW
Learning rate	$2\times 10^{-5}$
Weight decay	0.0
Scheduler	Trainer default
LR warmup	warmup_ratio = 0.03

Appendix

Table 5: Training hyperparameters used for HeLM. We fine-tune Qwen2.5-VL (7B) as the HLP for our model.

ID	Task Name	Language Instruction Template	Vars.	Variation Type
Intra-episode Tasks (Working Memory)
A1	Press $N$ Times	Press the blue button $N$ times.	3	$N\in\{1,2,3\}$
A2	Press in Order	Press the [Order] buttons in order.	6	Order (B, G, R)
A3	Wipe the Window	Wipe the [Order] part of the window in order. Each part of the window is wiped from left to right.	6	Order (bottom, middle, top)
A4	Find the Object	Open the drawers in the following order to find the toy hamburger, and stop when you find it in the drawer: top_left, top_right, bottom_right, bottom_left.	4	Target Location
A5	Pick, Place, Press	Pick and place the banana from the [A] plate to the [B] plate, press the orange button, and then pick and place the banana from the [B] plate to the [C] plate.	6	Transfer Path (Red, Blue, White)

Appendix

Table 6: Detailed description of the Real-World Evaluation Suite. The suite consists of five intra-episode tasks (A1–A5) for working memory and five inter-episode tasks (E1–E5) for episodic context, covering a diverse range of non-Markovian reasoning challenges. Each inter-episode task contains two language instructions.

Figure 8: Four Types of Failure Cases. Examples of low-level, detection, order, and termination failures.

	HeLM (Ours)						Keyframe-HLP
Failure Type	A1	A2	A3	A4	A5	Avg	A1	A2	A3	A4	A5	Avg
1. Low-Level	0.0	4.4	0.0	0.0	17.8	4.4	6.7	4.4	0.0	0.0	0.0	2.2
2. Detection	0.0	0.0	3.3	33.3	8.9	9.1	3.3	4.4	40.0	0.0	53.3	20.2
3. Order	0.0	3.3	0.0	0.0	0.0	0.67	0.0	0.0	0	20.8	2.2	4.6
4. Termination	0.0	0.0	0.0	0.0	0.0	0.0	66.7	0.0	0.0	100	0.0	33.3

Appendix

Table 7: Failure Analysis of HeLM and Keyframe-HLP. Detailed breakdown of failure types across intra-episode tasks (A1–A5). HeLM exhibits no termination failures and only rare order violations.

Method	A1	A2	A3	A4	A5	Avg.
HeLM (Ours)	100.0	87.8	96.6	66.7	55.6	81.3
Keyframe-HLP	0.0	86.7	20.0	0.0	40.0	29.3
PCMB- $\pi_{0}$	0.0	0.0	0.0	0.0	0.0	0.0
Naïve- $\pi_{0}$	0.0	0.0	0.0	0.0	0.0	0.0

Appendix

Table 8: Strict task success rate on intra-episode tasks. A trial is counted as successful only if all subtasks are completed in the correct order with proper termination. Compared to partial subtask success , baselines collapse to near-zero on most tasks, revealing systematic order and termination failures masked by the diagnostic metric.

	Task length (number of subtasks)
Task	1	2	3	4	5	6	7
Press N Times (A1)	100	100	100	100	100	100	100
Press in Order (A2)	100	100	93	80	80	77	88

Appendix

Table 9: Success rates across task lengths. We evaluate HeLM on extended variants of A1 (Press N Times) and A2 (Press in Order). Task length denotes the number of required subtasks. Entries report success rates (%).

Method	A1	A2	A3	A4	A5	Avg.
HeLM (base)	100	92.2	96.6	66.6	73.3	85.7
HeLM (Smol)	88.8	70.4	66.6	41.6	25.9	58.7

Appendix

Table 10: Evaluation results on different model combinations. The baseline configuration consists of Qwen2.5-VL (7B) as the HLP and

\pi_{0}

(3.5B) as the LLP, while the efficient variant utilizes SmolVLM (500M) and SmolVLA (450M).

Method	T1	T2	T3	T4	T5	T6	T7	T8	T9	T10	Avg.
Baselines from LIBERO-Mem paper
$\pi_{0}$ (h=1)	50.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	5.0
SlotVLA (h=1)	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
SlotVLA (h=8)	50.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	5.0
Naive E-SlotSSM	50.0	0.0	33.3	0.0	14.3	0.0	0.0	0.0	30.0	20.0	14.8
Our additional evaluation

Appendix

Table 11: Results on LIBERO-Mem benchmark. Top: baseline results from [ 10 ] . Bottom: our additional evaluation with

\pi_{0.5}

baseline and HeLM. The metric follows the official codebase implementation.

Panel of Figure 9 — Figure 9: VLM System prompt used for HLP Detect mode.

Panel of Figure 10 — Figure 10: VLM system prompt used for HLP Update mode.

Figure 11: Breakdown of non-Markovian task. Intra-tasks (A1–A5) and Inter-tasks (E1–E5).

Panel of Figure 12 — Figure 12: Example of Taskspec. This figure shows an example of TaskSpec for task press button in human order (E2).

Panel of Figure 14 — Figure 14: Example of HeLM HLP train data. It contains example of HLP training data of HeLM ( Press the button N times ). There are four types of train data: detect(false), detect(true), update(intra), and update(inter).

Memory-Augmented VLMs

Memory-augmented vision-language models use persistent or retrieved context to improve reasoning across long inputs and interactions.

See all 163 papers

HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation
Apr 20, 2026Zijian Zeng, Fei Ding, Huiming Yang +1Long-Horizon Robotic Manipulation Robot Failure Recovery
Tsinghua University · Alibaba Group · Bengbu University
Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
Aug 5, 2026Houze Xu, Jizhong Li, Ziyi YeMemory-Augmented VLMs Vision-Language-Action Models
Fudan University
Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation
Jul 8, 2026Hongyu Qu, Jianzhe Gao, Xiaobin Hu +6Memory-Augmented VLMs Robotic Manipulation
Nanjing University of Science and Technology · Zhejiang University · National University of Singapore

Apr 20, 2026J/K move · Enter open · S save

HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation

Zijian Zeng, Fei Ding, Huiming Yang, Xianwei Li

Tsinghua University · Alibaba Group · Bengbu University

Long-Horizon Robotic Manipulation Robot Failure Recovery Robotics Vision-Language-Action Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models fail systematically on long-horizon manipulation tasks despite strong short-horizon performance. We show that this failure is not resolved by extending context length alone in the current reactive execution setting; instead, it stems from three recurring execution-loop deficiencies: the memory gap, the verification gap, and the recovery gap. We present HELM, a model-agnostic framework that addresses these deficiencies with three components: an Episodic Memory Module (EMM) that retrieves key task history via CLIP-indexed keyframes, a learned State Verifier (SV) that predicts action failure before execution from observation, action, subgoal, and memory-conditioned context, and a Harness Controller (HC) that performs rollback and replanning. The SV is the core learning contribution: it consistently outperforms rule-based feasibility checks and ensemble uncertainty baselines, and its effectiveness depends critically on access to episodic memory. On LIBERO-LONG, HELM improves task success rate by 23.1 percentage points over OpenVLA (58.4% to 81.5%), while extending the context window to H=32 yields only a 5.4-point gain and same-budget LoRA adaptation remains 12.2 points below HELM. HELM also improves long-horizon performance on CALVIN and substantially boosts recovery success under controlled perturbations. Ablations and mechanism analyses isolate the contribution of each component, and we release LIBERO-Recovery as a perturbation-injection protocol for evaluating failure recovery in long-horizon manipulation.

Abstract

Memory-Augmented VLMs

Explore similar work

HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation

Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models

Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation