In long-horizon manipulation, a robot may need to take different actions under identical observations, depending on task progress and the outcomes of earlier attempts. We refer to this ambiguity as decision aliasing. Resolving it requires the policy to maintain memory of its execution history. To reduce the cost of processing full observation histories, recent methods build compact event memories using perceptual, semantic, or kinematic cues. However, these criteria may miss events that matter for task success. We instead propose an outcome-grounded principle: remember events that substantially change the predicted probability of eventual success. Following this principle, we introduce CGEM, a critic-gated event memory that uses a critic to select events with large estimated changes in success probability. Positive and negative changes are encoded as progress and failure records, respectively. Failure records are discarded once the corresponding subtask is completed. In a ten-subtask diagnostic environment with exactly computed success probabilities, CGEM achieves 88.0% success, compared with 0.0% for both a 36-step observation window and a kinematic event memory. Removing failure records reduces success to 1.7%.
Figures & tables
Figure 1: Decision aliasing and the proposed CGEM architecture. (a) Memory distinguishes untried, failed, and completed interactions under the same observation. (b) A critic selects events for a bounded memory of progress and failure records. Both the critic and policy read the memory.
Figure 2: Memory scaling and sparse value changes on ASC . (a) Task success versus N , with the failure-blind ceiling ρN from Proposition 2 . (b) The top 1% of steps carry 79% to 96% of the squared value changes, measured on 3000 reference-policy episodes per N .
N=6
N=10
Memory
SR
SC
SR
SC
None
0.0
11.6
0.0
5.6
Window 12
0.0
14.5
0.0
13.8
Window 36
60.0±3.6
80.6
0.0
44.6
Uniform
0.0
16.9
0.0
10.2
Kinematic
0.0
11.4
0.0
5.6
Table 1: Task performance on ASC . SR and SC are percentages. SR is reported as mean ± sample standard deviation over three seeds. Detector diagnostics are in Table 2 .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Memory
E-rec
Sign
Uniform
3.6
–
Kinematic
12.1
100
Heuristic union
60.3
n/a
Without failure records
28.4
99.4
Without force
85.8
68.3
Without consolidation
85.8
99.3
Appendix
Table 2: Detector diagnostics at N=6 . E-rec is the percentage of ∑tΔVt2 on written steps. Sign is the accuracy (%) of stored progress/failure labels.
Figure 3: What should a robot remember? The jumps in its odds. P denotes a contact-only press and G a grasp-and-place, and subscripts index stations. The exact Vt jumps at progress and failure outcomes, while the home-pose observation (shaded) never changes. CGEM writes the signed outcome jumps and skips the small changes when the robot leaves home. It keeps failure records until the subtask succeeds, which allows the switch to variant B at t=105 , and then deletes them.
Figure 4: Energy-identity check. The line shows p(1−p) and the points show estimates of E∑tΔVt2 from 3000 reference-policy episodes per N , with error bars of ±2 standard errors.
N
Tˉ
p(1−p)
E∑ΔVt2
# {κt≥.05}
2
47.3
.1399
.1379 ± .0039
2.49
4
102.5
.1522
.1498 ± .0045
3.13
6
152.4
.1686
.1667 ± .0051
2.32
8
199.9
.1873
.1959 ± .0055
1.87
10
243.2
.2047
.2065 ± .0056
1.23
Appendix
Table 3: Top: energy identity ( Proposition 1 ), where Tˉ is the mean horizon and ± is one standard error. Bottom: SR (%) at N=6 versus capacity K .
labels
margin κmin−κ0
θ=0.1
Critic
in reward
κ0
κmin
dt
value only
AUROC
recall
FP/min
MC Mode A @1k
no
.066
.015
−.051
−.051
.71
.12
.07
MC Mode 7 @2k
no
.098
.081
−.017
−.076
.83
.74
.43
IQL Mode 7 @6.5k
no
.084
.087
+.003
−.079
.86
.76
.12
MC Mode 9 @2k
yes
.080
.191
+.111
−.071
.74
1.00
.01
IQL Mode 9 @7k
yes
.066
.221
+.155
−.071
.72
1.00
.04
Appendix
Table 4: Event-detection metrics of real critics (smoothed dt , 87 test episodes, 42 failure windows). “Labels in reward” marks critics whose reward contains the evaluation annotations. AUROC is per step, with −dt scoring steps inside annotated windows against all other test steps.
Figure 5: Failure-window recall against false alarms per expert minute as the threshold varies for smoothed dt . Solid lines show the full TD residual, and dashed lines show its value-only component.
Long-horizon robot manipulation requires policies to track completed subtasks and critical interaction events. However, existing memory mechanisms heavily rely on external models or predefined update rules. To address this, we propose OnEvoMemory, a value-guided memory module for pretrained robot policies. It maintains recent context, high-value experiences, and salient transitions, while learning which experiences should be retained from trajectory outcomes. Offline demonstrations initialize the memory prior, whereas successful and unsuccessful online rollouts refine memory selection, helping the policy recognize task-stage transitions and avoid repeating completed subtasks. Experiments on long-horizon manipulation benchmarks show that OnEvoMemory improves the performance of the base VLA policy through both offline initialization and online memory evolution.
Zhongxi Chen, Shenqi Zong
Shanghai Jiao Tong University · Tsinghua University
Long-horizon manipulation requires robots to retain task-relevant history while revising conclusions no longer supported by physical evidence. Retaining observations alone does not specify which completed conditions should be withdrawn after a disturbance or which obligations require repair. We introduce REMM, Revisable Embodied Memory for Manipulation, a supervisory memory layer for a frozen G0.5 vision-language-action executor. REMM separates an append-only evidence history from revisable task state and explicitly links derived relations and completed conditions to their supporting facts. When accepted evidence changes a supporting fact, it withdraws dependent conclusions and schedules targeted recovery at verified skill boundaries without updating the executor. Across 51 physical trials, including operator-assisted runs, REMM improves mean recorded task progress over the strongest baseline from 57% to 91% in intervention recovery and from 43% to 70% in sequence recall. Complementary video diagnostics show improved grounding of retained history into the next instruction, including comparisons with π0.5-compatible high-level planner adaptations. These results support revisable memory as a practical interface for continued manipulation without policy retraining. Project: https://revanthsenthil.github.io/remm/.
Robotic manipulation tasks exhibit varying memory requirements, ranging from Markovian tasks that require no memory to non-Markovian tasks that depend on historical information spanning single or multiple interaction trials. Surprisingly, simply extending observation histories of a visuomotor policy often leads to a significant performance drop due to distribution shift and overfitting. To address these issues, we propose Gated Memory Policy (GMP), a visuomotor policy that learns both when to recall memory and what to recall. To learn when to recall memory, GMP employs a learned memory gate mechanism that selectively activates history context only when necessary, improving robustness and reactivity. To learn what to recall efficiently, GMP introduces a lightweight cross-attention module that constructs effective latent memory representations. To further enhance robustness, GMP injects diffusion noise into historical actions, mitigating sensitivity to noisy or inaccurate histories during both training and inference. On our proposed non-Markovian benchmark MemMimic, GMP achieves a 30.1% average success rate improvement over long-history baselines, while maintaining competitive performance on Markovian tasks in RoboMimic. All code, data and in-the-wild deployment instructions are available on our project website https://gated-memory-policy.github.io/.