Pretrained vision-language-action (VLA) policies fail on memory tasks, behaviour that depends on events observed long before the current frame, because their training data is overwhelmingly Markov and mapping episodic memory directly to action chunks is hard. Yet the same policy, finetuned once to follow a privileged oracle's grounded-subgoal text, nearly solves these tasks: the difficulty lies entirely in mapping memory to a grounded symbolic subgoal. Existing work entrusts this stage to a single VLM that selects keyframes and reasons over raw frames, and falls far short of the oracle: key identities and dynamics are buried in noisy pixels, the reasoning spans hundreds of frames in an unbounded context, and hallucination corrupts the memory itself. We instead present GAMMA (Grounded multi-Agent Memory with Mechanical Audit), which builds the stage as a collaborating two-agent memory system: a detector tool (SAM-3) for spatial grounding, a VLM writer that distils each replan window into one grounded text line, an append-only text memory bank, and a VLM reasoner that reads the bank and emits the next grounded subgoal at the oracle's cadence, wrapped in a propose-verify harness that admits, defers, corrects, or rejects every agent claim against mechanical visual evidence, at training-corpus construction and at deployment alike. On a sixteen-task memory-manipulation suite GAMMA recovers a large fraction of the oracle ceiling with the policy held fixed, reaching the ceiling on several tasks, and transfers with unchanged contracts to a second benchmark.
Figures & tables
Figure 1: The memory medium is the key difference. Top: a MemER-style single-VLM keyframe pipeline remembers pixels: an opaque selector drops decisive frames and the VLM emits a stale, wrong subgoal. Middle: Gamma remembers verified grounded text: the propose–verify harness gates every writer line into the append-only bank (lines from a real episode) and the reasoner looks up exactly the line it needs. Both feed the same fixed subgoal-conditioned π0.5 : what differs is the memory system in between. Bars: suite-average success vs. the privileged oracle.
Figure 2: Overview of Gamma . Each replan tick, the SAM-3 detector grounds the window’s frames; the writer (Agent-1) distils them into at most one grounded event line; the reasoner (Agent-2) reads the append-only bank, without pixels, and emits the next grounded subgoal. The agents’ only interface is the bank; every claim must pass the propose–verify harness against visual evidence.
Configuration
Memory mechanism
Success
π0.5 end-to-end
none
0.179
π0.5 + memory module (FrameSamp+Modul)
latent frame memory
0.445
MemER-style keyframe pipeline
VLM keyframe selection
0.424
π0.5 + oracle ungrounded subgoals
privileged, no coordinates
0.496
π0.5 + oracle grounded subgoals
privileged
0.841
Table 1: The policy is not the bottleneck, and grounding sets the ceiling: oracle grounded subgoals take π0.5 from 0.179 to 0.841 ; the same oracle without coordinates reaches 0.496 , memory injected into the policy 0.445 . Our task is to supply the grounded stream without privilege (all rows RoboMME-reported).
Figure 3: The harness on real claims. Left (ButtonUnmaskSwap), the claim “completed: press the first button”: the button surviving every window frame refutes it— defer , and the policy retries (top); its disappearance under the gripper registers the press— admit (bottom). Right (RouteStick, stroke 3): the writer says “right target, clockwise”; from the start pad A , the next pad B and the stroke’s ink, the reader finds the ink bulging counterclockwise (61 of 65 points) with B to the left. Both words are contradicted: the writer’s line is reject ed, the reader’s admitted.
Temporal
Spatial
Object
Procedural
Method
PickX
BinF
SwingX
StopC
VU
BU
VUS
BUS
PH
VRP
VPB
VPO
MC
IP
PL
RS
Avg
e2e
π0.5
42.9
30.0
35.6
6.7
20.4
22.2
18.7
6.7
11.3
0.4
31.1
25.8
26.0
1.6
2.9
4.7
17.9
SAM2Act+
76.0
40.0
25.3
0.0
27.3
32.0
18.0
26.7
17.3
5.3
24.7
20.0
29.3
0.0
0.0
0.0
21.4
mem-VLA
TokenDrop+Modul
83.6
34.4
86.0
5.3
28.2
29.3
28.4
21.3
21.3
22.0
59.6
36.0
62.0
7.1
32.4
51.6
38.0
FrameSamp+Modul
87.3
39.6
92.0
42.0
32.7
25.1
24.4
18.2
22.9
30.4
60.0
32.0
77.8
7.6
53.6
66.7
44.5
VLM-mem
MemER
79.3
56.7
59.3
0.0
81.3
72.0
38.0
21.3
70.7
25.3
30.0
26.0
82.7
6.7
16.7
12.0
42.4
Table 2: Closed-loop success (%) on the sixteen memory tasks, grouped as in RoboMME Table 3: end-to-end VLAs, memory-VLA architectures, VLM symbolic-memory pipelines, privileged oracles (gray). Baseline and oracle rows are RoboMME-reported; both Gamma rows are means over three serving seeds at 30 episodes/task/seed behind the oracle rows’ own fixed π0.5 (per-seed numbers in Appendix E ). The 0.8B agents row swaps only the agents’ base model (Section 5.3 ). Bold: best non-privileged result per task, excluding the 0.8B row.
Temporal
Spatial
Object
Procedural
Configuration
PickX
BinF
SwingX
StopC
VU
BU
VUS
BUS
PH
VRP
VPB
VPO
MC
IP
PL
RS
Avg
Writer / reasoner base: Qwen3.5-9B
Gamma (full harness)
98.9
71.1
68.9
35.5
86.7
84.4
75.6
35.6
71.1
65.6
75.5
66.7
62.2
3.3
98.9
56.7
66.0
w/o defer (press-verify gate)
95.9
70.1
67.2
36.8
86.1
56.7
75.4
13.1
71.6
67.9
73.2
65.6
64.4
6.5
97.8
55.3
62.7
w/o reject (extractor-owned claims, duplicates)
95.6
71.3
66.7
13.5
85.7
83.2
76.1
36.1
70.4
65.3
74.8
63.1
67.2
3.3
3.3
6.7
55.1
w/o correct (grounding rebinding)
98.3
72.3
66.7
36.1
85.9
86.2
76.4
35.1
70.9
61.3
71.4
60.2
61.2
3.3
97.5
57.3
65.0
Table 3: Harness ablation on two axes, in the layout of Table 2 . Top : one verdict disabled at a time at 9B (three seeds, 30 episodes/task/seed); each coloured cell loses at least four points, in the removed verdict’s colour— defer blue, reject orange, correct violet. Bottom : only the base model is replaced; grey marks the same four-point loss on the no harness rows.
Figure 4: The audit loop: every tick is logged as text (left, a real ButtonUnmaskSwap tick whose press claim is deferred because the button is still detected); each failed episode is attributed mechanically to one defect class (middle); the defects become the findings on the right, each turned into a predicate validated by its closed-loop delta.
RoboMemArena, task / subtask success (%)
Transferring
Occlusion
Counting
Sequence
Average
π0.5 (no subgoals)
20.0 / 42.8
12.7 / 17.2
14.3 / 50.9
60.0 / 71.6
21.5 / 38.7
MemoryVLA ( Shi et al., 2025 )
15.0 / 37.2
7.3 / 13.1
14.3 / 55.1
37.5 / 65.2
15.0 / 35.3
MemER ( Sridhar et al., 2025 )
20.0 / 36.1
16.4 / 33.2
27.1 / 65.1
65.0 / 79.1
27.3 / 49.1
PrediMem ( Lei et al., 2026 )
22.5 / 45.2
27.3 / 38.4
45.7 / 69.3
72.5 / 89.5
38.5 / 55.2
Gamma (ours)
27.4 / 37.6
19.2 / 50.1
52.3 / 67.5
80.7 / 90.3
38.8 / 59.0
ground-truth subtask feed (privileged)
32.5 / 54.8
33.6 / 49.8
51.4 / 75.6
85.0 / 92.3
46.1 / 64.8
Table 4: RoboMemArena by task family (the benchmark’s Table 2 layout): baseline and privileged rows as reported; Gamma on held-out layouts, 50 episodes per task. Bold: best non-privileged result per column.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: The propose–verify harness (companion to Algorithm 1 ), one example verdict per claim type: admit commits the claim and advances the plan; defer holds it with no effect—the same subgoal keeps being issued until its evidence appears; correct rebinds a mis-grounded coordinate to the evidence-consistent detection ( reject omitted for clarity). The same contract generates the training corpus.
Figure 6: Three more harness cases from real episodes (9B agents, serving seed 7), in the grammar of Figure 3 . A , StopCube: the press is an extractor-owned claim—the cube’s arrivals at the target are counted from detections, the next one is predicted from the observed period, and the press fires at the measured oracle lead; the reasoner’s own press claims are rejected until then. B , PatternLock: the dot-route reader takes the lit-dot sequence as stroke endpoints; the writer’s words are wrong on both strokes it wrote and it never wrote the third, so its lines are rejected and the reader’s admitted. C , VideoUnmaskSwap: the shuffle is written as a chain of six move lines, each admitted only with a measured container displacement in its own window, plus the summary; the reasoner’s lookup at execution matches the oracle’s coordinate to within one pixel.
Task (seed 7, 30 episodes)
strokes reached
sense correct
side correct
both
episodes fully correct
RouteStick
79
69
73
66
23/30
PatternLock
85
–
–
80
27/30
Appendix
Table 5: Reader record vs. the live oracle on the held-out evaluation demonstrations: strokes are counted once the oracle issues them; for PatternLock a stroke is correct when its eight-way word matches. On the development annotations the readers score 33/33 strokes and 10/10 episodes respectively.
Guard
Claim type
Evidence
Verdict
Triggers (seed 7)
Press-verify gate
progress (press)
pressed object still detected in the window
defer
not logged
Press latch
progress (press)
object occluded under the arm for consecutive frames
admit
not logged
Arrival-timed press (StopCube)
progress (press)
counted arrivals + predicted period; writer’s press rejected until then
reject / extractor
not logged
Stale-coordinate re-snap
grounding
cited coordinate matches a demo location but no current detection; nearest class-compatible detection
correct
not logged
Container-coordinate authority
grounding
deterministic fold of the admitted move chain disagrees with the cited container
correct
not logged
Direction words (RouteStick)
demo record
reader’s stroke sense/side supersede the writer’s
reject writer
1,640 words / 30 eps
Appendix
Table 6: Serve-side guards of the released server (9B configuration). Word counts are the number of writer/reasoner direction words overridden by the reader over the 30 seed-7 episodes of each task (every tick re-issues the stroke, so one stroke accrues many overrides).
Task
v1
v2
v3
v4
v5
v6
v7
v8
PickXtimes
9
6
8
8
9
10
9
10
BinFill
9
2
7
6
8
8
6
7
SwingXtimes
7
3
6
1
6
5
8
8
StopCube
3
3
1
1
2
5
5
5
VideoUnmask
10
8
10
10
10
10
10
8
ButtonUnmask
5
8
6
7
9
7
7
9
Appendix
Table 7: Closed-loop screening by round: successes out of 10 episodes per task (seed 7). v1 introduced the two-agent system with the SAM-3 detector; v2 moved completion registration to the reasoner (reverted in v3); v4 recorded shuffles as move chains; v5 exposed the train/serve geometry mismatch ( † stopped after 5 and 3 episodes); v6 added the arrival predictor and the fold authority; v7 is the released corpus and checkpoints; v8 added the deterministic demonstration readers and the press latch. The v8 column is the first 10 episodes of the final seed-7 run (Table 14 ).
Task
Ticks
Content%
Gnd med px
>14 px%
Trans.
Adopted%
Lag
Premature%
ButtonUnmask
237
60.3
5.7
37.8
9
100.0
2
10.0
ButtonUnmaskSwap
680
37.4
7.8
41.3
17
100.0
1
48.3
StopCube
154
72.1
5.0
0.0
20
50.0
0
0.0
VideoUnmaskSwap
77
100.0
38.2
90.9
0
–
–
0.0
VideoRepick
114
93.9
25.6
69.5
6
100.0
1
0.0
VideoPlaceOrder
124
98.4
7.1
13.1
10
100.0
0
0.0
Appendix
Table 8: Tick-level diagnosis of GroundSG+QwenVL (10 episodes/task, one seed). Binding tasks: near-perfect content, off-instance grounding (VUS: 91% of coordinates beyond the radius—and its episodes never reach the first oracle transition). Direction tasks: content collapse. Press-counting: missed (StopCube adopts half the transitions) and premature switches (BUS: 48% premature despite the implementation’s hard-coded hold periods).
Figure 7: Three-way tick-level diagnosis (10 episodes/task, seed 7). (a) Grounding error on content-matched ticks for the three binding tasks: both single-VLM pipelines put most cited coordinates beyond the 14 px wrong-instance radius (dashed)—the right words about the wrong object—while Gamma ’s coordinates are detector readings carried by admitted bank lines and land within a few pixels. (b) ButtonUnmaskSwap episode 1, each system against the live oracle of its own run (Table 11 ): MemER declares the second press five ticks before its oracle registers the first and never returns; Gamma ’s writer claims the first press at step 128, the claim is deferred while the button is still detected and admitted one tick later, and the stream follows the oracle thereafter. (c) The family profile of Table 9 : binding fails by grounding, press-counting by timing, direction by content.
Family (tasks)
Tick-level metric
GroundSG+QwenVL
MemER
Gamma
Binding (VUS, VRP, VPO)
coordinates beyond wrong-instance radius (%) ↓
51.7
36.2
13.5
Press-counting (BU, BUS)
premature subgoal switches (%) ↓
38.5
67.4
17.0
Direction (PL, RS)
subgoal text matches oracle (%) ↑
6.5
19.5
100.0
Appendix
Table 9: Tick-level diagnosis of the single-VLM pipelines and Gamma on their shared tasks (10 episodes/task, seed 7): each fails at the stage a harness verdict owns.
GroundSG+QwenVL
MemER
Gamma
Task
content
>14 px
prem.
content
>14 px
prem.
content
>14 px
prem.
ButtonUnmask
60.3
37.8
10.0
45.6
7.7
64.9
70.6
0.0
28.6
ButtonUnmaskSwap
37.4
41.3
48.3
50.5
35.9
69.2
57.7
2.5
12.8
StopCube
72.1
0.0
0.0
–
–
–
89.1
0.0
23.8
VideoUnmaskSwap
100.0
90.9
0.0
100.0
81.2
0.0
82.6
1.0
40.0
VideoRepick
93.9
69.5
0.0
74.4
24.4
5.3
83.1
11.9
8.3
Appendix
Table 10: Per-task tick-level diagnosis of the three systems on the tasks each was run on (10 episodes/task, seed 7): per-tick subgoal content match with the live oracle (%), share of content-matched coordinates beyond the 14 px wrong-instance radius (%), and share of predicted switches that are premature (%). Content is a strict per-tick string match: Gamma ’s deferred switches lower it on the binding tasks while it adopts 91 – 100% of oracle transitions with zero median lag. “–”: not run or no coordinate in the subgoal.
step
live oracle (this run)
predicted subgoal
writer claim → verdict
GroundSG+QwenVL
0
press the first button at ⟨64,154⟩
press the first button at ⟨62,146⟩
128
press the second button at ⟨61,103⟩
press the second button at ⟨58,99⟩
192
pick up the container at ⟨109,84⟩ that hides the red cube
press the second button at ⟨58,99⟩
288
pick up the container at ⟨109,84⟩ that hides the red cube
pick up the container at ⟨94,118⟩ that hides the red cube
outcome: failure (episode ends at step 1312)
Appendix
Table 11: Case study, press-counting: ButtonUnmaskSwap episode 1 (seed 7) as seen by the three systems. Live oracle and prediction per tick; for Gamma , the writer’s claims and their verdicts.
step
live oracle (this run)
predicted subgoal
writer claim → verdict
GroundSG+QwenVL
0
pick up the correct cube at ⟨83,101⟩ for the first time
pick up the correct cube at ⟨82,123⟩ for the first time
outcome: failure (episode ends at step 128)
MemER
0
pick up the correct cube at ⟨83,101⟩ for the first time
pick up the correct cube at ⟨89,124⟩ for the first time
outcome: failure (episode ends at step 112)
Appendix
Table 12: Case study, binding: VideoRepick episode 8 (seed 7). Both single-VLM pipelines pick a cube at the demonstration-time position; Gamma ’s bank tracks the correct cube’s two moves.
step
live oracle (this run)
predicted subgoal
writer claim → verdict
GroundSG+QwenVL
0
move to the nearest right target by circling around the stick clockwise
move to the nearest right target by circling around the stick counterclockwise
outcome: failure (episode ends at step 48)
MemER
0
move to the nearest right target by circling around the stick clockwise
move to the nearest right target by circling around the stick clockwise
48
move to the nearest right target by circling around the stick counterclockwise
move to the nearest right target by circling around the stick clockwise
Appendix
Table 13: Case study, direction: RouteStick episode 0 (seed 7). The stroke sense is read at chance by the VLM pipelines and mechanically by Gamma ’s reader; progress is advanced by admitted completion claims.
Figure 8: A dropped decisive event, on one held-out VideoUnmaskSwap episode (seed 7, episode 9) seen by both systems. Top: the demonstration shuffles the three containers six times over frames 73–109 (SAM-3 container detections in red). Bottom left: MemER’s keyframe memory holds only execution frame 0 at every tick and never a demonstration frame (the demonstration is attached to each query as a raw video clip instead); its prediction points at the container standing where the red cube’s container started , now hiding the blue cube, 34 px from the oracle. Bottom right: Gamma ’s bank carries the six moves as individually admitted lines and the resulting state; the reasoner’s lookup lands within 2 px of the oracle.
π0.5 executor, plan fed on benchmark stage predicates (privileged)
3.5 / 23.6
2.4 / 40.7
11.4 / 36.5
35.0 / 68.1
10.0 / 41.1
π0.5 executor, plan fed on segment-completion conditions (privileged, 10 ep.)
7.5 / 24.6
4.5 / 41.9
22.9 / 43.3
65.0 / 83.8
19.2 / 46.1
Gamma full pipeline: writer + harness, claims settled two ticks
27.4 / 37.6
19.2 / 50.1
52.3 / 67.5
80.7 / 90.3
38.8 / 59.0
Gamma harness predicates only, no writer
27.5 / 37.3
18.9 / 49.2
52.0 / 67.2
81.0 / 90.7
38.7 / 58.6
Gamma writer + harness, claims admitted on the tick they are verified (10–20 ep.)
1.2 / 20.2
3.2 / 45.1
24.3 / 42.1
72.5 / 86.2
19.2 / 46.8
Appendix
Table 16: RoboMemArena by task family (task / subtask success, %). Family sizes: sequence 4 tasks, counting 7, occlusion 11, transferring 4. The upper block is our protocol; the lower block reproduces the authors’ Table 2 under their own protocol for reference.
Writer (autoregressive rollout)
Reasoner
Task
exact%
recall%
precision%
known%
ticks
exact%
PickXtimes
99.2
97.6
97.6
97.5
260
80.0
BinFill
94.4
76.4
98.6
94.3
396
68.2
SwingXtimes
95.4
87.0
95.3
99.1
262
94.7
StopCube
80.0
65.2
91.9
96.3
240
88.7
VideoUnmask
100.0
100.0
100.0
100.0
126
100.0
Appendix
Table 17: Open-loop quality of the released agents on the held-out validation split ( 5,904 ticks, 1,211 event ticks). The direction tasks are where the writer’s own lines are weakest, which is why their demonstration records are reader-owned at deployment (Appendix A ).
Class
Text prompt
Class
Text prompt
red_cube
a small red block on the table
target
a flat square target marker on the table
green_cube
a small green block on the table
bin
a white open box
blue_cube
a small blue block on the table
peg
a long cylindrical peg on the table
container
white container box
stick
a thin stick standing on the table
button
a grey push button on the table
arm
robot arm
Appendix
Table 18: SAM-3 text prompts per object class (score threshold 0.35 ).
Figure 9: Frames behind the PickXtimes bank above (episode 3, agent view; markers are the coordinates the admitted line cites). Each completed: pick up line is admitted at the tick that shows the cube leaving the table in the gripper, and each place line at the tick the cube reaches the target; counting the fourth pick is then a count over admitted lines.
Figure 10: Frames behind the ButtonUnmaskSwap bank above (episode 1). Left: the containers are admitted once they are detected over the cubes. Middle: after the swaps, the summary line records which container hides which cube, and its three coordinates are the detections in this window. Right: the press completion is admitted at the tick the button is occluded under the gripper—the evidence the defer gate waits for.
Figure 11: Frames behind the VideoRepick bank above (episode 1); the marked cube is the same physical object in all three. Left: the cube the demonstration picks, at ⟨106,136⟩ . Middle: the end of the demonstration, after the shuffle has moved it to ⟨135,114⟩ , the coordinate the tracker lines carry. Right: the admitted pick at execution, on that coordinate rather than on the demonstration-time one.
Figure 12: Frames behind the RouteStick bank above (episode 13). Left and middle: the two demonstration strokes whose turning sense the deterministic reader extracts from the ink the stick leaves (the pale arc), and on which the writer’s own words measured at chance. Right: the arm retracing stroke 1 at execution, with the arc-tip detection the writer records.
Frozen vision-language-action (VLA) policies offer broad manipulation skills but execute open-loop action chunks without tracking task progress, so the agent cannot reliably decide whether to continue, retry, or terminate. External memory is a natural remedy, yet we find it can be harmful when attempted actions are recorded as completed progress: transient execution failures become persistent task-state errors, and such a memory can underperform no progress memory at all. We propose Achievement-Grounded Memory (AGM), a lightweight closed-loop framework for frozen VLA policies. AGM represents a task as a static subgoal sequence with a dynamic progress pointer and advances the pointer on physically verified achievement rather than on attempts. Proprioceptive gripper-load cues decide when to verify; coherent point tracking verifies grasps, and language-conditioned cross-view comparison, read by a single trained 2.43M-parameter verification head, verifies placements. The policy, tracker, and encoder remain frozen, the head is the only trained component, and deployment needs no auxiliary vision-language model. On the RoboMME Counting benchmark, AGM reaches 100.0% on PickXTimes and 84.0% on BinFill, surpassing the strongest memory-augmented baseline by 7.7 points on the four-task average, and the gains carry over to a physical robot, where AGM reaches 100.0% and 82.0%. These results suggest that reliable embodied memory depends more on disciplined state updates than on memory capacity.
Hongbo Gao, Zeyu Ni, Xin Wen +4
Harbin Institute of Technology · The University of Sydney · StellarEdge AI
Vision-language-action (VLA) models have driven rapid progress in robotic manipulation, demonstrating strong fine-grained control and promising performance on long-horizon tasks. However, many existing VLAs lack explicit access to interaction history, making them vulnerable to perceptual aliasing: similar current observations and robot states at different task stages may induce action ambiguity and lower success rate. Existing methods incorporate temporal or progress cues through feature conditioning, action-prior modification, or sampling guidance. However, methods that jointly fine-tune memory modules and the base VLA incur additional policy-training costs, motivating the separation of trainable history-conditioned steering from frozen base-policy refinement. We propose ActMem-VLA, a dual-expert handover architecture that augments a frozen, fine-tuned VLA with a memory plugin comprising a Mamba-based memory module and a lightweight PreAction Expert (PAE). Specifically, Mamba encodes executed-action history into memory that conditions PAE alongside current context. With these inputs, PAE steers task progression during early, high-noise denoising, then passes the partially denoised action to the frozen Action Expert (AE) to refine action details during the remaining low-noise steps. The fine-tuned base VLA remains frozen throughout training, while only the Mamba module and PAE are jointly optimized. On LIBERO-Mem, ActMem-VLA achieves 80.8% average success across all ten tasks, compared with 65.2% for π0.5 and 49.5% for MemoryVLA, while introducing only 3.45% additional parameters. Across four real-world tasks, it improves the average success rate over π0.5 by 28.8%.
Yaxin Zhao, Dianye Huang, Chenwei Wang +2
Harbin Institute of Technology, Harbin, China. · Medical Intelligence and Robotic Cognition (MIRoC) Lab, Department of Mechanical Engineering, The University of Hong Kong (HKU), Hong Kong SAR, China. · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China
Long-horizon, multi-stage Vision-Language-Action (VLA) policies often require visual information that is no longer present in the current observation. Retaining the complete observation history is computationally expensive and highly redundant, motivating selective visual memory. We introduce RePlanMem-VLA (Replanning-Guided Keyframe Memory for VLA), which turns a policy's own plan revisions into an online memory-write signal. Under receding-horizon execution, consecutive action chunks predict some of the same future actions. RePlanMem-VLA treats unusually large disagreement between these temporally aligned predictions as evidence of a task-relevant event. We first compare 35 disagreement measures across single-task autoregressive decoder-only and Flow-Matching Transformers trained from scratch, then validate the selected signals on pretrained OpenVLA-OFT and π0.5 policies over MIKASA-Robo-VLA tasks. The resulting detectors achieve ROC-AUC of 0.981-0.997. We then use the selected signals in RePlanMem-VLA to trigger updates to a bounded visual memory, with a robot-masked write filter suppressing redundant frames. In the evaluated memory-retention experiments, RePlanMem-VLA preserves task-relevant observations until the later decision point in 100% of episodes. Real-robot replay data further show that event selection and redundancy suppression extend beyond simulation. These results establish plan revision as a policy-internal signal for controlling what a VLA remembers. Website: https://vika787.github.io/RePlanMem/.
Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev +6
MIRAI · Independent Researcher · Central University +2