History-dependent robotic manipulation requires task context and interaction history, as the current observation alone is insufficient to determine the next action. Hierarchical frameworks use task memory for high-level subgoal reasoning and low-level policies for execution. However, memory updates require accurate completion assessment: similar observations may correspond to different execution phases, and prematurely recording completion can mislead subsequent subgoal reasoning. We present EventSync, a hierarchical framework that coordinates task reasoning and action execution through event-driven memory updates. A vision-language model generates subgoals from task plans and interaction history, while a vision-language-action model executes them. A lightweight Event Monitor combines visual observations with end-effector trajectories to determine whether the current subgoal is complete. The framework records the current subgoal in its completed-subgoal history and triggers subsequent reasoning only after obtaining completion evidence. With visual history in the executor, EventSync achieves 64.5% overall success on RoboMME, exceeding the best baseline reported in the benchmark by 20.0 percentage points. Further analysis shows that this update mechanism helps build a reliable completed-subgoal history. Experiments on four real-robot tasks with different memory requirements validate the framework's effectiveness in real-world robotic manipulation.
Figures & tables
Fig. 1: Execution-phase ambiguity and erroneous memory updates. The pump head occupies the same position during pressing (a) and release (c), but must move downward in (a) and upward in (c). From (a), continuing to the bottom and returning to the fully released position completes the first pump. Recording a return before reaching the bottom as completion incorrectly triggers the second pump, causing the progress recorded in memory to exceed actual execution progress.
Fig. 2: Overview of EventSync. Top: the Memory Reasoner combines task memory and current context to generate grounded subgoals for the Action Executor. Completion evidence from the Event Monitor updates the completed-subgoal history and triggers subsequent reasoning. Bottom: visual, trajectory, and event-condition tokens feed shared Transformer fusion for completion prediction and recurrent event-increment prediction.
Fig. 3: Real-robot setup and four history-dependent tasks: (a) SwingXTimes, (b) UnmaskCube, (c) ImagePickPlace, and (d) SequencePickPlace. Blue headers indicate reference inputs; green headers indicate execution.
Method
Counting
Permanence
Reference
Imitation
Overall AVG
Bin
Pick
Swing
Stop
AVG
Video
Button
Video
Button
AVG
Pick
Video
Video
Video
AVG
Move
Insert
Pattern
Route
AVG
Fill
Xtimes
Xtimes
Cube
Umsk
Umsk
UmskS
UmskS
HighL
Repick
PlcBtn
PlcOrd
Cube
Peg
Lock
Stick
π0.5
30.0
42.9
35.6
6.7
28.8
20.4
22.2
18.7
6.7
17.0
11.3
0.4
31.1
25.8
17.2
26.0
1.6
2.9
4.7
8.8
17.9
GroundSG + QwenVL
52.0
92.7
7.3
0.0
38.0
88.7
24.0
30.7
14.0
39.3
15.1
25.3
54.0
31.8
31.6
71.6
3.3
6.7
6.0
21.9
32.7
MemER
56.7
79.3
59.3
0.0
48.8
81.3
72.0
38.0
21.3
53.2
70.7
25.3
30.0
26.0
38.0
82.7
6.7
16.7
12.0
29.5
42.4
FrameSamp+Modul
39.6
87.3
92.0
42.0
65.2
32.7
25.1
24.4
18.2
25.1
22.9
30.4
60.0
32.0
36.3
77.8
7.6
53.6
66.7
51.4
44.5
TABLE I: Task success rates (%) on RoboMME. Each category includes four task results and their average; Overall AVG is the average over all 16 tasks. Results for the four baselines are means over repeated evaluations reported in RoboMME [ 9 ] . Bold indicates the highest value in each column. Results are displayed to one decimal place.
Fig. 4: Task-memory and subgoal update strategies with the base executor. Only EventSync uses the Event Monitor. (a) Success versus online high-level calls; the dashed line shows asynchronous success. (b) Mean episode wall-clock time. Calls and time are normalized to Fixed 1:1.
Fig. 5: Memory–execution alignment on SwingXtimes. Top: paired executions; arrows indicate commanded targets. Bottom left: cumulative arrivals recorded in memory (solid) and verified in task order by the simulator (dashed); shading marks memory ahead, and dotted lines mark displayed frames. Bottom right: percentages of premature episodes, premature completion records (one per nonterminal operation), and steps with memory ahead (episode average). Statistics cover 50 paired episodes per method.
Completion monitor
Base executor
FrameSamp+Modul executor
Qwen3-VL-4B
59.8
61.4
Event Monitor (w/o trajectory)
59.0
62.5
Event Monitor
62.0
64.5
TABLE II: Completion monitor comparison on RoboMME. Overall task success rates (%) are displayed to one decimal place. Bold indicates the highest value in each column.
Structured event prediction
High-level visual history
Base executor
FrameSamp+Modul executor
✓
✓
62.0
64.5
×
✓
57.1
61.5
✓
×
49.9
54.5
TABLE III: Memory component ablations on RoboMME. ✓ and × indicate retained and removed components. Values are overall task success rates (%) with the two executors.
Method
Swing XTimes
Unmask Cube
Image PickPlace
Sequence PickPlace
π0.5
1/10
3/10
2/10
2/10
FrameSamp+Modul
5/10
5/10
4/10
2/10
EventSync (w/o Event Monitor)
0/10
7/10
8/10
7/10
EventSync
9/10
7/10
9/10
7/10
TABLE IV: Real-robot task success (successful trials / total trials). Bold indicates the highest count in each task.