Retrieving useful context for a robot requires matching its task, execution stage, and the continuation that should follow. We introduce Robota-Embed, to our knowledge the first vision-language-action (VLA) embedding model to jointly encode all three modalities in both current-state queries and recorded future continuations. Predictive retrieval training aligns these endpoints, enabling historical continuations to serve as context at inference time. We construct a multi-source corpus of 1,695,684 cleaned training anchors and introduce Robota-Embed-Bench, with 4,000 queries and a shared 8,000-entry gallery. Robota-Embed achieves 47.33% clip-level and 81.77% episode-level R@1, exceeding action-free Gemini Embedding 2 by 22.09 and 33.47 percentage points. Controlled retrieval studies show that removing native action degrades retrieval more than removing vision. We study two complementary applications. Embed2ICL uses retrieved experience for training-free and training-based policy conditioning, increasing success from 47.0% to 56.3% in DROID-Sim and from 86.4% to 92.8% on the LIBERO difficult subset, while improving task progress on real robots. Embed2Evolve uses execution feedback to grow task-local memory and refine context use around a frozen GPT-6 Astra policy. Across five ManiSkill tasks, it achieves 68% success, compared with 24% for GPT-6 Astra Direct and 33% for a harness independently evolved with Omni-Embed-Nemotron-3B under the same protocol. Relative to the latter, it uses approximately 79.5% fewer policy tokens per test rollout, counting both successes and failures. These results show that a unified, action-aware retrieval space can support robot in-context learning and task-local context evolution.
Figures & tables
Figure 1: One joint VLA embedding, two context applications. Robota-Embed jointly encodes vision, language, and native action from human, robot, and simulated experience to retrieve recorded continuations. Left: retrieval accuracy and modality contributions on Robota-Embed-Bench. Right: retrieved demonstrations condition robot policies in Embed2ICL; task-local memory and context use evolve around a fixed GPT-6 Astra policy in Embed2Evolve. Token savings include failed final tests and exclude development; decision/time savings use matched successful pairs. These efficiency comparisons use Direct as the baseline.
Figure 2: Robota-Embed architecture and objectives. Each query is paired with the next four seconds of recorded video and actions. Three objective groups train a joint VLA space.
Figure 3: Multi-source training-data curation. We split episodes before sampling anchors, localize and verify interaction crops, and pair accepted anchors with video and actions from the next four seconds. Query filtering uses only the instruction and current observations.
Clip-level
Episode-level
Model
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
CLIP-style text–video embeddings
CLIP ViT-L/14
1.2
5.8
9.8
4.06
3.2
7.7
11.6
5.98
Perception Encoder B16
2.0
8.1
12.8
5.52
5.0
10.1
15.2
8.01
InternVideo2
1.6
7.3
11.4
4.82
4.1
9.1
13.2
6.94
Unified VL embeddings
Table 1: Retrieval on Robota-Embed-Bench. Four-source average recall and MRR (%) on the shared 8,000-clip gallery. Bold marks column-wise best results.
Figure 4: Selected retrieval diagnostics. (a) The paired future clip ranks first. (b) A nearby clip from the same episode ranks first, while the paired clip ranks seventh. Both top-1 returns are episode-level matches; only (a) is also a clip-level match.
Figure 5: Embed2ICL: one retriever, two policy interfaces. Reranked DROID videos warm up DreamZero’s visual cache; top-1 EgoDex matches supply Context Tokens for Fast-WAM’s native cross-attention. Retrieval vectors select experience; videos or tokens condition the policy.
Reference condition
Cube → Bowl
Can → Mug
Banana → Bin
Average
No warm-up
61
13
67
47.0
Random reference
50
( −11 )
7
( −6 )
51
( −16 )
36.0
( −11.0 )
Omni-Embed-Nemotron-3B
Top-1
57
( −4 )
5
( −8 )
68
( +1 )
43.3
( −3.7 )
+ reranking
62
( +1 )
19
( +6 )
72
( +5 )
51.0
( +4.0 )
Robota-Embed Top-1
63
( +2 )
16
( +3 )
72
( +5 )
50.3
( +3.3 )
Table 2: Training-free DreamZero-DROID in DROID-Sim. Success rates (%); parentheses show percentage-point changes from no warm-up. Reranked conditions share the same agent.
Suite / ID
Abbreviated task
Official release
Embed2ICL
Spatial / S4
Black bowl → plate
46
49
( +6 )
Goal / G3
Bowl → top drawer
42
50
( +16 )
Goal / G9
Wine bottle → rack
45
47
( +4 )
LIBERO-10 / L4
Two mugs → plates
43
44
( +2 )
LIBERO-10 / L6
Mug–pudding arrangement
40
42
( +4 )
Five-task average (%)
86.4
92.8
( +6.4 )
Table 3: Training-based ICL on the LIBERO difficult subset. Successes out of 50 per task; last row: average success (%). Parentheses give percentage-point gains over the official release.
Task
No ICL
Embed2ICL
Training-free: DreamZero
Banana → Bowl
21.67
40.00
( +18.33 )
Training-based: Fast-WAM
Pick Fruit
75.00
79.17
( +4.17 )
Split Cup
66.67
87.50
( +20.83 )
Table 5: Embed2ICL on real robots. Mean task progress on task-specific 0–100 scales over 12 rollouts per task and condition. Parentheses show gains computed before rounding.
Figure 6: Retrieved references and real-robot execution. Each task pairs a retrieved reference (top) with real-robot execution (bottom). References come from DROID for DreamZero/YAM and EgoDex for Fast-WAM. Timestamps are relative to each clip’s start.
Figure 7: Embed2Evolve. Development grows memory and optimizes its use; testing fixes both. GPT-6 Astra proposes actions for a fixed controller. The model comparison replaces Robota-Embed in both phases. Images show the PullCubeTool task.
Task
GPT-6 Astra Direct
GPT-6 Astra + random history
Evolve + Omni-Embed- Nemotron-3B
Embed2Evolve
Poke Cube
45
15
45
80
Stack Cube
10
20
20
60
Lift Peg Upright
30
40
55
65
Pull Cube with Tool
10
10
20
50
Two-Robot Stack Cube
25
15
25
85
Average (%)
24.0
20.0
33.0
68.0
Table 6: Embed2Evolve in ManiSkill simulation. Success (%) over 20 trials per task and condition; mean policy tokens per test rollout, including failures.
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
Source
Domain ID
Action dimensions
Rate (Hz)
EgoDex
3
57
30
DROID
8
10
15
LIBERO
5
10
10
RoboMIND–Franka-dual
12
20
10
Appendix
Table 7: Native action interfaces. Dimensions and rates describe native actions before normalization and padding. Domain IDs route actions to their Cosmos3-Edge projectors.
Objective
Constraint or positive relation
Weight
Margin
Predictive retrieval alignment
Clip-level retrieval
Exact future-clip identity; bidirectional InfoNCE
1.00
–
Multimodal grounding and discrimination
Text auxiliary
Text-only query; matched task
0.15
–
Image auxiliary
Image-only query; matched interaction object
0.20
–
Action auxiliary
Disabled; future identity uses multimodal context
0.00
–
Appendix
Table 8: Robota-Embed training objective. Loss coefficients and ranking margins for each term, organized into the three objective groups in Eq. 2 .
Source domain
Training rows
EgoDex [ 31 ]
1,345,160
DROID [ 32 ]
26,563
LIBERO [ 33 ]
136,671
RoboMIND–Franka-dual [ 34 ]
187,290
Total
1,695,684
Appendix
Table 9: Training-data composition. Filtered anchor-level training examples by source.
Domain
R@1
R@5
R@10
MRR
Median rank
Mean rank
Clip-level retrieval
EgoDex
36.70
96.60
99.30
62.22
2
2.68
DROID
74.10
99.00
100.00
85.27
1
1.41
LIBERO
38.50
87.00
96.60
60.33
2
2.92
RoboMIND-FD
40.00
85.00
91.90
60.39
2
3.87
Average
47.33
91.90
96.95
67.05
–
2.72
Appendix
Table 10: Robota-Embed retrieval by source. Recall and MRR are percentages; lower mean ranks are better. Overall median ranks are 2 for clip retrieval and 1 for episode retrieval.
Clip-level
Episode-level
Domain
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
VLM2Vec-V2
EgoDex
32.40
92.00
99.20
56.90
78.13
97.81
99.88
86.04
DROID
58.70
97.80
99.90
75.15
99.19
100.00
100.00
99.51
LIBERO
4.70
16.20
28.10
12.68
10.75
30.44
46.25
22.01
RoboMIND-FD
0.40
1.40
3.00
1.96
1.19
3.94
6.94
3.95
Appendix
Table 11: Action-free VL baselines by source. Recall and MRR are percentages on the shared gallery. Average weights sources equally; episode-level metrics first average within each episode.
Clip-level
Episode-level
Setting
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
Original
47.33
91.90
96.95
67.05
81.77
94.91
98.06
86.96
Candidate text masked
12.78
33.65
43.10
23.06
23.17
36.83
46.69
30.32
Same-instruction gallery
47.62
92.45
97.25
67.46
82.53
95.23
98.20
87.55
Appendix
Table 12: Aggregate benchmark-validity controls. Candidate-text masking and same-instruction retrieval use a fixed model. Recall and MRR (%) are averaged equally across four sources.
Candidates
R@1
R@5
Domain
Mean
Median
Model
Random
Model
Random
EgoDex
4.06
4
37.60
29.76
98.40
95.09
DROID
2.71
2
74.20
54.64
99.20
97.46
LIBERO
54.51
50
38.60
1.90
87.00
9.50
RoboMIND-FD
473.59
342
40.10
0.24
85.20
1.19
All domains
133.72
25
47.62
21.63
92.45
50.81
Appendix
Table 13: Same-instruction gallery difficulty. Candidate counts and clip-level recall (%) within query-specific galleries. Random recall uses the same restricted candidates. The final row averages source means and recalls equally; its candidate median is the cross-source aggregate. RoboMIND-FD denotes RoboMIND–Franka-dual.
Clip-level
Episode-level
Domain
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
Candidate text masked
EgoDex
4.00
14.30
21.40
9.73
9.00
16.81
24.75
13.62
DROID
4.40
13.30
21.20
9.69
6.50
16.81
25.25
12.57
LIBERO
13.40
36.40
49.30
25.03
23.19
39.56
53.87
32.09
RoboMIND-FD
29.30
70.60
80.50
47.77
54.00
74.12
82.88
63.01
Appendix
Table 14: Benchmark-validity controls by source. Recall and MRR are percentages. Candidate-text masking preserves video and action; same-instruction retrieval restricts each query’s gallery. Original-protocol results are in Table 10 . Episode-level metrics use the within-episode and per-source reducer from Section 3.3 .
Setting
Clip R@1
Episode R@1
Cosmos3-Edge 4B (frozen readout)
0.65
1.17
Robota-Embed
47.33
81.77
Appendix
Table 15: Retrieval adaptation. Four-source average retrieval (%) on the shared Robota-Embed-Bench gallery. The frozen readout and Robota-Embed use the same VLA query/target fields.
Trainable scope
R@1
MRR
Head only
13.8
24.00
Backbone + head
78.7
88.00
Appendix
Table 16: Trainable scope. Clip-level retrieval on the controlled DROID split (%). Both rows use a learned retrieval head; the second also updates the backbone.
Objective (Eq. 2 )
R@1
MRR
Lalign
75.6
86.20
Lalign+Lreg
77.0
87.10
Lalign+Lground+Lreg
78.7
88.00
Appendix
Table 17: Objective groups. Clip-level retrieval on the controlled DROID split (%). Each row adds an objective group from Equation 2 .
Visual input
R@1
MRR
w/o vision (retrained)
76.8
86.69
Full frame
77.9
87.40
SAM3 object zoom
78.7
88.00
Appendix
Table 18: Visual input. Clip-level retrieval on the controlled DROID split (%). All rows are retrained for their stated visual interface.
Setting
Input
Clip R@1
Δ clip R@1
Full VLA
VLA
78.7
—
No action
VL
59.8
−18.9
No vision
LA
76.8
−1.9
No language
VA
20.2
−58.5
Appendix
Table 19: Training-time modalities. Matched DROID retraining after removing one modality (%). V/L/A denote vision, language, and action.
Query input
Clip R@1
Episode R@1
Δ clip R@1
Full VLA
47.33
81.77
—
VL (no action)
20.08
37.36
−27.25
LA (no vision)
23.00
39.50
−24.33
VA (no language)
7.13
14.73
−40.20
L (text only)
16.23
31.09
−31.10
V (image only)
1.68
3.66
−45.65
Appendix
Table 20: Inference-time modality availability. Four-source average retrieval (%) with the final checkpoint and gallery fixed. Only the query inputs are changed. V/L/A denote vision/language/action; Δ gives the percentage-point change in clip-level R@1 from Full VLA.
Query setting
Input
R@1
R@5
R@10
MRR
Δ R@1
Full query
VLA
78.7
99.4
100.0
88.00
0.0
Action masked
VL
50.7
90.3
94.9
67.60
−28.0
Image masked
LA
60.9
91.8
96.1
74.17
−17.8
Text masked
VA
1.9
7.2
12.0
5.41
−76.8
Appendix
Table 21: DROID query-only masking. Clip-level recall and MRR (%) with the model and gallery embeddings fixed. Δ R@1 gives the percentage-point change from the full VLA query.
Gallery
Top-10 search (ms)
Encode + retrieve latency (ms)
Single-stream QPS
Est. GPU memory (GB)
1M
2.66
160.36
6.24
11.29
2M
5.14
162.84
6.14
15.39
5M
12.60
170.30
5.87
27.69
10M
25.04
182.74
5.47
48.19
15M
37.47
195.17
5.12
68.69
Appendix
Table 22: Retrieval scaling on one NVIDIA A100. Measured latencies cover Top-10 search alone and query encoding plus search. Throughput is derived from the latter; GPU memory is estimated.
Figure 8: Selected qualitative evidence across embodiments. Each query searches only the illustrated target source. The top-ranked clips share placement or removal roles with their queries.
Figure 9: Training-free Embed2ICL. Robota-Embed retrieves candidate VLA clips, and the Reranking Agent selects a task-relevant reference containing completion evidence. Its video initializes DreamZero-DROID’s visual key–value cache. The policy remains frozen while live observations update the cache during execution.
Figure 10: DROID references for training-free Embed2ICL. Each row shows one task’s fixed reference from a single camera, before, at, and after the detected final gripper release. DreamZero-DROID receives all three synchronized four-second video streams.
Figure 11: Training-based Embed2ICL. Offline encoding caches each candidate’s retrieval vector, Context Tokens, and context mask. Online cosine top-1 search selects a reference and loads its tokens and mask. The prefilling adapter projects the tokens to Fast-WAM’s context width and appends them to the native text and proprioceptive conditioning for cross-attention.
Figure 12: LIBERO G3: reference and final states. Successes out of 50 per condition.
Suite / ID
Official release
Side branch
Embed2ICL
Spatial / S4
46
46 ( 0 )
49 ( +6 )
Goal / G3
42
44 ( +4 )
50 ( +16 )
Goal / G9
45
44 ( −2 )
47 ( +4 )
LIBERO-10 / L4
43
43 ( 0 )
44 ( +2 )
LIBERO-10 / L6
40
43 ( +6 )
42 ( +4 )
Five-task average (%)
86.4
88.0 ( +1.6 )
92.8 ( +6.4 )
Appendix
Table 23: LIBERO integration ablation. Successes from 50 trials per task and condition; the final row gives the five-task average (%). Parentheses show percentage-point changes from the official release; bold marks the better ICL setting.
Task
No ICL
Side branch
Embed2ICL
Pick Fruit
75.00
77.08 ( +2.08 )
79.17 ( +4.17 )
Split Cup
66.67
75.00 ( +8.33 )
87.50 ( +20.83 )
Appendix
Table 24: Real-robot integration ablation. Mean normalized task progress (%) from 12 rollouts per task and condition. Parentheses show percentage-point changes from Fast-WAM without ICL.
Integration
Random tokens
Retrieved tokens
Side branch
85.6
88.0
Embed2ICL
88.0
92.8
Appendix
Table 25: Retrieved versus random Context Tokens. Each integration is evaluated on the LIBERO difficult subset with 50 trials per task and condition. Results are five-task average success rates (%).
Component
Configuration
Policy
GPT-6 Astra, reasoning effort low
Simulator
ManiSkill 3.0.1; SAPIEN 3.0.3
Current RGB
One 256×256 image per camera per decision
Cameras
Four for single-arm tasks; five for TwoRobotStackCube
Control rate
20 Hz; two ticks per primitive (0.1 s simulation time)
Decision budget
At most eight primitives per GPT decision
Appendix
Table 26: Embed2Evolve runtime and observation interface. Simulation time is distinct from client execution time, which includes API waiting.
Preset
k
Cadence δ
History H
α
β
A
2
2
12
0.10
0.03
B
1
4
6
0.10
0.10
C
2
1
8
0.15
0.05
Appendix
Table 27: Initial context configurations. All presets enable historical images and are evaluated on the same ten development seeds.
Figure 13: From Direct control to a selected harness. PullCubeTool’s R4 → R5 → R8 parent–child sequence grows memory while reducing retained history and retrieval frequency. The bottom filmstrips show the same fresh-test initialization completed successfully by both systems. Task execution time includes API waits.
Task
Round
Episodes
Anchors
k
Cadence
H
α/β
Poke Cube
8
80
1,610
2
2
6
.15/.05
Stack Cube
2
20
454
2
2
12
.10/.03
Lift Peg Upright
6
60
1,116
2
1
6
.20/.05
Pull Cube with Tool
8
80
1,667
2
3
4
.15/.05
Two-Robot Stack Cube
5
50
1,126
2
3
6
.10/.03
Appendix
Table 28: Selected Embed2Evolve harnesses. Bank sizes count development episodes and indexed anchors. Cadence counts GPT decisions; H counts retained action/feedback pairs.
Figure 14: Task-wise development trajectories. Success uses ten trials per round; decisions average that round’s successful trials. Hollow diamonds mark selected harnesses. Lines connect evaluated candidates, not necessarily parent–child configurations.
Omni-Embed- Nemotron-3B
Robota-Embed
Task
Completed
Selected
Completed
Selected
Poke Cube
8
6
8
8
Stack Cube
6
3
4
2
Lift Peg Upright
8
4
8
6
Pull Cube with Tool
3
1
8
8
Two-Robot Stack Cube
7
5
8
5
Appendix
Table 29: Independent development under matched search rules. Completed rounds count all numbered rounds; selected rounds identify the configuration and memory frozen for testing.
Metric
Test population
Five-task aggregation
Task success
All 20 trials per task in all four conditions
Mean of task success rates.
Policy decisions and execution time
Matched initializations successful under Direct and Embed2Evolve
Mean of task-wise relative reductions; speedup averages Direct/Embed2Evolve time ratios.
Policy tokens
All 20 trials per task for Direct, evolution with Omni-Embed-Nemotron-3B, and Embed2Evolve, including failures
Relative reduction between equal-task per-rollout means.
Reference API cost
All 20 trials per task for Direct and Embed2Evolve, including failures
Relative reduction between equal-task per-rollout means at fixed input/output rates.
Appendix
Table 30: Embed2Evolve test-metric accounting. All five tasks receive equal weight. Paired comparisons and reference API costs use Direct and Embed2Evolve.
Task
Paired n
Policy decisions
Execution time (s)
Control ticks
Tick change (%)
Poke Cube
7
34.86 → 22.71
419.30 → 382.16
133.0 → 181.4
+36.4
Stack Cube
2
45.00 → 10.00
499.03 → 186.35
90.0 → 65.0
−27.8
Lift Peg Upright
5
57.20 → 12.40
556.06 → 234.46
152.0 → 149.6
−1.6
Pull Cube with Tool
1
80.00 → 26.00
962.27 → 489.55
347.0 → 337.0
−2.9
Two-Robot Stack Cube
4
87.25 → 33.00
1054.67 → 751.59
206.0 → 312.8
+51.8
Appendix
Table 31: Efficiency and physical control on shared successes. Direct → Embed2Evolve means over the same n matched successful initializations for all three metrics. Execution time includes API waiting. Tick change is 100(Evolve/Direct−1) ; positive values indicate more physical control ticks despite fewer GPT decisions.
Method
Input tokens
Output tokens
Total tokens
Ref. cost (USD)
Poke Cube
Direct
693,733.05
16,052.50
709,785.55
7.7400
Embed2Evolve
375,759.60
12,903.40
388,663.00
4.4028
Stack Cube
Direct
935,086.30
18,401.05
953,487.35
10.2709
Embed2Evolve
656,130.60
19,000.10
675,130.70
7.5113
Appendix
Table 32: Policy token usage and reference cost per rollout. Means include all 20 test rollouts per task and condition, regardless of outcome. Direct denotes GPT-6 Astra Direct; the final group averages task means equally. USD estimates use the fixed input/output rates above.
Method
Modalities
Query → candidate
Action role
Cross-source
Policy coupling
Lifetime
Deep Multimodal
3D/L/A
3D+L → trajectory
Candidate key
No
Independent
Offline corpus
CLASP
L/S/A
L → state+action
Candidate key
No
Independent
Offline corpus
VLA-R
V/L/A
scene V/L → action token
Candidate key
No
Direct control
Action vocab.
ROSER
P
prototype → window
None
No
Independent
Offline corpus
RAEA
V/L
V/L → V/L record
Returned only
Yes
Coupled
Persistent bank
REGENT
S
state → demo state
Returned+blend
No
Coupled
Task buffer
Appendix
Table 33: Representative physical-experience retrieval interfaces. V/L/A/S/P denote vision, language, native action, state, and proprioception; 3D denotes point clouds. Training-only action use is listed under action role; cross-source galleries span datasets or interfaces.
Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi-layer key-value task memory. During execution, a lightweight online actor fuses learned time- and arm-indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re-running the foundation model. Fast-VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO-Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9-17.1x and requires 1.22 GiB of post-prefill online resident memory. Task-memory swaps further show that the actor relies on episode-matched task memory. These results show that task context can be established once while closed-loop control remains responsive throughout execution. Project page: https://garfield-kh.github.io/FastVLA
Kehong Gong, Guanli Hou, Xiaoxu Zheng +6
Huawei Technologies Co., Ltd · Huawei Central Research Institute
Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations can cover every task and situation a robot will encounter, making the ability to learn from context at deployment essential for generalization. Such in-context learning (ICL), however, remains largely beyond the reach of existing robotic policies. The broad agentic capabilities of commercial vision-language models (VLMs), such as GPT-6 Astra, raise a compelling question: can these models learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable robot behavior from a new initial state without gradient updates or persistent changes to task-specific parameters? We introduce GPT-Policy, a general-agent framework for in-context robot learning. GPT-Policy integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome. We evaluate its reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-robot trials, human video demonstrations improve task completion even without robot action labels, while aligned action references yield further gains on contact-sensitive tasks. These findings position GPT-Policy as a step toward robot adaptation through in-context learning, providing an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.
Dongzhou Cheng, Taoran Yi, Ye Fang +12
Morphi Robot · Shanghai Innovation Institute · Huazhong University of Science and Technology +7
General-purpose vision-language models offer a promising way to zero-shot robot control: \gptastra{} excels at open-ended and language- or image-conditioned manipulation but remains substantially weaker on high-precision and long-horizon tasks. We introduce RoboICL, an in-context robot-control framework that narrows these gaps without robot-specific parameter updates or a learned VLA. RoboICL separates demonstration context, which provides recorded examples when available, from interaction memory, which accumulates the model's own actions and observed outcomes. Both use a shared observation--action--receipt--observation grammar. To preserve experience across task stages, RoboICL combines sampled demonstration blocks with bounded anchored memory. Fixed anchors keep earlier rollout interactions available for in-context learning, while the latest interaction supports immediate error correction. Across 30 RoboDojo tasks, using zero shot for Open and one demonstration elsewhere, RoboICL improves on official zero-shot \gptastra{} by 20--27 progress-score points in every category. It leads the leaderboard baselines on Memory and Open, achieves comparable performance to the strongest Precision baseline, and remains competitive on Long-Horizon. Its 30-task Overall score is 50.64, versus 33.68 for the strongest baseline. On a separate ten-task subset, RoboICL scores 60.60, within 2.00 points of the π0.5 + \gptastra{} hybrid approach. On three real-robot tasks, mean progress rises from 14.45 at zero shot to 63.33 at one shot and 78.89 at three shots. On two development tasks, optional Jev-gated action reuse reduces \gptastra{} calls by 33--48%. Code is available at https://github.com/Mosi-AI/RoboICL.
Fangcheng Liu, Yeqing Shen, Anda Cheng +7
Samsung Research, Beijing, China · Samsung Robotics eXperience · Shanghai Jiao Tong University