Bridging Frontier Reasoning and Robot Execution: From Autonomous Demonstration Generation to Dense Language Supervision

Bosung Kim; Alexander Trevithick; Ruiyi Wang; Prithviraj Ammanabrolu

cs.ROOct 2, 2026

Bridging Frontier Reasoning and Robot Execution: From Autonomous Demonstration Generation to Dense Language Supervision

Authors: Bosung Kim, Alexander Trevithick, Ruiyi Wang, Prithviraj Ammanabrolu

Organizations: University of California, San Diego · NVIDIA

Abstract

Recent advances in frontier models enable robot manipulation from only a few demonstrations, but high inference latency limits their use for real-time robot control. To bridge this gap, we study two complementary approaches that connect frontier reasoning with low-latency local execution. First, we use a frontier model to autonomously generate demonstrations that supplement human demonstrations for training a fast local policy. We augment its in-context examples with corrective demonstration segments that show how to recover from physical errors, improving generation reliability. Generation time and cost decrease as successful examples accumulate in context, suggesting a path toward more efficient data collection. At deployment, a harness combines frontier-generated instructions with a fast local policy, enabling efficient execution while preserving the frontier model's ability to guide and correct actions. With low-latency execution delegated to the local policy, the bottleneck shifts to its capacity to reliably follow the frontier model's diverse instructions. Our second bridge introduces dense language supervision across three nested granularities---primitive, atomic, and composite---with multi-aspect descriptions at each level. Across long-horizon tasks in RoboCasa 365 and BEHAVIOR-1K, where instructions change as execution progresses, the combined supervision achieves the highest performance under both oracle and frontier-model instructors, demonstrating more reliable instruction following through the policy's language interface. Finally, we evaluate both bridges together on a crossword task that combines semantic planning and manipulation within a fixed time budget. These results support autonomous demonstration generation and dense language supervision as complementary components for connecting frontier reasoning to low-latency local execution.

Figures & tables

Figure 1: Autonomous manipulation from five successful in-context demonstrations. Task views and success counts are shown for Fable 5 and GPT-6 Astra across four tasks, with ten trials per model and task. Each trial starts from a reset environment and has a one-hour timeout, with no subsequent human intervention.

Figure 2: Corrective segments for pickup. Arrows illustrate adjustment directions; they are not recorded actions or model inputs.

Figure 3: Effect of adding corrective demonstration segments to the in-context examples. Success counts compare the baseline and augmented conditions for Fable 5 and GPT-6 Astra across four manipulation tasks, with ten trials per condition.

Figure 4: Generation efficiency as successful demonstrations accumulate in context. Bars show time in minutes (left axis), and lines show API cost in USD (right axis) for GPT-6 Astra over ten successful episodes per task. Episode indices count successful rollouts; time and API cost from failed attempts are excluded.

Figure 5: Average elapsed task time and task success across four control configurations: GPT-6 Astra alone, Astra with corrective demonstration segments (CS), the local policy alone, and the harness with Astra. Each panel reports one manipulation task, with $n=20$ trials per configuration and task. Time is averaged over successful trials only and includes reasoning, planning, tool calls, and API latency, excluding hardware-fix intervals. Success rate uses all 20 trials.

(a) Temporal granularity
Composite (task-level)	Open the cabinet. Place the apple and banana from the cabinet into the bowl. Then close the cabinet.
Atomic (subtask-level)	Open the cabinet $\rightarrow$ transfer the apple into the bowl $\rightarrow$ transfer the banana into the bowl $\rightarrow$ close the cabinet.
Primitive (apple transfer only)	Reach for the apple $\rightarrow$ grasp the apple $\rightarrow$ lift the apple $\rightarrow$ carry it over the bowl $\rightarrow$ lower the apple $\rightarrow$ release it into the bowl $\rightarrow$ retract the arm.
(b) Instruction formats for apple transfer
Arm/gripper kinematics	Grasp the apple, lift it out of the cabinet, carry it over the bowl, and release it into the bowl.
Spatial destinations	Move the apple from inside the cabinet into the bowl on the counter.

Table 1: Illustrative language supervision for MakeFruitBowl , an example task from RoboCasa 365, with an apple and a banana in the cabinet and a bowl on the counter. (a) The complete task and subtask sequence, with primitives shown only for apple transfer. (b) Four descriptions of that same apple-transfer subtask. Object names and kinematic descriptions must match the demonstrated episode.

	RoboCasa 365		BEHAVIOR-1K
Language Supervision	Oracle	Fable 5	Oracle	Fable 5
Task-level (fixed)	0.169		0.076
Subtask-level	0.406	0.381	0.143	0.084
Task + Subtask	0.417	0.402	0.154	0.091
Hierarchical only	0.456	0.440	0.143	0.084
Multi-aspect only	0.425	0.406	0.160	0.103

Table 2: Effect of language supervision on long-horizon task performance under oracle and frontier-model instructors. Values average seeds 42 and 142: success rate on RoboCasa 365 and mean subgoal progress on BEHAVIOR-1K. Table 10 gives both seed results.

Language Supervision	Opus 5		Gemini-3.1-Pro		Fable 5
	Original	Rephrased	Original	Rephrased	Original	Rephrased
Subtask-level	.346	.333 (-.013)	.325	.329 (+.004)	.392	.350 (-.042)
Hierarchical only	.413	.408 (-.005)	.400	.388 (-.012)	.446	.400 (-.046)
Multi-aspect only	.371	.363 (-.008)	.350	.350 (.000)	.417	.408 (-.009)
Hierarchical + Multi-aspect	.488	.471 (-.017)	.467	.454 (-.013)	.492	.483 (-.009)

Table 3: Task success rates on RoboCasa 365 by source planner (seed 42). Rephrased instructions are generated by Fable 5, preserving the task goal while changing the wording. Parentheses report changes between the displayed original and rephrased rates.

Figure 6: The crossword task. (a) Example board states and the robot workspace. (b) Mean total score after one hour across five games for four control configurations. Per-game scores are reported in Table 13 .

Figure 7: First 20 minutes of crossword execution under direct control and the frontier harness, both using GPT-6 Astra. Colored intervals show the frontier model’s initial planning and subsequent observation and reasoning, as well as robot action execution. Ticks mark model-requested camera observations. Each timeline represents one run per configuration. Full 60-minute timelines appear in Figure 11 .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Five interfaces for robot control, distinguished by the frontier model’s output and the components that translate it into physical actions.

Task	Corrective segments
Operate a Claw Machine	8
Stack Cups	8
Fold a Cloth	16
Close a Lid	16
Physical Crossword Puzzle	32

Appendix

Table 4: Number of corrective demonstration segments used per task.

Figure 9: Illustrative corrective demonstrations for crossword cube pickup. Each adjacent pair shows the initial offset (left) and subsequent approach (right) in a recorded wrist-camera sequence. The top row shows B and L cube pickups; the bottom row shows a second B cube pickup and a W cube pickup. Cyan arrows indicate illustrative alignment directions toward the target, manually added for visualization; they are not recorded action vectors or overlays supplied to the model. Timestamps indicate elapsed time within each episode.

Figure 10: Policy success rates with mixed and human-only training corpora. “Human demo + Synthetic” uses 20 human and 20 autonomously generated demonstrations per task; “Human demo only” uses 40 human demonstrations per task. Each bar reports success over 20 evaluation trials. Both conditions achieve 41/80 successes across the four tasks.

Primitive	Segments	Physical interpretation
Transport	6,256	Held object translates approximately horizontally.
Grasp	4,086	Gripper closes on an object.
Reach	4,066	Empty arm approaches an object with the gripper open.
Release	3,408	Gripper opens to release an object.
Lower	3,356	Held object descends.
Retract	3,057	Empty arm withdraws after release.

Appendix

Table 5: Composition of the RoboCasa primitive annotation corpus.

Source	Segments	Share
Object motion within the segment	14,935	55.1%
Inheritance within the grasp-to-release span	10,236	37.8%
Expanded temporal window	1,147	4.2%
Unresolved; decomposition label retained	795	2.9%
Simulator-derived total	26,318	97.1%

Appendix

Table 6: Sources of primitive-segment object labels. Simulator-derived coverage includes inherited identities and is not an annotation-accuracy estimate.

Check	Result
Transport with more than 6 mm of gripper-finger motion	0.0%; median 0.7 mm
Grasp containing gripper closing	96.9%
Release containing gripper opening	93.3%
Retract starting with gripper at least 80% open	74.4%; median openness 0.87
Retract overlapping release	0 / 3,057

Appendix

Table 7: Physical consistency checks for primitive annotations. These checks measure agreement with motion signals, not downstream task success.

Task	Subgoals
BreadAndCheese	2
ThawInSink	2
StartElectricKettle	2
ClearCuttingBoard	3
GatherCuttingTools	3
RinseBowls	3

Appendix

Table 8: RoboCasa 365 evaluation tasks by execution-plan depth. Subgoals count distinct instruction phases, not evaluation predicates.

Condition	Annotation levels in prompt	Training format
Task-level	Episode-level goal only	{goal}
Subtask-level	Current subgoal	{subgoal}
Task + Subtask	Episode-level goal + current subgoal (no primitive supervision)	{goal} . {subgoal}
Hierarchical only	Episode-level goal + corresponding hierarchy instruction (e.g., primitive: “lift the arm”)	{goal}. Current step: {hierarchical instruction}
Multi-aspect only	Current subgoal	{sampled subgoal}
Hierarchical + Multi-aspect	Episode-level goal + corresponding hierarchy instruction	{goal}. Current step: {sampled hierarchical instruction}

Appendix

Table 9: Prompt construction for the six language supervision conditions. Both hierarchical conditions pair the episode-level goal with the instruction corresponding to the sampled segment’s hierarchy level, using “Current step:” as a fixed phrase.

	RoboCasa 365		BEHAVIOR-1K
Language Supervision	Oracle	Fable 5	Oracle	Fable 5
Task-level (fixed)	0.169 (0.175 / 0.163)		0.076 (0.050 / 0.102)
Subtask-level	0.406 (0.408 / 0.404)	0.381 (0.392 / 0.371)	0.143 (0.130 / 0.156)	0.084 (0.070 / 0.098)
Task + Subtask	0.417 (0.425 / 0.408)	0.402 (0.408 / 0.396)	0.154 (0.140 / 0.168)	0.091 (0.080 / 0.102)
Hierarchical only	0.456 (0.463 / 0.450)	0.440 (0.446 / 0.433)	0.143 (0.130 / 0.156)	0.084 (0.070 / 0.098)
Multi-aspect only	0.425 (0.433 / 0.417)	0.406 (0.417 / 0.396)	0.160 (0.150 / 0.170)	0.103 (0.090 / 0.116)

Appendix

Table 10: Language supervision results: mean, followed by (seed 42 / seed 142). Metrics are success rate for RoboCasa 365 and mean subgoal progress for BEHAVIOR-1K. The task-level condition uses a fixed task instruction and is shown once per benchmark.

Metric	Opus 5	Gemini- 3.1-Pro	Fable 5
Step-count match	29/30	25/30	27/30
Mean semantic similarity	.886	.855	.895

Appendix

Table 11: Plan quality against oracle references: step-count match and mean semantic similarity from direct 0–1 judgments of individual steps by Fable 5.

Primitive Action	Subtask-level	Hierarchical only	threshold
Object up ("lift the object up")	0.250	0.833	16 cm
Object down ("lower the object down")	0.139	0.750	16 cm
Arm up ("move the arm up")	0.111	0.750	34 cm
Arm down ("move the arm down")	0.111	0.778	34 cm
Arm forward ("move the arm forward")	0.056	0.556	24 cm
Arm back ("move the arm back")	0.028	0.444	24 cm

Appendix

Table 12: Single-seed policy compliance rates for primitive kinematic instructions.

Cell IDs prohibited in instructions
Configuration	Game 1	Game 2	Game 3	Game 4	Game 5	Mean score
Fable 5 (direct control)	6 / 0	4 / 0	4 / 0	6 / 0	4 / 0	4.8
GPT-6 Astra (direct control)	8 / 0	4 / 0	10 / 1	10 / 1	8 / 0	8.0
GPT-6 Astra harness	14 / 2	16 / 2	14 / 1	14 / 1	14 / 1	14.4
GPT-6 Astra harness + dense supervision	16 / 2	16 / 2	14 / 1	16 / 1	16 / 2	15.6
GPT-6 Astra harness	4 / 0	4 / 0	2 / 0	2 / 0	2 / 0	2.8

Appendix

Table 13: Crossword results after one hour per game. Each game cell reports total score / completed words. Scores award

+2

per correctly placed cube and

+2

per completed valid word; the final column reports mean total score. The final two rows prohibit cell IDs in frontier-generated instructions.

Configuration	Successful trials	Accuracy
Fable 5 (direct control)	5/20	25%
GPT-6 Astra (direct control)	13/20	65%
Local policy only	10/20	50%
GPT-6 Astra harness	10/20	50%

Appendix

Table 14: Crossword pick-and-place accuracy over 20 trials per configuration, excluding retries.

Figure 11: Full 60-minute crossword activity timelines for GPT-6 Astra, one run per configuration. Ticks mark camera observations requested by the model. Cumulative action-execution time is 2 min 40 s under direct control and 9 min 8.55 s with the harness; these are individual runs, not averages across games.

Explore similar work

RDGen: Demonstration Generation for High-Quality Robot Learning via Reinforcement Learning
May 29, 2026Zijian Zhu, Menglin Zou, Zhuang Li +2RL for Robotics Robot Skill Learning
Demonstration-Free Robotic Control via LLM Agents
Jan 28, 2026Brian Y. Tsui, Alan Y. Fang, Tiffany J. HwuRobotic Control Language Model-Based Control
All authors are independent researchers.
How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
May 16, 2026Bosung Kim, Ruiyi Wang, David Acuna +5Robot Policy Learning Robot Skill Learning
University of California, San Diego · 2NVIDIA

May 29, 2026J/K move · Enter open · S save

RDGen: Demonstration Generation for High-Quality Robot Learning via Reinforcement Learning

Zijian Zhu, Menglin Zou, Zhuang Li, Yaojie Tu+1

RL for Robotics Robot Skill Learning Synthetic Data Generation for Robotics

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robot control. However, their performance remains fundamentally constrained by the availability of high-quality robot trajectory data. In current robot learning practice, such data are primarily collected through human teleoperation, which is labor-intensive, costly, and difficult to scale. In this paper, we propose RDGen, a sim-to-real reinforcement learning framework for generating high-quality robot demonstrations. Rather than employing reinforcement learning solely as the final control policy, RDGen leverages trained RL policies as a structured trajectory generator. The system consists of a VLM-based task parser that identifies task-relevant objects, a Grounding DINO-based object localizer, and an RL policy transferred from simulation to the real robot. Successful rollouts are then harvested as clean, high-quality demonstrations for downstream VLA training, while the simulation stage further provides a scalable source of additional trajectories at little marginal cost. Experiments on a pick-and-place task demonstrate that the transferred RL policy achieves a high task success rate. Compared with human teleoperation, RDGen produces significantly smoother trajectories and yields superior downstream VLA performance. These results indicate that RL-generated demonstrations can serve as more reliable and consistent supervisory signals for robot policy learning.

Abstract

Explore similar work

RDGen: Demonstration Generation for High-Quality Robot Learning via Reinforcement Learning

Demonstration-Free Robotic Control via LLM Agents

How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning