RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

Bohan Zhou; Xingbei Chen; Emily Huang; Weilin Ruan; Haojian Huang; Yehang Zhang; Zexi Li; Wenqian Li

cs.ROOct 8, 2026

RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

Authors: Bohan Zhou, Xingbei Chen, Emily Huang, Weilin Ruan, Haojian Huang, Yehang Zhang, Zexi Li, Wenqian Li, +12 more

Organizations: The Chinese University of Hong Kong · The Hong Kong University of Science and Technology · Knowin AI · The University of Hong Kong · Peking University · University of the Chinese Academy of Sciences

Abstract

Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present RoboAware, which builds on coding agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the $P^5$ schema and formulate a hierarchical MDP based on it. $P^5$ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce State-Locked Counterfactual Branching (SCB), which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose Execution-Aware Learning (EAL), which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a 77.0% overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.

Figures & tables

Figure 1 : Overview of RoboAware. (a) The $\mathrm{P}^{5}$ skill schema decomposes each skill into five stages—perceive, propose, pre-manipulate, perform, and post-manipulate—and unifies primitive skills ( e.g. , pick, place, push, pull) and compositional skills ( e.g. , handover) under one representation. (b) A hierarchical MDP selects a policy family $c_{i}\in\{\mathrm{Mod},\mathrm{E2E}\}$ at each $\mathrm{P}^{5}$ stage decision boundary; a coding agent then generates each corresponding code block $a_{i}$ . (c) EAL learns family-conditioned coordination: SCB evaluates modular and end-to-end branches from the same restored state into $\mathcal{D}_{\mathrm{SCB}}$ , Q-learning yields $Q_{\theta}(x,c)$ , and at test time a frozen coordinator selects best $c_{i}$ to condition code generation and execution.

Figure 2 : Method overview of EAL in RoboAware. (1) State-locked counterfactual search repeats Selection (UCB), Expansion ( SCB paired $\mathrm{Mod}$ / $\mathrm{E2E}$ branches from a restored $\xi_{i}$ ), Rollout (suffix return $G_{i}$ ), and Backup within budget $K$ , retaining successful, failed, and nonpreferred transitions in $\mathcal{D}_{\mathrm{SCB}}$ . (2) A VLM backbone with an MLP head maps context–family tokens ending in <VALUE> to $Q_{\theta}(x_{i},c_{i})$ , trained by Bellman updates on skill-completion and terminal-task feedback. (3) At deployment, frozen $Q_{\theta}$ induces $\rho_{\theta}$ and selects $c_{i}^{*}=\arg\max_{c}\rho_{\theta}(c\mid x_{i})$ ; a frozen coding agent generates a local code block against the fixed API library and executes it, looping on observable feedback at each $\mathrm{P}^{5}$ boundary.

Table 3

Method	LIBERO-Pro								RoboSuite	RoboTwin	Overall
Method	Spat-T	Spat-S	Obj-T	Obj-S	Goal-T	Goal-S	L10-T	L10-S	RoboSuite	RoboTwin	Overall
End-to-end policies
$\pi_{0}$ [ 7 ]	0.0	0.0	0.0	2.0	0.0	0.0	0.0	0.0	0.0	/	0.2
$\pi_{0.5}$ [ 20 ]	1.0	20.0	1.0	17.0	2.0	38.0	1.0	8.0	0.0	/	9.8
FastWAM [ 21 ]	30.0	0.0	10.0	0.0	10.0	0.0	10.0	0.0	0.0	81.9	14.2
FasterWAM [ 25 ]	50.0	0.0	10.0	10.0	10.0	10.0	10.0	0.0	0.0	85.6	18.6

Table 3 : Aggregate success rates (%) on LIBERO-Pro (instruction-redirection T / position-swap S ), RoboSuite, and RoboTwin. RoboSuite and RoboTwin use their full task pools under seeds

1

–

10

. Overall is the mean over available columns. On RoboTwin,

\pi_{0}

and

\pi_{0.5}

are incompatible with RoboTwin’s dual-arm setup, so results involving these models are marked “/” because of the embodiment mismatch. For fair comparison, global cross-episode memory is disabled for baselines that rely on memory. All seeds are tested in one trial .

Table 6

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

$z_{i}$	$\phi^{\mathrm{in}}$	$\mathcal{C}_{i}$	$\phi^{\mathrm{out}}$	Fail
perceive	target ID/pose $\notin h_{i}$	$\{\mathrm{Mod},\mathrm{E2E}\}$	typed target info	retry / $\bot$
propose	pose/mask $\in h_{i}$ $\land$ eef target $\notin h_{i}$	$\{\mathrm{Mod}\}$	valid eef pose	retry / $\uparrow$ perceive / $\bot$
pre-manipulate	approach target available	$\{\mathrm{Mod},\mathrm{E2E}\}$	pre-contact reached	replan / $\uparrow$ / $\bot$
perform	pre-manip. $\phi^{\mathrm{out}}$	$\{\mathrm{Mod},\mathrm{E2E}\}$	contact/gripper postcond.	retry / $\rightarrow$ post-manip. / $\bot$
post-manipulate	perform $\phi^{\mathrm{out}}$	$\{\mathrm{Mod},\mathrm{E2E}\}$	retreat/release/stabilize	retry / $\uparrow$ / $\bot$

Appendix

Table 1 :

\mathrm{P}^{5}

stage contracts and admissible families. At stage

z_{i}

of skill

g_{i}

, the coordinator selects

c_{i}\in\mathcal{C}_{i}(z_{i},g_{i})

before code generation; APIs must match

(z_{i},c_{i})

( Table 2 in the main text). Columns:

\phi^{\mathrm{in}}

—entry predicate;

\mathcal{C}_{i}

—admissible families;

\phi^{\mathrm{out}}

—registered success postcondition; Fail—successor after failure/timeout (retry

z_{i}

, return upstream

\uparrow

, or terminate

\bot

). Under

\mathrm{Mod}

\phi^{\mathrm{out}}

is the typed modular return (e.g. pose at perceive , eef pose at propose ); under

\mathrm{E2E}

, it is covered by the atomic call postcondition. propose is Mod-only; an E2E call may complete the skill without visiting it.

Setting	Value
Fine-tuning method	full-FT (backbone + MLP value head)
Thinking	Disabled
Optimizer	AdamW
Minibatch size	$16$
Learning rate $\eta$ / schedule	$10^{-5}$ , cosine, $5\%$ warmup
Gradient updates $M$	$10{,}000$

Appendix

Table 2 : Coordinator Q-fitting settings. Symbols match Algorithm 1 .

Setting	Value
Backbone	Qwen3.5-0.8B
Image viewpoint	agent-view RGB + depth visualization
Image resolution $R{\times}R$	$224{\times}224$
Proprioception encoding	structured text
Depth encoding	colormap image
History window $H$ (full transitions)	$4$

Appendix

Table 3 : Q-value model settings.

API	Stage	Input	Output
Perceive_object_pose	perceive	task instruction + agent-view RGB-D + calibration	target pose + mask/geometry provenance
Propose_eef_pose	propose	RGB-D geometry + target mask + robot state + task context	ranked end-effector target + screening provenance
Plan_to_eef_pose	pre / post-manipulate	scene observation, robot state, target pose, optional attachment	continuous, smooth, collision-aware qpos sequence
Perform_gripper	perform	gripper open/close command	updated gripper state
Perform_E2E	perceive , pre-manipulate , perform , post-manipulate	route ID + task instruction + RGB-D + proprioception	closed-loop control until postcondition, failure, or budget

Appendix

Table 4 : Typed input/output contracts for the high-level embodied APIs in Table 2 in the main text. The E2E route uses route_id to select one frozen backend.

Policy family	Code-block space	Permitted API pattern	Backend choice
$\mathrm{Mod}$	$\mathcal{A}_{\mathrm{Mod}}$	stage-admissible modular API composition	fixed selected modular backends
$\mathrm{E2E}$	$\mathcal{A}_{\mathrm{E2E}}$	Perform_E2E $(\texttt{route\_id},\ldots)$	retained frozen backend selected inside $a_{i}$

Appendix

Table 5 : Policy-family bindings for generated code blocks. Mod composes stage-admissible APIs; E2E invokes one frozen backend through route_id .

Method	Success@1 $\uparrow$	Success@3 $\uparrow$	Success@5 $\uparrow$	Proposal time (s) $\downarrow$	Screening time (s) $\downarrow$	Time@1 (s) $\downarrow$
AnyGrasp [ 61 ]	13/18	16/18	17/18	7.98	3.46	36.39
GraspGen [ 62 ]	11/18	12/18	12/18	4.54	3.16	24.84
NeuGraspNet [ 18 ]	12/18	13/18	13/18	12.90	1.56	39.05
EconomicGrasp [ 63 ]	10/18	12/18	12/18	5.03	1.11	20.68
RNGNet [ 64 ]	7/18	8/18	8/18	18.89	1.34	46.33
NeuGraspNet+ (ours)	18/18	18/18	18/18	14.36	2.23	49.04

Appendix

Table 6 : Grasp proposal results on the 18-task Panda benchmark. All baseline methods use their native proposal rankings; NeuGraspNet uses our final selection pipeline. All methods share the same downstream cuRoboV2 motion planner. Success@

k

indicates whether a successful grasp is found within the first

k

evaluated proposals. Proposal and screening times are averaged over all 18 tasks; Time@1 is averaged over successful Top-1 trials.

	Planner-native fresh (main)			Fixed grasp		Conditional diagnostic
Planner	Plannable $\uparrow$	Task success $\uparrow$	Pipeline / Plan (s) $\downarrow$	Task success $\uparrow$	Plan (s) $\downarrow$	Grasp success $\uparrow$	Plan (s) $\downarrow$
cuRoboV2 [ 19 ]	18/18	18/18	3.82 / 0.254	15/18	0.192	18/18	0.935
DRP/IMPACT [ 65 ]	16/18	14/18	6.23 / 0.418	11/18	0.576	16/18	1.526
VAMP [ 66 ]	12/18	11/18	73.30 / 9.320	6/18	0.931	14/18	1.598
Avoid Everything [ 67 ]	–	–	–	–	–	2/18	3.677

Appendix

Table 7 : Motion-planning results on the 18-task Panda benchmark. The planner-native fresh protocol is the main end-to-end comparison: each planner independently resolves endpoints, screens proposals, plans, and executes. Pipeline time includes candidate screening, endpoint resolution, and planning; Plan time measures the final selected trajectory only. The fixed-grasp protocol uses a shared grasp target. The conditional diagnostic uses cuRobo-success anchors and should not be ranked directly against the planner-native results. Avoid Everything was evaluated only in the conditional pose-goal diagnostic.

	LIBERO-Pro		RoboSuite		RoboTwin 2.0
Method	Full	G–L	Full	G–L	Full	G–L
Vision-Language-Action Policies
OpenVLA-OFT [ 68 ]	4/14	5/14	0/4	0/4	0/10	1/10
PRTS [ 69 ]	6/14	7/14	1/4	1/4	0/10	4/10
RDT2 [ 70 ]	0/14	0/14	0/4	0/4	0/10	0/10
SpatialVLA [ 71 ]	0/14	0/14	0/4	0/4	0/10	1/10

Appendix

Table 8 : Success counts across three evaluation suites. Full measures successful completion of the entire task, whereas G–L measures whether the policy successfully grasps and lifts the target object, providing a partial-task measure of manipulation capability. Each entry reports the number of successful evaluations over the total number evaluated; the best result in each column is shown in bold. ⋆ The

\pi

0.5-RLinf-SFT policy uses the same LIBERO-trained checkpoint across all three suites, without suite-specific adaptation. ‡ For RynnVLA-002, the current public loader reinitializes part of the VQ encoder when loading the released checkpoint.

	Full					G–L
	All trials			Successful only		All trials			Successful only
Method	Steps	Time (s)	Hz	Mean (s)	Median (s)	Steps	Time (s)	Hz	Mean (s)	Median (s)
Vision–Language–Action policies
OpenVLA-OFT	$547.4$	$34.2$	$6.171$	$7.8$	$7.5$	$503.1$	$117.0$	$6.368$	$41.0$	$35.6$
PRTS	$459.8$	$39.5$	$3.017$	$11.7$	$11.5$	$429.2$	$105.6$	$3.506$	$41.5$	$30.9$
RDT2	$720.0$	$126.6$	$0.885$	—	—	$720.0$	$287.1$	$0.771$	—	—

Appendix

Table 9 : Execution efficiency on LIBERO-Pro under the Full and G–L criteria. Full denotes end-to-end task success, while G–L denotes successful grasp and lift of the target object. For each criterion, All trials aggregates both successful and failed episodes and reports the average number of executed steps, average execution time, and effective control frequency. Successful only reports the mean and median completion time over successful episodes only. A dash indicates that the method achieves no successful trial under that criterion and therefore has no successful-only timing statistic. Lower all-trial steps or execution time do not necessarily indicate greater efficiency, since failed episodes may terminate before completing the full rollout horizon; these quantities should therefore be interpreted jointly with the corresponding success counts in Table 8 .

	Full					G–L
	All trials			Successful only		All trials			Successful only
Method	Steps	Time (s)	Hz	Mean (s)	Median (s)	Steps	Time (s)	Hz	Mean (s)	Median (s)
Vision–Language–Action policies
OpenVLA-OFT	$720.0$	$179.2$	$2.337$	—	—	$720.0$	$111.2$	$7.042$	—	—
PRTS	$575.5$	$160.0$	$1.779$	$37.6$	$37.6$	$561.5$	$94.5$	$3.092$	$18.2$	$18.2$
RDT2	$720.0$	$261.7$	$0.726$	—	—	$720.0$	$202.5$	$0.856$	—	—

Appendix

Table 10 : Execution efficiency on RoboSuite under the Full and G–L evaluation criteria. Full requires successful completion of the entire task, whereas G–L requires successful grasp and lift of the target object. For each criterion, All trials aggregates both successful and failed evaluations and reports the average number of executed steps, average execution time, and effective control frequency. Successful only reports the mean and median execution time computed only over evaluations satisfying the corresponding success criterion. A dash denotes that no evaluation is successful under that criterion, so successful-only timing statistics are unavailable; NR denotes a statistic that was not recorded. Lower all-trial step counts or execution times do not necessarily indicate higher execution efficiency, since failed evaluations may terminate early rather than complete the full rollout horizon. All-trial efficiency should therefore be interpreted jointly with the corresponding success counts reported in Table 8 , where method references are also provided.

	Full					G–L
	All trials			Successful trials		All trials			Successful trials
Method	Steps	Time (s)	Hz	Mean (s)	Median (s)	Steps	Time (s)	Hz	Mean (s)	Median (s)
Vision–Language–Action policies
OpenVLA-OFT	$792.0$	$1273.4$	$3.879$	—	—	$675.6$	$1292.2$	$3.434$	$51.3$	$51.3$
PRTS	$790.0$	$1620.3$	$1.370$	—	—	$464.5$	$654.0$	$2.139$	$78.9$	$53.0$
RDT2	$792.0$	$1372.8$	$0.347$	—	—	$790.0$	$1359.1$	$0.334$	—	—

Appendix

Table 11 : Execution efficiency on RoboTwin 2.0 under the Full and G–L evaluation criteria. Full requires successful completion of the entire task, whereas G–L requires successful grasp and lift of the target object. For each criterion, All trials reports execution statistics computed over the complete evaluation set, including both successful and failed trials: Steps , Time (s) , and Hz characterize rollout length, execution time, and effective control frequency, respectively. Successful trials reports the mean and median execution time over trials satisfying the corresponding success criterion. A dash denotes that no trial is successful under that criterion, so successful-trial timing statistics are unavailable. Lower all-trial step counts or execution times do not necessarily indicate greater execution efficiency, since failed trials may terminate before task completion and therefore incur shorter rollouts. These efficiency statistics should therefore be interpreted jointly with the corresponding success counts reported in Table 8 , where method references are also provided.

Parameter	Symbol	Setting
Discount factor	$\gamma$	$1$
Perception reward weight	$w_{\mathrm{p}}$	$0.2$
Manipulation reward weight	$w_{\mathrm{m}}$	$0.3$
Task-success reward weight	$w_{\mathrm{t}}$	$0.5$
Smooth- $L_{1}$ threshold	—	$1$
Neighboring-root draws	$S$	$4$

Appendix

Table 12 : Hyperparameters of RoboAware.

Method	Planner	Memory	Seed-0 reference	SAM3	Scripted hybrid	VLA
A: Full Harness VLA	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\pi_{0.5}$ SFT
B: w/o Memory	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\pi_{0.5}$ SFT
C: w/o Reference	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\pi_{0.5}$ SFT
D: w/o SAM3	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\pi_{0.5}$ SFT
E: w/o Memory & SAM3	$\checkmark$	$\times$	$\times$	$\times$	$\checkmark$	$\pi_{0.5}$ SFT
F: Bare SFT	$\times$	$\times$	$\times$	$\times$	$\times$	$\pi_{0.5}$ SFT

Appendix

Table 13 : Component configurations for the Harness VLA ablation study. All variants use the same

\pi_{0.5}

SFT backbone.

Method	Success / valid	Success rate (%)	Drop (pp)	95% CI (pp)	S $\rightarrow$ F	F $\rightarrow$ S
A: Full Harness VLA	15/24	62.5	–	–	–	–
B: w/o Memory	11/24	45.8	16.7	$[-8.3,\ 41.7]$	7	3
C: w/o Reference	12/24	50.0	12.5	$[-12.5,\ 37.5]$	6	3
D: w/o SAM3	14/24	58.3	4.2	$[-8.3,\ 20.8]$	2	1
E: w/o Memory & SAM3	9/24	37.5	25.0	$[4.2,\ 45.8]$	7	1
F: Bare SFT	10/24	41.7	20.8	$[0.0,\ 41.7]$	6	1

Appendix

Table 14 : Paired ablation screening on 24 tasks using seed 42. Drop is defined as the success rate of Full minus that of the corresponding variant, in percentage points (pp). Confidence intervals are 95% task-bootstrap intervals for the paired drop. S

\rightarrow

F counts tasks solved by Full but failed by the ablated variant; F

\rightarrow

S counts the reverse. All 24 episodes completed without errors in each variant.

Method	Success	Failure	Error	Valid / scheduled	Valid SR (%)	Coverage (%)	$n_{\mathrm{pair}}$	Paired $\Delta$ (pp)	95% CI (pp)
A: Full Harness VLA	64	49	127	113/240	56.6	47.1	–	–	–
B: w/o Memory	20	98	122	118/240	16.9	49.2	113	$\mathbf{-38.9}$	$[-50.0,\ -27.4]$
E: w/o Memory & SAM3	24	91	125	115/240	20.9	47.9	109	$\mathbf{-35.8}$	$[-48.2,\ -23.4]$

Appendix

Table 15 : Multi-seed confirmation on 80 tasks with seeds

\{41,42,43\}

, yielding 240 scheduled episodes per variant. Valid SR is the number of successful episodes divided by the number of successful and failed episodes, so execution errors are excluded. Coverage is the fraction of the 240 scheduled episodes that completed without errors. Paired differences compare each variant against Full on task–seed pairs that completed without errors in both conditions:

\Delta=\mathrm{SR}_{\mathrm{variant}}-\mathrm{SR}_{\mathrm{Full}}

on that jointly valid subset of size

n_{\mathrm{pair}}

, which need not equal the gap between the separately reported valid success rates.

Figure 1 : Additional LIBERO-Pro comparison on “Pick the akita black bowl between the plate and the ramekin and place it on the plate,” complementing Figure 4 in the main text. RoboAware-EO approaches with a misaligned gripper and fails to secure the bowl. RoboAware-MO completes a grasp but selects the wrong object (the cookie box). RoboAware grounds the spatially described bowl and finishes the pick-and-place, succeeding by switching policy families across $\mathrm{P}^{5}$ boundaries.

Figure 2 : Failure modes of the unsuccessful rollouts. Grasping instability is the largest identified failure mode (29.8%), followed by constrained contact or insertion failures (16.7%). Grounding, placement, and multi-object sequencing account for 5.3%, 6.1%, and 4.4%, respectively. Others (37.7%) denotes failed rollouts for which the saved evaluation records are insufficient to identify a specific cause (29.8% from LIBERO-Pro and 7.9% from RoboTwin).

Explore similar work

Playful Agentic Robot Learning
Jun 17, 2026Junyi Zhang, Jiaxin Ge, Hanjun Yoo +17Robot Policy Learning Robot Skill Learning
University of California, Berkeley · Impossible Research
Encore: Few-Shot Agentic Discovery of Manipulation Strategies
Sep 29, 2026Yifan Kang, Zihan Wang, Zhiwen Fan +1Robot Policy Learning Language-Conditioned Robot Manipulation
Massachusetts Institute of Technology · AI Open Source Foundation · Texas A&M University +1
RHO: Your Coding Agent is Secretly a Roboticist
Jun 15, 2026Karim Elmaaroufi, Justin Svegliato, Sarunas Kalade +3Robot Policy Learning AI Coding Agents
University of California, Berkeley · Embodied Science · AMD

Jun 17, 2026, cs.ROJ/K move · Enter open · S save

Playful Agentic Robot Learning

Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu+16

University of California, Berkeley · Impossible Research

Robot Policy Learning Robot Skill Learning Agent Skill Learning Robot Learning

Open paper AskPDF arXiv

Current agentic robot systems can write executable Code-as-Policy programs, observe feedback, and revise behavior across multiple attempts, but they remain largely task-driven: reusable skills are acquired only after explicit instructions. We study Playful Agentic Robot Learning, where an embodied coding agent uses self-directed play as a continual skill-learning stage before downstream tasks arrive. We introduce RATs, Robotics Agent Teams designed for play-time skill acquisition. During play, RATs proposes novel yet learnable exploratory tasks, plans and executes robot-code policies, verifies intermediate progress, diagnoses failures, retries with dense, step-level feedback, and distills successful executions into a persistent code skill library. At test time, the agent reuses relevant skills from this frozen library to help solve new tasks. Experiments in LIBERO-PRO and MolmoSpaces show that play-learned skills improve held-out downstream tasks over no-play and random-play baselines, with 20.6 and 17.0 percentage-point gains over CaP-Agent0 on LIBERO-PRO and MolmoSpaces, respectively. Moreover, the learned skills can be plugged into other inference-time Code-as-Policy agents by simply retrieving them into the context, improving RoboSuite and real-world transfer by 8.9 and 8.8 points, respectively, without finetuning the underlying model.

Abstract

Explore similar work

Playful Agentic Robot Learning

Encore: Few-Shot Agentic Discovery of Manipulation Strategies

RHO: Your Coding Agent is Secretly a Roboticist