Scaling Online Exploration for Robot Agents

Runhan Huang; Xiangcheng Zhang; Haoyu Chen; Yitong Li; Yen-Jen Wang; Bohan Lyu; Haotian Ye; Chuye Hong

Figure 2: System architecture and self-improving loop of RAX. The coding agent, powered by a frozen LLM, interacts with physical hardware across an isolation boundary using multimodal perception and control SDK interfaces. By reflecting on execution outcomes, the agent continually updates a persistent memory system containing working context, task notes, and a versioned repository of verified skills. During exploration, the agent compiles its perception and planning logic into standalone Python control programs that execute high-frequency loops directly on the robot. When encountering perception or planning failures due to environment variations, the program raises an exception ( raise Unsure() ), delegating control back to the agent to make adjustments and refine the codebase.

Figure 3: Agentic 3D perception and motion planning on a real robot. (a) The articulated air fryer model constructed programmatically by the agent from point-cloud measurements and fit to RGB-D observations, consisting of a fixed chassis and a drawer articulated via a prismatic joint along pull direction

d

relative to central axis

c

. (b) Action trajectories planned on the instantiated model with the drawer opened by

13.1\,\text{cm}

. End-effector poses are visualized as pyramids whose bases face the approach vector and whose thick edges indicate the finger closure axis. The underlying geometric estimator maintains high fidelity, yielding only a

6.5\,\text{mm}

displacement between the predicted basket rim and the post-opening physical measurement. Figure 27 illustrates the corresponding execution sequence on the physical robot.

Figure 4: Frontend–backend separation. The agent and its programs act only through the tool interface; the backend owns control, sensing, and safeguards on every platform.

Figure 5: A long-horizon task on the real RB-Y1. The agent composes six sub-tasks from skills learned in earlier sessions. It finds the cups from Gemini’s rim points and depth, measures the desk edge by picking points in the head image, and plans the base routes on a model of the scene.

Platform	Low-level control	Agent action space	Model-based helpers
LIBERO, LIBERO-Pro single-arm Franka Panda	EEF pose control (OSC)	EEF position and rotation deltas; gripper command	Pixel-to-3D projection from depth
RoboDojo dual-arm ARX X5	Joint position control (PD)	Joint-target chunks for both arms and grippers; bounded EEF steps	Forward kinematics, differential IK, pose algebra, collision-aware motion planning, robot-only preview
DexJoCo Franka Panda + Allegro hands	Arm: EEF pose control (OSC); hand: joint position control	Flange pose and sixteen hand joint targets per arm, single- or dual-arm	Pixel-to-3D projection from depth, robot-only preview
BEHAVIOR-1K Galaxea R1Pro	Base velocity control; trunk and arm joint position control	Base velocity, trunk and arm joint targets, grippers; closed-loop base moves from odometry	Closed-loop arm IK, arm and trunk motion planning, robot-only preview
HumanoidArena Unitree G1 + Dex3	Whole-body motion tracker	Whole-body reference frames: heading, 29 joint targets, hand open or close	Arm and waist motion planning to hand poses; kinematic model for forward kinematics and image projection
Franka FR3 real	Joint impedance control	Straight-line EEF moves, joint moves, force-controlled grasps	Inverse kinematics; path checks against workspace and table

Table 1: Low-level control on each platform. EEF: end effector; OSC: operational-space control; PD: proportional-derivative.

		Make toast	Hang mugs	Sweep blocks	Plug	Swap	Make	Pour by
Method	Budget	random	random	random	charger	blocks	kong	language
VPP2	–	38.0	13.5	1.3	16.7	26.0	78.7	0.0
Awomo-0.5	–	7.0	9.6	0.0	7.3	4.7	30.0	0.0
Simate-beta	–	7.7	11.4	0.0	29.3	38.7	76.0	0.0
Rex-M1 Preview	–	14.3	20.2	8.0	3.3	13.3	33.3	0.0
VPP2-Preview	–	13.7	14.5	2.7	6.0	22.7	34.7	0.0

Table 2: Scaling exploration on RoboDojo. Score (%) on unseen layouts; baselines from the top five policies of the RoboDojo-Sim leaderboard ( Chen et al., 2026a ) . Cells with more than one run report the mean. –: not run. † No program submitted. ‡ Agent + VLA; listed among agents and unranked on the leaderboard.

Method	books	organized	items	food	desk	the car	Avg.
	Boxing	Getting	Sorting	Clearing	Clean up	Loading
Robot Learning Collective	0.0	2.0	6.2	8.0	12.7	20.0	8.2
Comet	0.0	0.0	0.0	0.0	0.0	0.0	0.0
SimpleAI Robot	0.0	0.0	0.0	0.0	0.0	0.0	0.0
The North Star	0.0	0.0	0.0	0.0	0.0	20.0	3.3
Embodied Intelligence ‡	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Table 3: BEHAVIOR-1K. Q-score (%) on the ten public test instances; baselines from the five highest-ranked entries of the 2025 BEHAVIOR Challenge ( Li et al., 2023 ) . † Fully automated program. ‡ Privileged track.

Figure 6: The program RAX writes for boxing books on BEHAVIOR-1K ( Table 3 ). It detects the books, shelves and box in the head camera, tracks the robot’s motion by visual odometry, maps the scene, and plans paths with A*; the policy flow shows the camera view at each step.

			Human–object interaction			Human–scene interaction
Method	Demos	Budget	Football	DoubleDesk	P&PBox	OpenDoor	SitSofa	Boxing	VisNavi	Avg.
ACT	100	–	16.7	18.3	56.7	78.3	73.3	56.7	33.3	47.6
Diffusion Policy	100	–	45.0	36.7	75.0	85.0	78.3	76.7	23.3	60.0
Flow matching	100	–	13.3	38.3	73.3	70.0	15.0	70.0	38.3	45.5
$\pi_{0.5}$	100	–	10.0	43.3	71.7	66.7	73.3	70.0	23.3	51.2
WB-WAM	100	–	70.0	65.0	86.7	98.3	95.0	81.7	76.7	81.9

Table 4: HumanoidArena. Success rate (%) on the 60 official test episodes of each task, all methods with the SONIC tracker. Baselines from Wang et al. (2026b) and Qin et al. (2026) . Max: each task at its largest budget. ⋆ Three demonstrations without actions. –: not run.

		Single-arm						Bimanual
Method	Budget	Hammer	Mouse	Bucket	Tongs	Glasses	Water	iPad	Hanoi	Assembly	Microwave	Photo	Avg.
DP-T	–	81.3	62.0	83.3	22.7	53.3	84.0	8.0	24.7	4.7	73.3	56.7	50.4
DP-C	–	58.7	74.0	70.0	57.3	54.0	63.3	52.0	12.7	3.3	54.0	24.0	47.6
ACT	–	50.0	61.3	64.0	31.3	47.3	47.3	9.3	6.0	0.0	66.0	7.3	35.5
$\pi_{0.5}$	–	84.7	64.7	84.0	24.0	72.0	88.7	36.0	23.3	2.7	60.0	54.7	54.1
GR00T N1.5	–	67.3	85.3	72.0	12.7	27.3	72.7	12.7	0.7	0.7	50.7	40.7	40.2

Table 5: DexJoCo, rand-obj setting. Success rate (%) on the 150 official evaluation episodes of each task. Baselines from Wang et al. (2026a) , Park et al. (2026) (GR00T N1.6, Fast-WAM, PointWAM), Zhang et al. (2026) , and Li et al. (2026) . DP-T and DP-C: Diffusion Policy with transformer and CNN backbones. † One policy for all tasks. ‡ Mean over the ten tasks other than iPad. § With simulated tactile input. –: not reported.

		LIBERO					LIBERO-Pro, position perturbation
Method	Demos	Spatial	Object	Goal	Long	Avg.	Spatial	Object	Goal	Long	Avg.
OpenVLA	50	84.7	88.4	79.2	53.7	76.5	0.0	0.0	0.0	0.0	0.0
OpenVLA-OFT	50	97.6	98.4	97.9	94.5	97.1	–	–	–	–	–
$\pi_{0}$	50	96.8	98.8	95.8	85.2	94.2	0.0	0.0	0.0	0.0	0.0
$\pi_{0.5}$	50	98.8	98.2	98.0	92.4	96.9	20.0	17.0	38.0	8.0	20.8
X-VLA	50	98.2	98.6	97.8	97.6	98.1	0.0	2.0	1.0	0.0	0.8

Table 6: LIBERO and LIBERO-Pro. Success rate (%) on the 50 official initial states of each task and on the 50 position-perturbed states of LIBERO-Pro. Baselines from their papers, openpi (

\pi_{0}

,

\pi_{0.5}

), and the LIBERO-Pro leaderboard ( Zhou et al., 2025 ) ; PatchWAM reports only the mean, and VPP2 uses 10 initial states per task. † Developed on the perturbed setting under evaluation ( Section 12 ). ‡ Zero-shot with a skill library built on LIBERO-90; included in the average. –: not reported.

Agent	Agents	Envs	Budget	Time (h)	Success	Score
Codex	1	1	32	14.2	40.6	66.4
Codex	2	2	32	9.2	56.3	65.6
Claude Code	1	1	48	13.7	50.0	63.3
Claude Code	4	4	32	4.9	87.5	92.2

Table 7: Parallel exploration on make toast. Agent time until submission, and success rate and score (%) of the fully automated program on 32 unseen scenes.

Task	Budget	RGB-D	RGB-only	$\Delta$
Make toast random	16	2.5	0.0	$+$ 2.5
Make toast random	32	31.7	7.8	$+$ 23.9
Hang mugs random	16	36.9	13.9	$+$ 23.0
Sweep blocks random	16	53.3	33.3	$+$ 20.0
Average		31.1	13.8	$+$ 17.3

Table 8: Depth versus RGB-only perception. Score (%) on unseen layouts at matched exploration budgets. Cells with more than one run report the mean.

Figure 7: From exploration to code. Inference time (left) and output tokens (right) of each successful exploration episode, on base-2 logarithmic vertical axes; for rehearsals, inference time is the program’s computation time as measured by the harness, without the overhead of running in isolation. The token axis is linear below one thousand tokens so that zero can be shown. Filled dots mark episodes in which the agent takes part, alone or together with its program; hollow circles mark rehearsals, in which the program runs alone. On loading the car, raw points are faint and the line is a running median over five successes; output tokens of parallel episodes are attributed to the environment an agent was assigned, or split evenly across the episodes active at that moment.

Task	Actions from	Inference	Robot time	Output tokens
Air fryer	Agent	72.3 min	2.9 min	262.4 k
	Program	18 s	2.5 min	2.0 k
Bandu tower	Agent	17.2 min	9.3 min	88.4 k
	Program	10 s	10.9 min	2.2 k

Table 9: From the agent’s actions to a fully automated program on the real Franka. The first success with actions generated by the agent and a later unattended success of its program. Inference time: wall-clock time outside robot calls. The agent’s Bandu rebuild places six pieces, the program seven.

Method	Demos	Budget	Football	DoubleDesk	P&PBox	OpenDoor	SitSofa	Boxing	VisNavi	Avg.
WB-WAM	100	–	70.0	65.0	86.7	98.3	95.0	81.7	76.7	81.9
RAX	3	32–128	98.3	56.7	83.3	100.0	96.7	100.0	93.3	89.8
RAX ⋆	3	64–256	98.3	46.7	70.0	95.0	100.0	86.7	76.7	81.9

Table 10: HumanoidArena without action demonstrations. Success rate (%) on the 60 official test episodes of each task. RAX: largest budgets of Table 4 . ⋆ Three demonstrations without actions.

Figure 8: Active perception. Simulation, DexJoCo bimanual Tower of Hanoi ( Section 11.2 ): (1) in its first episode, the agent moves single joints of the Allegro hand and finds the white fingertip caps in the wrist camera; (2) from the caps’ points it fits the fingertip offsets relative to the flange, and, without a model of the hand, its thumb tip lies 1.1 mm from the official model; (3) after each placement, the program measures the disk again, on average 0.66 mm below the height it predicted over 150 evaluation episodes. Real robot, ManipulationNet peg-in-hole ( Section 14.4 ): the agent probes with closed fingers to measure the height of the transparent board and the tilt of the table.

Figure 9: Error recovery and safety on real robots. Each incident and the rule the agent derived from it. RB-Y1: a turn planned by hand pushes a cabinet, after which every base move is checked against the area the base sweeps; squatting to lift a bucket overloads a wrist, and standing up first resolves it. Franka: an unusual arm posture hits a camera, after which the agent plans only natural postures that keep the arm away from the cameras.

Figure 10: Human–robot interaction. Requests that the agent sent to the operator, summarized, with the scene before and after. Right: the fryer poses that the agent requested across runs 6 to 10, from its own measurements of the fryer’s axis and handle direction.

Figure 11: Continual learning on the real-world RB-Y1 mobile manipulator. Top: Modular skills acquired during prior exploration sessions and stored in the skill repository, complete with descriptions from their respective skill files. Middle: The final execution of a complex tidy-up task—a single high-level instruction carried out across six sequential stages, annotated with completion timestamps and invoked skills. Bottom: The agent’s generated code for each stage (condensed from execution logs with passed arguments). Code invoking pre-stored skills is color-coded by skill ( chair , cups , bucket , and box correspond to each skill’s run.py ), while ad-hoc control logic written directly by the agent is shown in black.

Figure 12: Adapting skills and programs on the real Franka. Each row is one program that already worked, with the task it learned (left image) and the operator’s change (green). Right: the program’s entry point after the change, condensed from the agent’s code, with new lines in green, modified lines in blue, and reused lines in black. Top: find the cube, where each image overlays frames from the left camera a quarter of a second apart, earlier ones fainter, to show how the object is hidden; the cube is first dropped into a cup and then replaced by a red wedge. Bottom: wiping with paper towels from a box; S and P are the towel program’s modules, imported unchanged.

	Franka FR3	Rainbow RB-Y1
Perception	RGB-D images from two exterior and one wrist camera; pixel to 3D point	Head RGB-D images and gaze control; pixel to 3D point; a vision-language model
Motion	Straight-line end-effector moves; joint moves with path checks	Joint, end-effector, and torso targets; IK over torso and both arms; base drive
Gripper	Grasp to a width with a set force	Close with a current limit, or set an opening
Model	None (the agent writes its own kinematics)	Forward kinematics; dry runs for collision, joint load, and center of mass

Table 11: Tools on the real robots.

Skill	What it asks of the agent
Manual robot control	Look first, explore with small moves, choose each action from the current observation, and confirm every condition of the task before handing off.
In-context action learning	When images of a completed task are supplied, infer the goal and useful strategy from them and adapt it to the current layout, without copying actions.
Experience memory	Record what worked right away, with the observable conditions under which it worked; retrieve it for similar situations and distill it as evidence grows.
Autonomous control	Start from a confirmed manual success, turn it into observation-driven code that can be supervised and resumed, and test it across layouts.
Transfer hand-off	Before exploration ends, write how each subtask and each object is handled, for a later session on new layouts.
Multi-environment exploration	Run parallel environments, track their scenes separately, and share one episode budget.

Table 12: Robot-agnostic skills given to every session.

Task	Benchmark id	Step limit	Partial credit	Evaluation	Agent
Make toast random	make_toast_random	1400	25/50/100	45	Codex
Hang mugs random	hang_mugs_random	800	15/40/100	50	Claude Code
Sweep blocks random	sweep_blocks_random	1000	–	30	Codex
Plug in charger	plug_in_charger	400	–	50	Claude Code
Swap blocks	swap_blocks	700	–	50	Codex
Make kong	make_kong	600	–	50	Codex

Table 13: RoboDojo tasks. Step limit at 25 Hz; partial credit: native score tiers; evaluation: number of official layouts.

Task	Benchmark id	Agent	Step limit	Budget	Envs	Program
Boxing books	boxing_books_up_for_storage	Codex †	36,341	128	6	fully automated
Getting organized	getting_organized_for_work	Claude Code ∗	23,506	128	2	agent-in-the-loop
Sorting items	sorting_household_items	Claude Code	23,711	128	8	agent-in-the-loop
Clearing food	clearing_food_from_table_into_fridge	Claude Code	19,602	128	4	agent-in-the-loop
Clean up desk	clean_up_your_desk	Claude Code	32,126	128	4	agent-in-the-loop
Loading the car	loading_the_car	Claude Code ∗	28,839	128	2	agent-in-the-loop

Table 14: BEHAVIOR-1K tasks and sessions. Step limit at 30 Hz; budget: exploration episodes allowed; envs: parallel environments; program: the program evaluated in Table 3 . ∗ Maximum reasoning effort. † Without demonstration keyframes or the vision-language model; 32 episodes used.

Task	Type	Step limit	Success condition	Budget
Football	HOI	2000	the ball passes the goal line between the posts	32
DoubleDesk	HOI	2000	the hammer rests inside the basket on the second desk	128
P&PBox	HOI	1450	the box rests on the shelf, inside its outline	128
OpenDoor	HSI	1800	the latch is released and the door is open by at least 60 ∘ , robot standing	32
SitSofa	HSI	2000	the pelvis is over the seat and below 0.66 m	32
Boxing	HSI	900	a palm comes within 0.18 m of the target on the bag	32

Table 15: HumanoidArena tasks. Step limit at 50 Hz; budget: largest exploration budget in Table 4 . HOI and HSI: human–object and human–scene interaction.

Task	Benchmark id	Arms	Cameras	Step limit
Hammer	hammer_nail	one	front, wrist	1000
Mouse	click_mouse	one	ego_right, wrist	1200 ∗
Bucket	pick_bucket	one	front, wrist	1200
Tongs	pinch_tongs	one	front, wrist	1000
Glasses	fold_glasses	one	front, wrist	1200
Water	water_plant	one	front, wrist	1000

Table 16: DexJoCo tasks, rand-obj setting. All cameras are RGB-D. ∗ Including 30 steps before control is handed over.

Budget	Interactive	Rehearsal	Used / budgeted	Submitted early	Agent time (h)
2	11	11	22 / 22	0 of 11	0.70 (0.44–1.28)
4	26	18	44 / 44	0 of 11	1.05 (0.51–2.46)
8	57	29	86 / 88	2 of 11	1.34 (0.59–3.02)
16	93	67	160 / 176	5 of 11	1.52 (0.72–2.89)
32	179	101	280 / 352	8 of 11	1.76 (0.98–3.89)

Table 17: DexJoCo sessions by exploration budget. One session per task. Episodes are summed over the eleven sessions; agent time runs from the start of the agent to evaluation and is the median over the eleven sessions, with the range in parentheses.

Figure 13: Perception and planning on a LIBERO-Pro layout, re-run with the agent’s code on the stored frames of one evaluation episode. (a) The agent-view image with Gemini’s points (diamonds) and the geometry that the program derives from depth: the top and axis of each can and the basket rim. (b) The point cloud of the agent-view camera, completed over the basket by the wrist camera and coloured by height above the table on the scale of (c) and (d), with the fitted cans, the basket rim, and the two planned placements, numbered in the order of execution. (c) The basket measured by the wrist camera in 4 mm cells, with the interior outlined. (d) Footprint reach, the highest point under the footprint of a can, and the two spots that minimize it; the circles show the footprint.

Figure 14: Why the program transfers to LIBERO-Pro. (a) Mean object positions over the 50 official and the 50 perturbed layouts, seen from above with the robot at the top; the perturbation exchanges the soup with the ketchup and the tomato sauce with the orange juice. Crosses mark the program’s first estimate of each can in every perturbed episode. (b) The distance between the program’s estimate of a can’s axis and the nearer can in the simulator, for the first agent-view estimate and for the wrist refit before each grasp, against the displacement of the two cans. The task takes the cans in either order, and in two perturbed episodes Gemini exchanged their labels; the rejected refits are those on the ketchup bottle.

Figure 15: Measuring the fingertips with the wrist camera, in the agent’s first episode, re-run on its stored frames. (a) The right wrist camera with the hand in its grasp posture; the agent segments the white fingertip caps and splits them into two clusters. (b) The hand’s point cloud in the frame of the measured flange, with the two clusters and the tip offsets the agent adopted (stars). (c) A side view of the clusters, their centroids and the adopted offsets, with the fingertips of the official hand model, which the agent never saw, for comparison.

Figure 16: The grasps the agent tried, seen by the wrist camera of the grasping hand in its exploration episodes, with panels 3 and 4 rotated so that the peg stands upright. The first four failed for the reasons given below them; the rolled vertical pinch carried both disks and became the grasp of the program.

Figure 17: Perception on an unseen layout and precision over the 150 formal episodes, re-run with the agent’s code. (a) The egocentric point cloud, coloured by height, with the measured disks, the peg axes, the target peg, the thumb and index targets of both pinches, and the predicted seat of the red disk. (b) The height of the red disk measured after placement, minus the height the program predicted. (c) The program’s estimate of the target peg against the hidden pose of the board, both relative to their means over the 150 layouts.

Figure 18: Perception on an unseen DoubleDesk scene, re-run with the agent’s code on the stored frames of one formal episode. (a) The head camera at the start, with the hammer the program finds and, enlarged, its axis and length. (b) The point cloud in the level frame, coloured by height, with the hammer and the bar fitted to it. (c) The last stance before the crossing, seen from above in the frame of the bar: the soles, the window from which the program starts a crossing, and the position of the left sole that the stance model predicted. (d) The basket found at the placing stance.

Figure 19: The stance model and the crossing window. (a) How far a left step from a squared stance places the left sole ahead of the right one, against the step amplitude: the agent’s measurements and the piecewise-linear model it wrote. (b) The positions of the two soles before the bar at every start of a crossing in exploration, rehearsal, and formal episodes, with the window that the program uses.

Versions	Main changes	Failures that motivated them	Completed
up to v8	First full chain; localization from the bar and the basket; crossing window and stance planner; grasp retries; basket approach by turning, then walking	Odometry off by 0.55 m; a walk onto the bar; a fall from 25.6 cm; a false miss that dropped the hammer	4/13
v9–v9g	Left step aimed by the stance model; handle at 60 ∘ ; deadlines; a subagent’s grasp measured at 3/13 against 13/22 and reverted	Late grasps; sweeps of the desk	5/10
v9h–v9k	Looser carrying posture; limited fine steps and backing off; window back to 21.5 cm	The arm over the head; the hammer lost; a fall from 22.4 cm	3/7
v9l–v9o	Limits on the trailing sole; stance model refitted on 60 transitions with a calibration episode; correction of the handle axis	A fall with the trailing sole 41.8 cm back; a handle at 85 ∘ ; a flipped hammer	8/12
v9p–v9t	Trailing-sole margin from the 27/27 against 4/13 record; search turn; stall guard; placing in a partly seen basket	A fall in rehearsal; stalled steps; a basket not seen	6/13
v9u–v9u5b	Re-approach; correction of lateral drift (9.5 cm, $n=11$ ); cycle guard; a faster place near the step limit	Three to six actions at the bar; a rehearsal cut off with 49 steps left	9/18

Table 18: Versions of the DoubleDesk program. From the agent’s own records. Completed counts interactive and rehearsal runs of each group of versions.

Figure 20: The outcome of every stage in every program run, in interactive and rehearsal episodes grouped by version (left), and for the submitted version on the 60 formal episodes, ordered by outcome (right). Each column is one episode; the counts give completed episodes.

Task	Instruction	Episodes	Before $\rightarrow$ after
Long 3	put the black bowl in the bottom drawer of the cabinet and close it	5	0 $\rightarrow$ 100
Long 8	put both moka pots on the stove	5	0 $\rightarrow$ 100
Long 9	put the yellow and white mug in the microwave and close it	4	0 $\rightarrow$ 2
Goal 1	put the bowl on the stove	5	2 $\rightarrow$ 96
Goal 5	push the plate to the front of the stove	4	2 $\rightarrow$ 44

Table 19: Tasks adapted to LIBERO-Pro. Success rate (%) on the 50 position-perturbed states before and after adaptation, with the exploration episodes it used.

Figure 21: Adapting to a mirrored cabinet. (a) An official layout of the third Long task and (b) the same layout under the LIBERO-Pro perturbation, which moves the cabinet to the other side of the table. (c) The adapted program’s check of the side, re-run on both first frames: level cells of the cabinet top above 1.08 m (orange) on either side of the robot, counted on the height map of the agent-view point cloud. (d) The adapted program on the perturbed layout, with the bowl in the drawer.

	ASPIRE	RAX
Programs per task	one per perturbation	one, adapted on 5 of 40 tasks
Instances from the evaluation distribution	30	0, or up to 5 when adapted
Rollouts on these instances	up to 75, plus evolutionary search	0, or up to 5 when adapted
Exploration on unperturbed layouts	–	up to 20 episodes
Evaluation under perturbation	after development on it	zero-shot, or after adaptation

Table 20: Exploration before evaluation under position perturbation. ASPIRE’s counts are its debugging and validation splits for one task and perturbation; evolutionary search adds

K

rollouts per debugging instance and iteration.

Figure 22: Find the cube, trial 9: adapting to a new way of hiding the cube. (a)–(d) Left exterior camera: the operator drops the cube into an upright cup, turns the cup over, and shuffles the four cups. (e)–(f) Right exterior camera: after the agent added a detector for this case, the robot sets the identified cup aside and grasps the cube. The identification comes from the updated detector run on the recorded wrist stream, and the run resumed from locating the cups. Time is from the first frame.

Figure 23: Find the cube, trial 11: a red wedge instead of the cube. (a)–(d) The operator tosses the wedge into an upright cup, turns the cup over, and shuffles; (c) is the wrist camera, which the agent’s detector watches. (e)–(f) On the resumed run, the robot sets the predicted cup aside and grasps the wedge on the first attempt. Left exterior camera except (c); time is from the first frame.

Figure 24: The find-the-cube program. During the shuffle it follows every cup on the wrist camera and remembers the cup that covered the cube (top); it also catches a cube that is moved or dropped into a cup (middle). It then lifts the cup that hides the cube and picks the cube up (bottom).

Figure 25: The air-fryer program. After its elbow hit the left camera in run 2, the agent re-registered the camera and planned only postures clear of both cameras; it releases the banana with the hand tilted 60 ∘ . One program handles four fryer layouts; the bottom row follows the run with the fryer rotated.

Part of the model	In each scene	Source
Position and orientation (axis $c$ , pull direction $d$ )	estimated	lid circle and handle direction in the point cloud
Handle face, arm height, lateral offset	estimated	handle points in the point cloud
Drawer opening (joint state)	measured	shift of the handle face after the pull
Radii, heights, handle dimensions	fixed	the agent’s measurements from its first manual run

Table 21: What the agent’s fryer model takes from each scene.

Figure 26: The agent’s estimators on its final run , computed by its own code. (a) Lid points from the wrist camera above the fryer; blue points are kept and grey points discarded by the trimmed least-squares circle fit (red), whose centre is the axis

c

. (b) Handle-arm points seen from above, around the lid circle; points outside the

\pm 0.25

rad window (dashed) are discarded, and

d

points to the median of the rest. (c) Handle points in the plane along

d

: the outer face in each 1.5 cm height slice (red bars) and the top of the arm (dashed). (d) Banana points along their principal axis, with the fitted centre line, its ends, and its middle (the grasp point). (e) Rim points after opening around the model’s prediction (dashed), and the fitted rim (green), 6.5 mm apart. Distances are in centimetres.

Figure 27: The air fryer task on the real robot. The agent’s final run, from the continuous recording of an exterior camera, with the time since the start of the run under each frame. Each row is one planned action, numbered as in Figure 3 (b): grasping the handle, pulling the drawer open by 13.1 cm in 3 cm steps, carrying the banana and releasing it with the hand tilted

60^{\circ}

over the basket, releasing the ball, and closing the drawer by a push that stops about 2 cm short followed by a push with closed fingers, which leaves the handle within 3.5 mm of its initial position. This camera was moved by a collision earlier in the session and was not used for perception.

Figure 28: Wiping a stain, trial 7b, from the left exterior camera. (a) The poured stain and the towel heap; (b) the grasped wad pressed on the stain; (c) scrubbing; (d) the towel is carried back; (e) the table after one pass. Time is from the first frame.

Figure 29: Wiping with paper towels, second wet run, from the left exterior camera. (a) The poured stain, the tissue box, and the bin behind it; (b) a sheet pulled from the box; (c) six sheets piled on the stain; (d) the pile pressed and grasped as a wad; (e) wiping, with the towel program’s zig-zag; (f) the wad released over the bin; (g) a leftover sheet picked up by the sweep; (h) the table afterwards, with a flat wet scrap the sweep missed. Time is from the first frame.

Figure 30: The wiping program. It finds the stain by comparing both exterior cameras with a clean reference and the towel heap in depth, grasps the heap as a wad, and wipes a zig-zag fitted to the reachable area. The wrist camera catches thin streaks that the exterior cameras miss, so the table is checked at both scales.

Figure 31: The ManipulationNet peg-in-hole challenge, final run, from a crop of the left exterior camera. (a) A peg inserted into L1 (3 mm clearance); (b)–(d) the rectangle peg entering L3 (0.1 mm clearance) corner first, turned upright, and seated; (e) the three pegs seated in L3. Time is from the first frame.

Figure 32: The peg-insertion procedure. The agent fits the board’s CAD model and each peg’s top outline to the wrist camera’s image edges. It drops the peg into the 3 mm row, probes by touch for the 1 mm hole, and tilts a corner into the 0.1 mm hole before turning the peg upright and pushing it down.

Figure 33: Bandu tower, final fully automatic assembly, from the left exterior camera. (a) The pieces after disassembly; (b)–(c) pieces are placed on the base; (d) the knob is held over the urn for visual alignment; (e) the completed tower. Time is from the first frame.

Figure 34: The Bandu assembly program, running fully automatically. It finds the pieces with both exterior cameras and Gemini, keeps a slot model of the demonstrated tower, and places each piece by a guarded place, a tactile search for the seat (with a tilt check for the urn), or a camera alignment of the knob over the urn’s opening.

Scaling Online Exploration for Robot Agents

Abstract

Figures & tables

Robot Learning

AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation

SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation

CAPEX: Efficiently Distilling Foundation Model Behavior into Deployable Robot Policies through Experience-Adaptive Reasoning

Abstract

Robot Learning

Explore similar work

AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation

SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation

CAPEX: Efficiently Distilling Foundation Model Behavior into Deployable Robot Policies through Experience-Adaptive Reasoning