cs.RONov 5, 2026
SaveRoboARCade: Benchmarking Agentic Robot Control
Organizations: Stanford University
Abstract
Improvements in multi-modal capabilities have enabled agents to act directly as robot policies, a setting we refer to as agentic robot control (ARC). We introduce RoboARCade, a benchmark of long-horizon manipulation tasks across three embodiments designed to test in-context adaptation, memory, and reasoning for precise control. The tasks build on existing open-source benchmarks, which we make harder by widening the range of initial conditions. RoboARCade is easy to use and extend: all tasks run on a single simulation backend, MuJoCo, each task is a self-contained data pack, and agents connect through interchangeable interfaces, so new tasks, agents, and interfaces can be added independently. The benchmark separates leading agents: GPT-6 Astra achieves 49.8% compared with 43.6% for Claude Fable 5.1, leaving substantial room for improvement. Comparing two interfaces, we find that a richer visual interface adds only 1.5 to 6.8 points at 1.7 to 2.3 times the cost. Failures stem mostly from choosing a poor manipulation strategy or executing a good one imprecisely, rarely from misunderstanding the scene, and agents incorrectly claim success on 24% to 39% of attempted subtasks. Agents can also improve without human input: one successful demonstration in context raises Astra's score from 53.5% to 73.0% on a five-task subset, and a prompt the agent writes and revises itself reaches 72.0%. Code, videos and full traces are available at roboarcade.org.
Figures & tables
Figure 1: Representative tasks from RoboARCade. The benchmark includes 22 tasks spanning single and bimanual manipulation, assembly, physical puzzles, and biolab-inspired procedures. This figure shows 20 of them. Tasks require combinations of visual understanding, precise control, memory, and multi-step planning. RoboARCade supports rendering with Blender for high-fidelity visual observations.
Figure 2: Different settings for robot control with FMs. Hierarchical VLA: a high-level VLM issues language subgoals to a VLA policy. Code-as-Policies: an FM generates a program that controls the robot through perception and action primitives. The FM is not involved during policy execution. Agentic robot control (ARC): the FM itself is the policy, observing the scene and issuing controller targets in a closed loop. This work focuses on the ARC setting.
| Model | Interface | Success (%) | Cost ($) | Turns/EEF | Calls/EEF |
|---|---|---|---|---|---|
| GPT-5.6-Sol | VIA | 14.0 | 25.05 | 3.6 | 3.3 |
| Claude Opus 5 | VIA | 30.3 | 60.00 † | 4.9 | 5.5 |
| Claude Fable 5.1 | Multi-view RGBD | 36.8 | 33.16 | 1.0 | 1.7 |
| VIA | 43.6 | 55.50 | 2.2 | 7.0 | |
| GPT-6-Astra | Multi-view RGBD | 48.3 | 13.89 | 1.3 | 1.3 |
| VIA | 49.8 | 31.84 | 2.3 | 3.0 |
Table 1: Benchmark on RoboARCade . All metrics are averaged over 22 tasks with 5 seeds each, over all trials. Success measures completed rubric subtasks. Turns/EEF and Calls/EEF divide a trial’s model turns (requests to the model, each of which can issue several tool calls) and tool calls by the number of distinct end-effector (EEF) commands the agent gives the simulator. GPT-6 Astra has the highest success rate even using a minimal multi-view RGBD interface. † A lower bound: two Opus trials each lost the usage record of one interrupted response. Success-only metrics, EEF command counts and wall time per command are in Appendix Table 6 .
Figure 3: Failure Breakdown for RoboARCade. A-C : These plots are over all rubric subtasks, including those the agent never reached, the share succeeded (A), with correct visual reasoning and approach (B), and with correct visual reasoning (C) within the first tries at the subtask. Success requires correct visual reasoning, approach and precision. GPT-6 Astra generally performs best. More tries mainly improve approach and success. Visual reasoning changes little across agents. D: This plot breaks down agent behavior on subtasks it attempted. Stronger agents are more likely to succeed on later tries. When they get stuck, weaker agents fail more on approach, while stronger agents fail about as often on approach as on precision. Stronger models also make fewer incorrect success claims, though GPT-6 Astra still incorrectly claims success on 24% of subtasks.
| Method | Success (%) | Cost ($) | Turns/EEF | Calls/EEF |
|---|---|---|---|---|
| GPT-6-Astra baseline | 53.5 | 27.60 | 2.5 | 3.4 |
| In-context success demo (ICL) | 73.0 | 30.55 | 3.1 | 4.2 |
| Text prompt, round 1 (T1) | 62.5 | 30.76 | 3.0 | 3.3 |
| Text prompt, round 2 (T2) | 72.0 | 22.75 | 2.3 | 3.2 |
| No successful episode available | ||||
| GPT-6-Astra baseline | 0.0 | 48.00 | 3.6 | 4.1 |
Table 2: Autonomous improvement on RoboARCade . Top: We evaluate GPT-6-Astra where metrics are averaged over five partially successful tasks with 10 trials per task and method. ICL loads a full successful episode before the agent attempts the task and raises the mean rubric score from 53.5% to 73.0%. A text-only prompt reaches 72.0% after two rounds of revision using graded rollouts (T2). Bottom: For a task with no successful Astra episode, T1 prompt is written from the five failed baseline trials and the T2 prompt from those and the five T1 trials, evaluated on 5 fresh trials each. Although, a small sample size, T2 is able to achieve 100% performance for a task that had zero full successes. Full metrics are in Appendix Table 7 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Tasks | Robot | Assets and code |
|---|---|---|---|
| AutoBio ( Lan et al., 2026 ) | PCR setup, Start incubation, Vortex sample | Panda | Digitized lab instruments and consumables from AutoBio https://github.com/autobio-bench/AutoBio |
| Fabrica ( Tian et al., 2025 ) | Beam, Cooling manifold, Gamepad, Stool | Panda | Multi-part assembly CAD models from Fabrica https://github.com/yunshengtian/Fabrica |
| LIBERO ( Liu et al., 2023 ) | Serve warm drinks, Start making breakfast, Tidy counter | Panda | Kitchen scenes and objects from LIBERO https://github.com/Lifelong-Robot-Learning/LIBERO |
| OGBench ( Park et al., 2025 ) | Lights Out (6 by 7) | Panda | The puzzle-4x6 button board of OGBench, extended to 6 by 7, https://github.com/seohongpark/ogbench |
| SPD | Toss bottles in bin, Cup pyramid, Handover dominos, Jenga tower, Hang mugs and rack plates | Bimanual YAM | Bimanual VR teleoperation scenes and objects from SPD https://spd.bot |
| RoboDojo ( Chen et al., 2026b ) | Strongbox, Organize table and plug in charger, Pack tools into toolboxes, Sort Lego bricks | Bimanual Panda; bimanual YAM for Strongbox and Sort Lego | Task objects from RoboDojo https://github.com/RoboDojo-Benchmark/RoboDojo and the Lego bricks from https://abc.bot |
Table 3: Task sources and asset credits. The benchmark, paper and repository each of the 22 tasks draws its scene and objects from, and the embodiment it runs on.
| Previous try failed by, then next try | Sol VIA | Opus VIA | Fable RGBD | Fable VIA | Astra RGBD | Astra VIA |
|---|---|---|---|---|---|---|
| After a precision miss (share of retries) | 50% | 50% | 52% | 57% | 52% | 55% |
| then done | 9% | 12% | 27% | 37% | 26% | 33% |
| then precision miss | 66% | 67% | 56% | 46% | 52% | 50% |
| then wrong approach | 21% | 21% | 15% | 17% | 21% | 16% |
| then wrong object | 4% | 0% | 2% | 0% | 2% | 1% |
| After a wrong approach (share of retries) | 42% | 44% | 40% | 36% | 43% | 41% |
Table 4: Where a retry lands. Each retry is a try at a subtask that follows a failed try at the same subtask, grouped by why the previous try failed. For each group, the share of retries that ended done, in a precision miss, in a wrong approach or on the wrong object. Shares are computed per task, then averaged over tasks. After a wrong object, only 5 to 14 retries per model exist, so those shares are noisy.
| Sol VIA | Opus VIA | Fable RGBD | Fable VIA | Astra RGBD | Astra VIA | |
| Ended the episode believing it was done, but not solved | 15/24 (63%) | 34/57 (60%) | 24/53 (45%) | 26/64 (41%) | 25/66 (38%) | 20/60 (33%) |
| Left a named single-object subtask undone for another subtask | 58/77 (75%) | 37/73 (51%) | 56/95 (59%) | 24/75 (32%) | 70/118 (59%) | 50/105 (48%) |
| Ended the episode saying it could not do it (not counted) | 6 | 6 | 4 | 4 | 6 | 1 |
| Subtask verification accuracy | 27.7% | 45.4% | 45.9% | 64.0% | 48.4% | 57.6% |
Table 5: Subtask verification. Wrong decisions over all decisions of each kind. Subtask verification accuracy is the share of right decisions over both kinds together. Budget cut-offs and episodes the agent ended saying it could not finish are left out, since they claim nothing.
| Task | Baseline | ICL | T1 | T2 |
|---|---|---|---|---|
| Beam | 5 / 2 | 8 / 1 | 8 / 1 | 9 / 0 |
| PCR setup | 5 / 2 | 8 / 0 | 5 / 1 | 5 / 0 |
| Cup pyramid | 3 / 7 | 5 / 4 | 5 / 3 | 7 / 3 |
| Stool | 3 / 2 | 4 / 1 | 3 / 0 | 4 / 1 |
| Handover dominos | 3 / 6 | 7 / 0 | 5 / 4 | 7 / 1 |
| All (of 50) | 19 / 19 | 32 / 6 | 26 / 9 | 32 / 5 |
Table 8: Autonomous improvement by task. Trials solved / trials that exhausted their simulation-step or wall-clock budget, out of 10 per task and 50 in total. Failures not ended by the budget were ended by the agent itself, declaring the task done: the “success claim” rows give their number and share of that arm’s failures. Gamepad, run separately with 5 trials per arm and no ICL arm, has no successful Astra episode: its prompts are written from failed attempts.
Figure 4: Top: This plot includes all rubric subtasks, even those the agent never reached. It shows the share with correct visual reasoning, the share with both correct visual reasoning and approach, and the share succeeded (i.e. correct visual understanding + approach + precision) in at least one of the first tries, so every level grows with . GPT-6 Astra generally performs best. More tries mainly improve likelihood of correct approach and success. Visual reasoning changes little. Bottom: This plot breaks down agent behavior on subtasks it attempted. Stronger models succeed at more subtasks on the first try or through monotonic progress, without reverting to an incorrect approach after finding the right one. When they get stuck, visual reasoning errors are rare (2 to 6%), and approach errors are as common as or more common than precision errors for four of the six agents. Stronger models also make fewer incorrect success claims, though GPT-6 Astra still claims success on 24% of subtasks it did not succeed at.