Robot-use Agents (RUAs) have shown promising zero-shot manipulation capabilities, but reliance on frontier, proprietary models limits research into further training. To democratize RUA research, we introduce uni-robot-agent, a unified harness for competitive open-weight robot use that supports further training. Without privileged inputs, an out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo Score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97 and many VLA/WAM baselines, and performs competitively on 19 real-world tasks. Our analysis across model generations, families, and sizes links effective robot use to four general capabilities, suggesting that future RUA training should develop them jointly.
Figures & tables
Figure 1: uni-robot-agent: results and system overview. Top: shared tools, execution, and feedback connect models to environment adapters, supporting generalizable agentic training across environments. Bottom: Simulation and real-world scores.
Figure 2: Five logical layers of uni-robot-agent. Requests flow from agent runtimes through orchestration and shared tools to domain services and native backends. Context policies and concrete backends are replaceable components; arrows summarize logical interactions rather than individual network connections.
Figure 3: Tool reuse across environments. Shared tools avoid N×M implementations. T : tool; A : adapter; E : environment.
Figure 4: Serial interaction and pipelining. Reasoning ( R ) overlaps sequential tools ( T ); T2 queues while T1 runs. Solid/dashed arrows show submissions/results; shading marks overlap. Timings are schematic.
Figure 5: Key design choices. Each column pairs two designs for physical consistency, action precision, or context management.
Model
Harness
Gen.
Prec.
Long-Hor.
Memory
Open
Avg. ↑
VLA & WAM
π0.5
–
8.17/13.38
5.50/12.40
14.67/23.54
4.67/5.89
1.67/1.98
6.93/11.44
Hy-Embodied-0.5-VLA
–
8.39/11.78
8.00/13.81
14.92/25.74
12.11/13.37
0.58/0.65
8.80/13.07
OpenWAM- α
–
14.83/20.71
9.25/18.45
25.33/34.93
9.11/10.41
1.08/1.41
11.92/17.18
Xiaomi-Robotics-1
–
17.00/23.54
18.83/26.69
23.67/38.39
6.56/7.81
3.58/3.94
13.93/20.07
Agent
Table 1: Results on RoboDojo. Each entry reports SR/Score, where success rate (SR) is a percentage and Score is on a 0–100 scale. Our uni-robot-agent configurations exclude task-verifier feedback. Depth-enabled results are shown in gray for reference; our primary results do not use depth. Avg. is the unweighted mean of each metric over the five capability dimensions. † GAP (Hybrid) incorporates π0.5 .
Figure 6: Representative real-world tasks across five dimensions. Scene photographs and close-ups show task details, with example instructions and illustrative tool calls.
VLA
WAM
uni-robot-agent
Capability
π0.5
Dream-1
GPT-6 Astra
Qwen3.8-Flash-Next
Spatial understanding
11.25
6.50
17.00
9.00
Object generalization
12.50
13.50
17.50
10.00
Instruction following
14.00
9.50
18.00
13.50
Compositional generalization
6.00
10.00
20.00
13.00
Unseen tasks
4.00
4.00
8.00
8.00
Table 2: Results on real-world evaluation. Each capability is scored out of 20; Total is out of 100. Per-task results and scoring details are provided in Appendix B .
Figure 7: Robot-use performance across model generations, families, and sizes. Success rate (top) and Score (bottom) on RoboDojo with uni-robot-agent. Robot use is neither exclusive to frontier models nor unique to the latest generation: GPT-5.6 Sol already achieves nontrivial performance, while Qwen3.8-27B succeeds where larger, earlier Qwen models fail. Shading indicates closed-source models.
Figure 8: Four capabilities associated with effective robot use. Correlation between each model’s RoboDojo Score and its errors in (a) general visual understanding, (b) spatial grounding, (c) coding in geometry, and (d) embodied reasoning. Lower error rates across these dimensions are associated with higher task performance, suggesting that effective robot use draws on a combination of these capabilities.
Figure 9: Robot-use performance and general model capabilities. Each point denotes a model, with benchmark performance on the horizontal axis and RoboDojo Score on the vertical axis. Model colors and symbols follow Figure 8 . Dashed lines show linear fits; Pearson correlations are computed across the six models.
Figure 10: Robot use with less geometric information. Recorded progress through 771 tool calls and final score across 42 tasks, with skills enabled.
Figure 12: Model-level resource–performance trade-offs. Astra, Qwen, and DeepSeek each use 252 episodes. Bars show score and mean resource usage per episode.
Figure 14: Model–action overlap in pipelined execution. Blue and green bars show model requests and physical actions; shading marks overlap. Percentages indicate the fraction of action time overlapping model requests. Each panel starts at the first action and uses its own time range.
Figure 15: Episode duration on five selected tasks. Pipelined and serial execution are compared using one episode per configuration and task, excluding simulator initialization.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Task
Task variation and scoring criterion
Max
Spatial Understanding
20
1
Pick and place (fruits)
Move cherries, bananas, apples, oranges, and melons from bowls or plates into a basket. One point for each fruit type placed successfully in every trial.
5
2
Fold rag
Fold large, small, and obliquely placed rags. One point for each condition folded successfully.
3
3
Bimanual plate pull-out
Lift the bottle with one arm and remove and stack the plate with the other. Each arm’s required action receives 0.5 points per trial.
4
4
Insert test tubes into rack
Transfer test tubes from one rack and insert them into another. Each pickup receives 0.5 points and each insertion 0.5 points.
4
Appendix
Table 3: Complete Real Eval test catalogue. Each dimension has a 20-point budget, and score caps follow the source specification. IDs match Table 4 .
VLA
WAM
uni-robot-agent
ID
Task
Max
π0.5
Dream-1
GPT-6 Astra
Qwen3.8-Flash-Next
Spatial Understanding
1
Pick and place (fruits)
5
2
3
5
3
2
Fold rag
3
1.75
1.5
3
1
3
Bimanual plate pull-out
4
3
0
2
1
4
Insert test tubes into rack
4
2
1
3
2
Appendix
Table 4: Per-task results on Real Eval. Each capability subtotal is out of 20, and Total is out of 100. Task IDs correspond to Table 3 ; scoring details are given in Appendix B.5 .
Model / pricing condition
Currency
Input
Cached input
Output
GPT-6 Astra ( OpenAI, 2026b )
USD
10.00
1.00
50.00
Qwen3.8-Flash ( Alibaba Cloud, 2026 )
CNY
0.80
0.10
2.70
DeepSeek-V4.1-Flash (peak) ( DeepSeek, 2026 )
CNY
2.00
0.04
8.00
DeepSeek-V4.1-Flash (off-peak)
CNY
1.00
0.02
4.00
Appendix
Table 5: Reference API rates for the cost analysis. Rates are per million tokens in the listed currency.
Figure 16: Agent-written triangulation in insert_tubes . Matching pixels marked A define viewing rays, illustrated schematically above the executed code excerpt. Recorded coordinates and reprojection errors connect the calculation to its output; the final frame follows subsequent grasping and insertion.
Figure 17: Geometric hypothesis testing in cover_blocks . The agent varies a candidate table height and computes cup dimensions from selected pixels. The schematic illustrates the ray–plane intersections computed by hit() in the code excerpt; the right-hand frame shows a later placement over the red block.
Figure 18: Grasp adjustment and supported sliding in align_blocks . The upper row summarizes four stages, each illustrated by a representative observation; a stage may contain multiple tool calls. Green and blue bars show physical tools and model requests; shading marks overlap. The marked interval is enlarged below, where the model prepares a clearance check during release. Image timestamps and the main axis are relative to the first displayed observation.
Figure 19: Orientation preparation during digit transport. The upper row summarizes four stages rather than individual tool calls; the lanes below show recorded execution. The enlarged interval shows a model request preparing rotation candidates while transport continues. The later divergence is retained to show how execution feedback affects the prepared plan. Colors and time conventions match Figure 18 .
Figure 20: Score variability with fewer rollouts. Shading shows the central 95% range across 10,000 subsamples of the depth-enabled Qwen evaluation. The dashed line marks the 50-rollout reference; the marker and error bar show the mean and range for six rollouts per task variant.
Coding agents are extending their reach into the physical world by writing and executing robot control programs. One might expect the agents to use the existing mature software stack that engineers have developed over decades to access sensors and control motion. Yet prior work primarily engineers complex custom harnesses to orchestrate agents for robot use, particularly by prescribing specialized workflows and providing bespoke interfaces. This raises the question: "Is such additional harness engineering necessary?" We introduce OpenRUA, a zero-abstraction harness that bypasses bespoke abstraction layers by providing off-the-shelf coding agents with only terminal access to the robot's native software interface ROS 2. OpenRUA employs a minimalist workspace-as-harness design, only offering ROS 2 documentation and basic tools while leaving the coding agent to organize its own work without orchestrating any agentic workflow. Within this workspace, OpenRUA recasts perception as file I/O and manipulation as coding. With Claude Code powered by Claude Opus 5, OpenRUA achieves success rates of 99.0% on CaP-Bench and 87.0% on LIBERO-PRO, demonstrating that an off-the-shelf coding agent can serve as a zero-shot visuomotor policy through the robot's native interface, without bespoke primitives or task-specific training. Under this minimalist design, further analysis reveals striking emergent behaviors of coding agents: (1) For perception, the agent spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes. (2) For manipulation, the agent spontaneously builds motion-control clients (e.g., gripper control) in 95.87% of episodes and closed-loop control programs (e.g., adjusting motion based on sensor feedback) in 50.13% of episodes. Our code is available at https://github.com/terminalworld/OpenRUA.
Zhaoyang Chu, Earl T. Barr, Claire Le Goues +4
University College London · Carnegie Mellon University
Frontier models can control robots, but reasoning through every reach, grasp, and retreat makes manipulation slow and token-intensive. We revisit code as policy with a different division of labor: models build executable tools, code handles multi-phase motions, and models decide what to do next. We introduce URAI (Universal Robot-Agent Interface), which couples a programming agent that constructs robot tools with an execution agent that uses them in a feedback loop. The programming agent writes reusable and task-specific tools from task intent and refines them through execution feedback and human guidance. The execution agent selects and parameterizes these tools from current observations; each call runs a complete motion locally before returning control to the agent. Unlike delegating subsequent decisions to a generated program, this design retains model-level decision-making between tool executions. Validated tool revisions persist across episodes without updating foundation-model weights, and a shared GUI and API make the same tools available to humans and agents. Across five RoboDojo tasks and four frozen execution agents, URAI raises aggregate success from 18.0% to 53.0% relative to direct fingertip control, with the largest gain on Swap Blocks; with the same tools, a program written in advance reaches only 24% against 56% for two agents deciding after each call. Three of the four agents also finish episodes 1.3-1.5 times faster with 1.5-1.7 times fewer execution-agent output tokens; DeepSeek-V4-Flash's cost barely changes. We further evaluate URAI on seven real-world AgileX dual-arm tasks, spanning object manipulation, cloth folding, and human-interactive tic-tac-toe. URAI connects the coding and decision-making capabilities of frontier agents, organizing robot control around reusable tools that agents can both invoke and revise.
Shijia Ge, Alex Zhou, Jianshu Zeng +12
Hexafuture Inc. · Peking University · Beijing Institute of Technology +1
The rise of large language models (LLMs) has opened a path toward generalist embodied AI, raising the question of how LLMs can act in the physical world. One straightforward approach is training the model to generate actions: collect robot-action data and video demonstration, refine LLM architecture with an action head, and train a vision-language-action (VLA) model at scale. However, we argue that coding agents have the potential to realize generalist embodied manipulation. All they need is a simple yet effective robot harness. We present RoboHarness: a robotic harness that gives LLM agents a visual-geometric control panel so that they can directly understand and invoke embodied tasks. With RoboHarness, Qwen3.8-Flash-Next outperforms state-of-the-art VLA models on BEHAVIOR Challenge 2025 tasks. The implementation is open source.