RoboHarness: A Simple Harness Outperforms VLA and World Action Models

Bince Qu; Wei Chen; Bo Zhang

cs.ROSep 29, 2026

RoboHarness: A Simple Harness Outperforms VLA and World Action Models

Authors: Bince Qu, Wei Chen, Bo Zhang

Organizations: Zhejiang University

Abstract

The rise of large language models (LLMs) has opened a path toward generalist embodied AI, raising the question of how LLMs can act in the physical world. One straightforward approach is training the model to generate actions: collect robot-action data and video demonstration, refine LLM architecture with an action head, and train a vision-language-action (VLA) model at scale. However, we argue that coding agents have the potential to realize generalist embodied manipulation. All they need is a simple yet effective robot harness. We present RoboHarness: a robotic harness that gives LLM agents a visual-geometric control panel so that they can directly understand and invoke embodied tasks. With RoboHarness, Qwen3.8-Flash-Next outperforms state-of-the-art VLA models on BEHAVIOR Challenge 2025 tasks. The implementation is open source.

Figures & tables

Figure 1: RoboHarness plugs into coding-agent harnesses such as Claude Code, tracks keypoints as geometric constraints, and verifies execution in a closed loop. On the first 10 BEHAVIOR Challenge 2025 tasks, Qwen3.8-Flash-Next attains a higher mean $q$ -score than the state-of-the-art VLA models shown.

Figure 2: Visualization of the unified coordinate frame on two embodiments: BEHAVIOR R1 Pro ( Li et al., 2024 ) (left) and RoboDojo ARX X5 ( Chen et al., 2026 ) (right).

Figure 3: Pipeline of the track_object tool: optical-flow keypoint tracking followed by depth back-projection into the robot coordinate frame.

Mode	Input points	Description
coordinate	arbitrary	move points to satisfy the given coordinate constraints

Table 1: Modes for keypoint-constraints tool

Name	Parameters	Meaning
adjust left eef pose	$(\Delta x,\Delta y,\Delta z,r,p,y)$	increment the left end-effector pose or rotation
adjust right eef pose	$(\Delta x,\Delta y,\Delta z,r,p,y)$	increment the right end-effector pose or rotation
adjust chassis	$(\mathrm{fwd},\,\mathrm{side},\,\mathrm{spin})$	move the chassis forward, sideways or spin
adjust height	$\Delta h$	raise or lower the body
adjust pitch	$\Delta\theta$	change the body pitch

Table 2: Adjustment tools. Displacements are incremental; chassis spin is a yaw increment.

Figure 4: Three predefined gripper points $\{F_{L},F_{R},G\}$ and a contact pose $(f_{l},f_{r},\theta)$ .

Figure 5: Two non-parallel line coincidences fix a unique $T\in SE(3)$ .

ID	Task	RLC	Comet	$\pi_{0}.5$ 4ep	G0.5 1ep	RoboHarness
0	turning on radio	0.6000	1.0000	0.1500	0.3000	0.4000
1	picking up trash	0.6667	0.7667	0.5500	0.8167	0.8667
2	putting away Halloween decorations	0.2000	0.5000	0.3714	0.4857	0.6857
3	cleaning up plates and food	0.1857	0.0000	0.3857	0.1786	0.2571
4	can meat	0.0000	0.0000	0.0333	0.0111	0.0000
5	setting mousetraps	0.3333	0.0000	0.1083	0.5083	0.8333

Table 3: BEHAVIOR Challenge 2025 task results. The RoboHarness column is Qwen3.8-Flash-Next, and each entry is the five-instance mean in Appendix A .

Figure 6: The 100 objects used to evaluate generalization across objects. The catalog is drawn from the BEHAVIOR object library.

Figure 7: Successes out of 100 objects versus estimated hours of robot data. Colors: VLA (blue), world action model (green), RoboHarness (red), and the ASPIRE-pick baseline (gray). The hour conversion is in Appendix C . Qwen3.8-Flash-Next and Qwen3.8-27B both have $0$ robot-data hours. Their markers are drawn $0.14$ axis units left and right of $0$ , respectively, so the two points remain visible.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Task	Max ticks
0	turning on radio	4299
1	picking up trash	10535
2	putting away Halloween decorations	27664
3	cleaning up plates and food	27392
4	can meat	23694
5	setting mousetraps	20343

Appendix

Table 4: Official maximum tick count for each BEHAVIOR Challenge 2025 task in this sample. The episode stops at this count unless the goal is already satisfied.

ID	Task	301	304	306	308	310	Mean
		slot 0	slot 3	slot 5	slot 7	slot 9
0	turning on radio	0.0000	0.0000	0.0000	1.0000	1.0000	0.4000
1	picking up trash	1.0000	0.3333	1.0000	1.0000	1.0000	0.8667
2	putting away Halloween decorations	0.5714	0.7143	0.4286	0.7143	1.0000	0.6857
3	cleaning up plates and food	0.2857	0.2857	0.2857	0.1429	0.2857	0.2571
4	can meat	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000

Appendix

Table 5: Per-instance

q

-score for the five-instance RoboHarness sample. Slots

0,3,5,7,9

are instances

301,304,306,308,310

Skill	Description
behavior-v2-baseline	contract of scene perception, task decomposition, primitive selection, and execution monitoring.
navigate-to-target	Drives the chassis toward a marked target.
open-doors-and-drawers	Open hinged doors or drawers.
pick-up-object	Plans and executes a grasp of a visible object.
place-object-in-container	Aligns a held object over an open container and releases it only after the object has settled inside.
traverse-narrow-passages	Drives chassis through tight doorway .

Appendix

Table 6: Embodied skills shipped with the embodied_claude_code plugin, excluding the trivial single-object skills close-box , cut-object , and stand-trash-can-upright . Each row is condensed to one sentence describing what the skill does.

#	Object	27B	Flash	luna	$\pi_{0.5}$	F-WAM	G0.5	OWAM	Ling	OVLA	ABot	Xiao	XVLA	Pick
1	alarm_clock	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
2	apple	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\times$
3	avocado	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	$\times$
4	banana	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	$\times$
5	bar_soap	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
6	baseball	$\times$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$

Appendix

Table 7: Per-object outcome on the 100 objects. A check is a success under the

5

cm rule and a cross is a failure. Columns are Qwen3.8-27B, Qwen3.8-Flash, gpt-6-luna,

\pi_{0.5}

, Faster-WAM, G0.5, OpenWAM-

\alpha

, LingBot-VA, OpenVLA-OFT, ABot-M0, Xiaomi-Robotics-0, X-VLA, and the ASPIRE pick function.

Agentic AI for Robotics

See all 39 papers

Show-Harness: Just a VLM Agent Can Play Robots
Sep 9, 2026Yanzhe Chen, Zechen Bai, Zhijun Cao +7Cross-Embodiment Robot Learning Language-Conditioned Robot Control
Show Lab, National University of Singapore
World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal
Sep 24, 2026Yehang Zhang, Haojian Huang, Yifan Chang +13Tool-Using Vision-Language Agents Robotic Manipulation
HKUST(GZ) · Knowin AI · CUHK
Demonstration-Free Robotic Control via LLM Agents
Jan 28, 2026Brian Y. Tsui, Alan Y. Fang, Tiffany J. HwuRobotic Control Language Model-Based Control
All authors are independent researchers.

Sep 9, 2026J/K move · Enter open · S save

Show-Harness: Just a VLM Agent Can Play Robots

Yanzhe Chen, Zechen Bai, Zhijun Cao, Wenzheng Zeng+6

Show Lab, National University of Singapore

Cross-Embodiment Robot Learning Language-Conditioned Robot Control VLMs for Robotics Vision-Language Agents Agentic AI for Robotics

Open paper AskPDF arXiv

Foundation vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play" robots through a compact semantic interface linking intent to action. Show-Harness exposes discrete semantic action units that VLMs can naturally reason over, while embodiment-specific interpreters deterministically ground them into local robot actions, keeping the VLM directly responsible for fine-grained physical decisions. Through the same interface, Show-Harness demonstrates the feasibility of (1) directly unlocking closed-source frontier VLMs for zero-shot robot control, and (2) adapting small-scale open-source VLMs for low-cost deployment with just a few GPU-hours of fine-tuning. We further develop GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to "play" robots across embodiments without specialized teleoperation hardware. Extensive experiments show that Show-Harness-equipped VLM agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and VLA paradigms. These results suggest that the right interface can unlock substantial embodied capability from foundation VLMs, without requiring additional model capacity or costly embodiment-specific pretraining.

Abstract

Agentic AI for Robotics

Explore similar work

Show-Harness: Just a VLM Agent Can Play Robots

World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal

Demonstration-Free Robotic Control via LLM Agents