Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks
Organizations: Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Sun Yat-Sen University · Jilin University ·
Abstract
LLM agents increasingly control robots using VLA policies, perception models, and procedural knowledge. These modules are difficult to reuse because their interfaces, embodiment assumptions, and conditions of use differ, and their benefits on physical tasks remain unclear. We package them as embodied skills that extend agent skills with explicit embodiment assumptions and conditions of use, building 250 action, perception, and experience skills from 1,052 resources. On this basis we introduce EmbodiedSkillBench, which runs an agent in a closed observation-action loop and records success, actions, tokens, and wall-clock time. It supports three modes: native control, skill-only execution, and Agent-Directed Reuse (ADR), a mode we propose in which the agent keeps its native controls and decides for each skill whether to invoke, modify, or ignore it. Across four agents on 100 perturbed LIBERO-Pro tasks, we find that skills help an agent only as far as they exceed what it can already do. Skill-only execution lifts GPT-5.6-Sol from 11% to 63% and Kimi-K3 and Claude-Opus-5 by about 20 points, while cutting their token usage by 58% to 85%. Yet all four agents then end up between 61% and 64%, and GPT-6-Astra falls from 92% under native control to 61%. This suggests that as agents grow stronger, the role of embodied skills shifts from supplying missing capability to reducing cost: GPT-6-Astra already solves most tasks through native control, so restricting it to skills removes more than it adds. Under ADR, which lets it keep native control and call skills selectively, GPT-6-Astra recovers to 85%, within 7 points of native control, while using 14.1% fewer tokens and 11.9% less time. With Jev as a skill choosing motion directions and gripper actions, it reaches 81% with 50% fewer tokens and 57% less time. The code is available at https://github.com/HanZhi306/EmbodiedSkill-Bench.
Figures & tables
| Domain SR (%) | Perturbation SR (%) | Overall | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Setting | L10 | Goal | Object | Spatial | Lang. | Object | Task | Swap | SR (%) | Actions | Tokens (k) | Time (min) |
| Selected | 41.7 | 67.9 | 79.2 | 62.5 | 92.0 | 84.0 | 40.0 | 36.0 | 63.0 | 2.38 | 803.2 | 4.40 |
| BM25 | 4.2 | 46.4 | 33.3 | 58.3 | 68.0 | 44.0 | 8.0 | 24.0 | 36.0 | 8.09 | 2,731.6 | 9.76 |
| Embedding | 25.0 | 39.3 | 45.8 | 58.3 | 72.0 | 64.0 | 8.0 | 24.0 | 42.0 | 12.73 | 2,918.7 | 11.03 |
| HarnessVLA (ESB) | 50.0 | 78.6 | 100.0 | 79.2 | 92.0 | 56.0 | 80.0 | 80.0 | 77.0 | 14.70 | 2,113.8 | 14.12 |
| Zetta (ESB) | 33.3 | 78.6 | 91.7 | 70.8 | 72.0 | 60.0 | 68.0 | 76.0 | 69.0 | 14.51 | 2,399.7 | 16.66 |
| Domain SR (%) | Overall | |||||||
|---|---|---|---|---|---|---|---|---|
| Configuration | L10 | Goal | Object | Spatial | SR (%) | Actions | Tokens (k) | Time (min) |
| With skill | 37.5 | 67.9 | 79.2 | 58.3 | 61.0 | 2.24 | 563.5 | 3.40 |
| With Jev skill | 58.3 | 92.9 | 79.2 | 91.7 | 81.0 | 20.90 | 1,142.0 | 8.74 |
| No skill | 70.8 | 100.0 | 100.0 | 95.8 | 92.0 | 16.59 | 2,300.6 | 20.52 |
| Agent-Directed Reuse | 62.5 | 89.3 | 95.8 | 91.7 | 85.0 7.6% | 12.87 22.4% | 1,976.3 14.1% | 18.09 11.9% |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Interface | Function supplied to the agent |
|---|---|
| observation, end-effector state | Current instruction, RGB, status, end-effector pose, and gripper state. |
| camera calibration, pixel depth, pixel-to-world projection | Camera intrinsics/extrinsics, metric depth, and calibrated back-projection of a selected pixel. |
| gripper contact | Identity-free gripper-width/contact evidence; the agent checks the images to assess a grasp. |
| IK pose check | Read-only feasibility check of a proposed full pose. |
| relative end-effector motion, absolute end-effector motion, full-pose IK motion | Bounded relative translation, absolute Cartesian motion, and full-pose IK motion. |
| gripper control | Open or close the gripper while holding the measured arm joints. |
| Configuration | Outcome | Actions | Tokens (k) | Time (min) |
|---|---|---|---|---|
| gpt-5.6-sol / no skill | Failure | 40 | 7,250.6 | 13.31 |
| gpt-5.6-sol / with skill | Success | 3 | 702.0 | 4.33 |
| Kimi-K3 / no skill | Failure | 25 | 4,736.5 | 87.70 |
| Kimi-K3 / with skill | Success | 3 | 317.5 | 3.35 |
| claude-opus-5 / no skill | Failure | 13 | 3,036.4 | 19.46 |
| claude-opus-5 / with skill | Success | 3 | 364.8 | 4.86 |
| Component | Embodied skill implementation |
|---|---|
| Free-space motion (both) | move to, move pose, rotate wrist, rotate pitch. |
| Gripper transitions (both) | set gripper, release. |
| Local VLA contact (both) | pi0 pick, pi0 doubled; Zetta (ESB) additionally installs VLA object grasp. |
| Current-scene perception (both) | SAM3 object localization, invoked as a separate read-only query. |
| HarnessVLA (ESB) planning | HarnessVLA (ESB) memory-guided workflow, with a global rule reference and task-memory schema. |
| Zetta (ESB) online orchestration | Zetta (ESB) closed-loop evolution workflow, Zetta (ESB) runtime critic, Zetta (ESB) Role1 recovery review. |
| Configuration | Outcome | Actions | Tokens (k) | Time (min) |
|---|---|---|---|---|
| Selected | Failure | 2 | 592.8 | 3.75 |
| BM25 | Failure | 16 | 3,989.0 | 15.42 |
| Embedding | Failure | 0 | 72.1 | 0.56 |
| HarnessVLA (ESB) | Success | 8 | 839.6 | 5.99 |
| Zetta (ESB) | Success | 9 | 1,334.9 | 7.01 |
| Configuration | Outcome | Actions | Tokens (k) | Time (min) |
|---|---|---|---|---|
| With skill | Failure | 6 | 1,648.8 | 9.01 |
| Agent-Directed Reuse | Success | 10 | 736.8 | 5.79 |
| No skill | Success | 23 | 3,350.8 | 33.98 |
| With Jev skill | Success | 14 | 632.8 | 5.42 |
| Waypoints | Sim. steps | Observed stage and transition |
|---|---|---|
| 1–3 | 0–110 | Ground the intended bowl, align, and descend to the first grasp pose. |
| 4–5 | 110–155 | Close and attempt a lift; the returned image shows an empty grasp. |
| 6–7 | 155–225 | Re-ground the shifted bowl and descend to a deeper contact pose. |
| 8–10 | 225–270 | Close, maintain closure, and verify a lift with the bowl retained. |
| 11–12 | 270–340 | Transport the bowl and align it above the plate. |
| 13–14 | 340–386 | Lower and settle; the official success condition becomes true. |