HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution

Kyochul Jang; Seohyeon Park; Ohchul Kwon; Sangjun Park; Junhyeok Choi; Seungyeop Yi; Chaeyun Kim; Sangkyu Lee

cs.ROOct 1, 2026

HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution

Authors: Kyochul Jang, Seohyeon Park, Ohchul Kwon, Sangjun Park, Junhyeok Choi, Seungyeop Yi, Chaeyun Kim, Sangkyu Lee, +4 more

Organizations: Seoul National University · University of Massachusetts Amherst · Google Research

Abstract

As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable tool and coordinating manipulation and, when needed, locomotion to complete the task. Existing benchmarks do not jointly evaluate these capabilities on a humanoid. We introduce HumanoidToolBench, an 18-task benchmark spanning three scenarios, three execution levels, and two tool-set modes, together with ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1. Evaluation of seven policies in simulation and three on the real robot reveals substantial gaps between selecting a suitable tool and completing the task. Focused GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued task execution under unrelated instructions. Code and data are available at https://snu-pi.github.io/HumanoidToolBench/.

Figures & tables

Fig. 1: Overview of HumanoidToolBench . A benchmark for humanoid tool use with 18 tasks spanning three scenarios, three execution levels, and standard and decoy tool sets. With 55 tool assets and 3.1k demonstrations, we evaluate seven policies in simulation and three on the real robot.

		Tool Use			Task Structure		# Human Demos
Benchmark	Embodiment	Tool-Use Scope	# tool assets	Selection	Hierarchical Levels	Decoy Variants	Sim	Real
LIBERO [ liuLIBEROBenchmarkingKnowledge2023 ]	Single-arm	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.1797,0.4883,0.1953}\large\bm{\checkmark}}$	6,500	0
RoboTwin 2.0 [ chenRoboTwin20Scalable2026 ]	Dual-arm	${\color[rgb]{0.7891,0.5938,0}\large\bm{\triangle}}$	11	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.1797,0.4883,0.1953}\large\bm{\checkmark}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	40
HumanoidBench [ sferrazzaHumanoidBenchSimulatedHumanoid2024 ]	Humanoid	${\color[rgb]{0.7891,0.5938,0}\large\bm{\triangle}}$	$1$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	0
SIMPLE [ weiSIMPLESimulationBasedPolicy2026 ]	Humanoid	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.1797,0.4883,0.1953}\large\bm{\checkmark}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	-	-
WOLF-VLA [ boukheddimiWOLFVLAWholeBodyHumanoid2026 ]	Humanoid	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	${\color[rgb]{0.1797,0.4883,0.1953}\large\bm{\checkmark}}$	${\color[rgb]{0.7773,0.1563,0.1563}\large\bm{\times}}$	0	0

TABLE I: Comparison of HumanoidToolBench with related manipulation, humanoid, and tool-use benchmarks.

\bm{\triangle}

, tool use included but not the primary evaluation scope; 0, not applicable or none released; a hyphen, not reported.

Fig. 2: Task structure. 3 scenarios × 3 execution levels × 2 tool-set modes = 18 tasks, built on a library of 55 tool assets spanning three functional categories. Execution levels progress from selection and pickup (L0) to stationary tool use (L1) and mobile tool use (L2). Decoy mode requires choosing the tool that meets the task’s spatial, affordance, or physical requirement.

Simulation: 3,003 trajectories
Scenario	L0 (S/D)	L1 (S/D)	L2 (S/D)
BallMove	289/244	100/100	100/100
BallRetrieve	388/313	100/100	100/100
IceBreak	316/253	100/100	100/100
Real robot: 91 trajectories
Scenario	Execution level	S	D

TABLE II: ToolBook coverage by task.

		Standard (S)			Decoy (D)
Family	Policy	L0	L1	L2	L0	L1	L2
IL	ACT [ zhaoLearningFineGrainedBimanual2023 ]

TABLE III: Simulation success rate (%) by scenario, execution level, and tool-set mode. Within each cell, values are reported in the order BallMove / BallRetrieve / IceBreak. Rows in each family are ordered by mean L1/L2 success. In each column, bold and underlined cells have the highest and second-highest mean over the three scenarios.

Explore similar work

IMBench: A Benchmark for Intuitive Robotic Manipulation
Jul 17, 2026Anurag Maurya, Sukhvansh Jain, Prajwal Avhad +9Robotic Manipulation Robot Policies
Manav Robotics, Bengaluru, India
CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation
Sep 27, 2026Yiheng Lyu, Xueying Jiang, Wenhao Li +2
Nanyang Technological University, Singapore · Independent Researcher
ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies
Jun 15, 2026Zenan Wu, Bingqing Wei, Lu Liu +8Robotic Manipulation Policies Compositional Generalization
1Beijing Academy of Artificial Intelligence · 2Peking University

Jul 17, 2026, cs.ROJ/K move · Enter open · S save

IMBench: A Benchmark for Intuitive Robotic Manipulation

Anurag Maurya, Sukhvansh Jain, Prajwal Avhad, Gautham Balachandran+8

Manav Robotics, Bengaluru, India

Robotic Manipulation Robot Policies Intuition Task-Relevant Information

Open paper AskPDF arXiv

Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints. They build an understanding of the physical world that helps them convert reasoning into actions and quickly adapt to new scenes, tasks, and rules. We refer to this capability as intuitive manipulation. Existing benchmarks fail to capture this integration: they evaluate physical reasoning in isolation from execution, or measure policy performance without requiring explicit reasoning. We introduce IMBENCH, a benchmark designed to evaluate intuitive manipulation as an integrated capability spanning perception, physical reasoning, action generation, and iterative execution. Our tasks require models to infer task-relevant physical structure and generate feasible action sequences under explicit constraints, including contact-rich manipulation, tool use, and multi-stage dependencies. We introduce a benchmark of 35 tasks, 14K filtered trajectories, and scalable tools for generating diverse scenarios. Experiments reveal a consistent gap: vision language models show partial physical reasoning ability but fail to produce executable plans, while state-of-the-art vision-language-action models struggle to satisfy task constraints and generalize across scenarios. These results identify intuitive manipulation as a missing axis in current foundation models and generalist robot policies, and position IMBENCH as a step toward evaluating and enabling more integrated, adaptive physical intelligence.

Abstract

Explore similar work

IMBench: A Benchmark for Intuitive Robotic Manipulation

CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation

ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policies