SwapBot: Tool-Centric Robot Manipulation via Interchangeable End-Effectors and Tactile Sensing

Yunchu Zhang; Ray Song; Junhan Jiao; Liyiming Ke; Yuchen Yang; Ruihan Gao; Emma Romig; Siddhartha Srinivasa

cs.ROOct 29, 2026

SwapBot: Tool-Centric Robot Manipulation via Interchangeable End-Effectors and Tactile Sensing

Authors: Yunchu Zhang, Ray Song, Junhan Jiao, Liyiming Ke, Yuchen Yang, Ruihan Gao, Emma Romig, Siddhartha Srinivasa,

Organizations: University of Washington

Abstract

Robot manipulation in everyday environments inherently involves substantial tool usage. Seemingly simple household tasks, such as cooking pan-fried tofu, require forceful cutting, delicate brushing, and precise placement using different tools under contact-rich interaction. Thus, the ability to autonomously swap and use task-appropriate tools is important for tool-centric manipulation. We introduce SwapBot, a modular, tool-centric end effector system featuring low-cost adapters that quickly convert common household tools into robot end effectors. Rather than pursuing a single universal gripper, SwapBot adopts a unified design principle that combines a fixed robot-side coupling interface with low-cost, tool-side adapters, enabling compatibility with diverse tools, including articulated ones, and multiple robot platforms. Each adapter integrates compact tactile sensors, supports fast autonomous swapping, and interfaces with an intuitive teleoperation system for efficient demonstration collection. User studies show that, during teleoperation, SwapBot reduces task completion time and operator workload. Using collected demonstrations on the robot, we train imitation learning policies to successfully perform tool-centric tasks like cooking tofu, stirring salad, flipping meat. Our ablation study shows that using tactile-augmented demonstrations improves policy performance across contact-rich skills, increasing success rates by 15 to 25 percentage points over RGB-only policies. These results highlight modular, tactile-equipped tools as a practical and extensible pathway toward robust and capable robot manipulation. Project website https://personalrobotics.github.io/SwapBot.

Figures & tables

Fig. 2 : ( Left ) SwapBot setup overview with the robot arm, robot-mounted tool adapter, electromagnetic coupling, the tool dock, and tactile I/O board. ( Middle ) The structure of the instrumented tools with integrated tactile sensing using piezoresistive film, showing sensor layers and placements, with examples for a knife, brush, and tongs. ( Right ) Characterization of tactile sensors, showing force–response curves for electrodes of different sizes. Larger electrode areas exhibit higher sensitivity, but also cause non-smooth signals with sudden jumps under load. Smaller electrodes have smoother signals and a wider measurable range.

Fig. 3 : SwapBot can be adapted to three different types of robots: Tycho, UR5, and Franka.

Fig. 4 : Overview of the hierarchical policy learning framework. The high-level tool-selection policy $\pi_{\text{sel}}$ takes top-view observations $\mathcal{O}_{top}$ and a goal image $\mathcal{O}_{g}$ as input and outputs a distribution over candidate tools. The selected tool index $\tau$ activates the corresponding low-level tool-specific control policy $\pi_{\text{tool}}^{\tau}$ . Each policy conditions on visual observations $\mathcal{O}_{t}$ , tactile feedback $\mathcal{T}_{t}$ , and proprioceptive state $s_{t}$ to generate end-effector actions. During long-horizon execution, the robot continuously conducts tool selection and skill execution in a closed-loop, which enables SwapBot to compose tool-specific skills for long-horizon tasks.

Feature	Specification
Fabrication Accessibility	3D-printed PLA, TPU, and Silicone, assembled with basic hand tools
Compatibility	Franka, UR, Tycho and other common platforms
Tool Support	Passive (knives, brushes, forks, spoons) and articulated (tongs, chopsticks)
Coupling & Tool Swapping	Self-aligning electromagnetic hub with kinematic gear
Retentive Force	Up to 50 N via electromagnet
Swap Time	$\sim$ 10 s with docking procedure

TABLE I : SwapBot End-Effector Specifications

Fig. 5 : Tool-centric teleoperation for two challenging and complex long-horizon cooking tasks: Dish 1: Celery Beef and Dish 2: Potato and Pepper Stir-Fry.

Fig. 6 : User study results for brushing (a–c) and cutting (d–f) tasks across three skill levels of participants. Error bars indicate 95 $\%$ confidence intervals across participants. We compare a general teleoperation tool, SwapBot without visual tactile feedback, and SwapBot with real-time visualized tactile feedback. Overall, the SwapBot interface yields faster completion times, higher success rates, and lower subjective workload than the general teleoperation tool baseline.

Panel of Figure 7 — Fig. 7 : Policy learning performance with and without tactile sensing. We average success rates over 20 evaluation rollouts per seed and run three random seeds; we then apply paired $t$ -tests for significance ( $*$ $p<0.1$ , $**$ $p<0.05$ , $***$ $p<0.01$ ). As the amount of data increases, tactile signals consistently provide a boost over RGB-only baselines and yield larger gains in low-data regimes.

	Brushing (num of demos)					Tong picking (num of demos)					Knife cutting (num of demos)
	20	40	60	80	100	20	40	60	80	100	20	40	60	80	100
Tactile Only	0.00	0.00	0.00	0.05	0.10	0.00	0.00	0.05	0.10	0.10	0.00	0.00	0.10	0.15	0.20
Egocentric RGB w/ Tactile	0.00	0.00	0.00	0.20	0.40	0.00	0.05	0.10	0.10	0.20	0.05	0.15	0.10	0.10	0.15
Top RGB w/ Tactile	0.00	0.15	0.20	0.20	0.50	0.05	0.30	0.25	0.30	0.35	0.20	0.40	0.40	0.40	0.50
RGBs only	0.00	0.30	0.35	0.45	0.70	0.10	0.10	0.30	0.40	0.45	0.10	0.20	0.15	0.50	0.75
RGBs w/ Tactile (Ours)	0.40	0.55	0.70	0.75	0.85	0.40	0.40	0.40	0.45	0.70	0.10	0.40	0.40	0.55	0.90

TABLE II : Comparison of success rates across baselines. We report single-seed results for all modality ablations in the table. To further assess robustness, we also run two additional seeds for RGB-only and our method and present their variance and statistical significance in Fig. 7 . We bold the best results in each column.

Method	Success Rate	Method	Success Rate
Egocentric RGB w/ Tactile	0.10	RGBs only	0.35
Top RGB w/ Tactile	0.20	RGBs w/ Tactile (Ours)	0.60

TABLE III : Overall success rates for whole-task execution by chaining the best-performing policy checkpoint from each stage.

Robotic Tool Use

Robotic tool use studies how robots select, grasp, and operate tools to perform tasks beyond their native capabilities.

See all 45 papers

May 28, 2026

Any-ttach: Quick End-effector Swapping Enables Manipulation Dexterity with Simplicity

Robotic manipulation dexterity is often pursued by building increasingly complex high-DoF multifingered hands. While many robotic hands are designed to replicate human morphology, the functional role of human hands suggests a different perspective: much of their complexity may exist to enable tool use and tool making. This observation motivates Any-ttach, a tool-centric manipulation framework that treats quick end-effector swapping as a mechanism for dexterity with simplicity. Any-ttach combines a low-cost automatic swapping mechanism for an open-close robot interface, a handheld device for collecting human demonstrations, and a task planning framework that composes learned, parameterized, and planned tool-use skills. The system supports diverse tools and end-effector modules, including daily tools, articulated tools such as scissors, Fin Ray fingers, and a low-cost anthropomorphic hand, through the same shared interface. Our experiments show that Any-ttach improves tool-swapping reliability, increases demonstration efficiency, reduces tool-pose variability, and supports diverse tool-use skills. In two long-horizon tasks, making a sandwich and preparing a cucumber, Any-ttach executes six tool-use subskills through end-effector switching and execution monitoring. These results suggest that robots can expand manipulation capability not only through more complex end-effectors, but also through rapidly exchangeable tools and end-effector modules. More details and videos are available at https://any-ttach.github.io/.

Weizhe Ni, Jinzhou Li, Haoyu Li +3

Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.

Long-Horizon Robotic Manipulation Robotic Tool Use Robotic Manipulation

Ask PDF

Sep 28, 2026

From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations

Scaling up robotic manipulation is primarily bottlenecked by the scarcity of real-world robot data. While recent approaches leverage human video demonstrations to mitigate this shortage, they remain computationally expensive and still rely on paired human-robot data for domain alignment. Although current state-of-the-arts excel at long-horizon tasks, they struggle with the delicate and precise control required for complex tool manipulation. To overcome these limitations, we introduce P2P-T, from Pixel to Poses for Tool Manipulation, a data-efficient, object-centric framework that learns tool use directly from human demonstrations. P2P-T bridges the cognitive and physical execution gap through a two-stage approach. First, pretraining an object-centric world model to extract stable pose priors; second, integrating these priors into an efficient, pose-aware low-level policy. By utilizing a robust automated data processing pipeline powered by modern foundation models, P2P-T completely bypasses the need for human-robot aligned data. This reduces overall training overhead drastically. With minimal per-task fine-tuning, our framework achieves a 73% improvement over the previous state of the art in execution performance on complex, real-world tool manipulation tasks that currently remain out of reach for standard large-scale pretrained models.

Bangjun Wang, Longyan Wu, Yukun Wei +8

The University of Hong Kong · Shanghai Innovation Institute · Xiaomi EV

Object-Centric World Models Robotic Tool Use Data-Efficient Robot Learning Robot Imitation Learning

Ask PDF

Oct 1, 2026

HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution

As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable tool and coordinating manipulation and, when needed, locomotion to complete the task. Existing benchmarks do not jointly evaluate these capabilities on a humanoid. We introduce HumanoidToolBench, an 18-task benchmark spanning three scenarios, three execution levels, and two tool-set modes, together with ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1. Evaluation of seven policies in simulation and three on the real robot reveals substantial gaps between selecting a suitable tool and completing the task. Focused GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued task execution under unrelated instructions. Code and data are available at https://snu-pi.github.io/HumanoidToolBench/.

Kyochul Jang, Seohyeon Park, Ohchul Kwon +9

Seoul National University · University of Massachusetts Amherst · Google Research

Mobile Manipulation Robotic Tool Use Robot Manipulation Benchmarks Humanoid Robotics Embodied Agent Benchmarks

Ask PDF

Abstract

Robotic Tool Use

Explore similar work

Any-ttach: Quick End-effector Swapping Enables Manipulation Dexterity with Simplicity

From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations

HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution