Organizations: Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA 92093, USA. · Sudo AI GmbH, 8050 Zurich, Switzerland.
Supernumerary robotic fingers augment human dexterity by enabling a single hand to perform complex "hold-and-manipulate" tasks that typically require bimanual coordination. However, existing devices are often hindered by bulky hardware that limits interaction to large objects and rely on explicit hand motion triggers that lack environmental context. To address these challenges, we present EgoThumb, a fully open-source, vision-driven supernumerary robotic thumb. EgoThumb features a compact, dual tendon-driven finger whose actuation is relocated to the forearm, reducing distal weight while enabling precise active joint control. The system utilizes a visual imitation learning policy that maps egocentric images from a palm-facing camera directly to continuous joint positions. By implicitly decoding human intent from visual cues capturing both the hand and the object of interest, EgoThumb achieves robust autonomous manipulation and strong generalization across diverse users and environments. All hardware designs and software implementations are fully open-sourced at https://ego-thumb.github.io.
Figures & tables
Fig. 2: Overview of the EgoThumb System Design. 1) On-body View : The system comprises a wearable thumb and a forearm-mounted actuation module. 2) Wearable Thumb : The thumb features dual tendon-driven joints for precise control and PLA phalanges with a silicone-gel coating for stable grasping. A palm-facing fisheye camera provides egocentric visual feedback. 3) Actuation Module : The actuation module houses three Dynamixel XL430-driven ratchet spools. This forearm-mounted configuration significantly reduces distal weight. 4) Control System : Our EgoThumb interfaces with teleoperation and deployment systems via custom communication middleware.
Fig. 3: Execution Sequences for the Four Evaluation Tasks. Policy rollouts demonstrating collaborative human-robot manipulation across four tasks: Pick Up Cube , Unscrew Cap , Stir Tea , Transfer Ball . For each task, the top row displays the user’s first-person view of the interaction, while the bottom row shows the corresponding egocentric palm-facing camera view used by the policy. The timelines below outline the coordinated sequence of autonomous EgoThumb actions and human motions.
Pick Up Cube
Unscrew Cap
Stir Tea
Transfer Ball
Avg.
Demonstrator
1.0
0.7
0.9
1.0
0.90
User 1
1.0
0.9
0.8
1.0
0.925
User 2
1.0
0.8
0.8
0.9
0.875
TABLE I: Success Rates across Different Hands. We report the performance when objects are placed within initial and target areas in the training data (Fig. 4 , Top, darker regions). EgoThumb system achieves strong performance across all four tasks when operated by the demonstrator, and it generalizes effectively to novel participants (User 1 and User 2).
Pick Up Cube
Unscrew Cap
Stir Tea
Transfer Ball
Demonstrator
1.0
0.8
0.6
0.2
User 1
1.0
0.7
0.3
0.1
User 2
1.0
0.7
0.7
0.2
Avg.
1.00
0.73
0.53
0.17
TABLE II: Success Rates in Expanded Initial and Target Areas. We evaluate our system on an expanded initial and target areas (Fig. 4 , Top, lighter regions) compared to the training data. Results show strong robustness on Pick Up Cube and Unscrew Cap , moderate robustness on Stir Tea , and a clear drop on Transfer Ball , likely due to increased foreground visual variation and its longer-horizon manipulation requirements.
Fig. 4: Task Area and “In-the-Wild” Environments. Top (Task Area): Darker areas represent the initial and target areas for training data, while the larger, lighter areas (including the original darker areas) represent the expanded areas used for the robustness test of object distributions. Pick Up Cube and Unscrew Cap utilize the green areas, expanded from 10 cm × 10 cm to 50 cm × 10 cm. Transfer Ball and Stir Tea use red for pick-up and blue for drop-off, expanded from 10 cm × 10 cm to 30 cm × 25 cm. For the Transfer Ball task, expanding the evaluation area shifts the target receptacle’s position (“Target Change”). Bottom (“In-the-wild” Environments): The “Seen Environments” panel displays the diverse environments included in the broadened training dataset. The “Unseen Environments” panel displays completely novel backgrounds used strictly to test out-of-distribution generalization. Labels #1 and #2 denote the specific seen and unseen test environments.
Envs
Seen #1
Seen #2
Unseen #1
Unseen #2
Demonstrator
0.8
1.0
0.7
0.9
User 1
0.9
1.0
0.7
0.6
User 2
0.7
1.0
0.6
0.7
Avg. User
0.80
1.00
0.67
0.73
TABLE III: Task Success Rates Across Different Backgrounds. We evaluate our policy on two seen and two unseen environments using the Stir Tea Task. The policy maintains robust performance across both seen and entirely novel (unseen) backgrounds for all users.
Fig. 5: Impact of Training Data Scale on Generalization. Scaling the demonstrations from 25 to 105 generally improves policy performance in both seen (solid lines) and unseen (dashed lines) environments across all users, proving that more data drives robust out-of-distribution generalization.
System Configuration
Success Rate
Full System
0.7
w/o Latency Matching
0.0
w/o Image Cropping
0.0
TABLE IV: Ablation Study Results. Disabling either latency matching or image cropping results in catastrophic policy failure, confirming their necessity for robust visuo-motor control.
Human hand grasp adaptation depends mainly on the synergy between physical structure and biological feedback. Inspired by this biomechanical principle, the Safe Thumb-Index Robotic (STIR) Hand was developed as a minimal, lightweight, and low-cost two-digit prototype featuring an asymmetric thumb-index configuration. By pairing an underactuated, tendon-driven mechanical design with flexible strain gauges embedded into silicone-encapsulated soft joints, the system achieves passive grasp adaptation while establishing both internal proprioception and external perception. Unsupervised analysis was carried out on a dataset of the STIR hand grasping 20 different objects, along with an object classification task and an ablation study to highlight the contribution of the soft joint sensors. The object classification task discriminated object size, shape, and material stiffness with a high classification accuracy. In contrast to traditional industrial grippers and robotic hands, the STIR Hand demonstrates that sensorized compliant joints significantly improve overall sensitivity and ensure safe grasping, while remaining independent of additional fingertip tactile elements or external vision systems. Finally, a comparison to similar devices grasping identical objects validates the utility of the STIR Hand.
Jonas Papenbrock, Shubhan Patni, Tomaso Lisini Baldi +4
Department of Information Engineering and Mathematics - University of Siena. · Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague
In this work we study if a robotic hand using proprioception alone can grasp diverse objects with no visual observation. We present a modular dexterous grasping architecture that separates global arm motion from local contact control. An independently controlled arm guides the hand toward the object, while a reinforcement learning policy grasps and stabilizes it using only hand proprioceptive feedback. We call this a blind grasp reflex: grasping without images, object poses, or geometric observations. A learned stable-grasp score determines when the object is securely held, allowing the arm to begin post-grasp manipulation. This separation makes grasping a reusable hand-level skill that can be combined with independently designed arm controllers for various manipulation tasks. Experiments in simulation and on hardware demonstrate robust blind grasping across diverse objects and seamless composition with a range of arm controllers. Moreover, despite never observing contact geometry, the learned grasp score closely aligns with an independent physics-based measure of grasp stability. The resulting approach follows a simple principle: see to reach, feel to grasp. Project page: https://blindgraspreflex.github.io.
Alexander Alexiev, Tzu-Yuan Lin, Sang Min Kim +3
Department of Mechanical Engineering, Massachusetts Institute of Technology
Current dexterous manipulation systems commonly rely on external or wrist-mounted cameras for visual feedback. However, such sensing setups are prone to occlusion and can limit performance on tasks requiring close-range, multi-view perception. In this work, we design a vision-enhanced fingertip module that brings clear visual perception directly to the fingertips and present FingerViP, a learning system that leverages fingertip visual perception for dexterous manipulation. Specifically, the fingertip module integrates miniature-camera sensing within compact-footprint hardware and is installed on each finger of a multi-fingered hand, which substantially improves visual perception by providing comprehensive, multi-view observations of objects and surrounding environment. Building on the integrated fingertip modules, we develop a diffusion-based whole-body visuomotor policy conditioned on a third-view camera and multi-view fingertip vision, enabling the robot to effectively learn complex manipulation skills directly from human demonstrations. To improve view-proprioception alignment and contact awareness in the policy observation representation, each fingertip visual feature is augmented with its corresponding camera-pose encoding and per-finger joint-current encoding. We validate the effectiveness of multi-view fingertip vision and demonstrate the robustness and adaptability of FingerViP across a range of challenging real-world tasks, including retrieving objects behind an occluding curtain, achieving a 56.6% relative improvement in average success rate over the baseline using an external third-view camera and a wrist-mounted camera. Videos, code, and hardware designs are available on our project page: https://fingervip.github.io/.
Zhen Zhang, Weinan Wang, Hejia Sun +4
The Chinese University of Hong Kong · The Hong Kong Polytechnic University