cs.ROOct 16, 2026

No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?

Authors: Ayush Deshmukh, Nalin Agarwal, Elle Miller, Sethu Vijayakumar

Organizations: All authors are with the School of Informatics, University of Edinburgh, Edinburgh, UK.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Humans manipulate objects within their hands through touch alone with remarkable ease, but for robots, achieving sustained real-world manipulation without vision remains an open challenge. Although reinforcement learning (RL) can generate sophisticated blind manipulation policies in simulation, transferring them to a real dexterous hand remains hindered by continuously changing contacts, motion-induced tactile information drift, and the gap between simulated and physical dynamics. We study this problem through Baoding ball rotation on the Shadow Dexterous Hand Lite, using only proprioception and binary tactile feedback as observations. We achieve successful zero-shot transfer through a combination of a sensor-aligned, coupling-aware embodiment in simulation, tactile calibration, and two novel episodic domain randomisations: slew-rate action limits and STuck-AT Taxels. On the physical hand, the policy achieves sustained blind Baoding rotation, completing \textbf{225 consecutive \mathbf{180^\circ}rotations(112.5fullrotations)atrotations (112.5 full rotations) at\sim0.36 ;\text{rot/s}} in a single continuous trial without any vision or object-state information. Furthermore, we evaluate the contribution of tactile sensing relative to proprioception alone and identify the hand regions where tactile information is most important. This is the strongest reported hardware result for Baoding ball manipulation in both rotation count and speed, irrespective of the sensing modality used. By effectively bridging the sim-to-real gap, our framework sets a new benchmark for sustained tactile-driven dexterous manipulation on physical hardware.

Figures & tables

Fig. 2: FSR setup on the Shadow Dexterous Hand Lite. ( a ) The physical hand, with the FSR placements labelled. ( b ) The simulation model of the same hand, with the joints labelled; the FSRs are shown in red.
Fig. 3: Fixed tendon coupling on the Shadow Hand. The same 45∘45^{\circ} PIP flexion command applied to both simulation embodiments ( a ): the coupling mechanism used in this work, fitted with BioTac fingertips. ( b ): the coupling mechanism as supplied in Isaac Lab.
Fig. 4: Method. ( Top left ) Policy training in simulation. The agent’s actions are passed through the corrected coupling mechanism before the physics step, and the resulting state is returned as proprioceptive and binary tactile observations. ( Bottom left ) Zero-shot deployment on hardware at 60 Hz.. The trained policy is transferred without modification, with tactile calibration and actions rate-limited during execution. ( Right ) Randomization applied at each episode reset, including the STAT and SlewDR terms introduced in this work, together with the standard ball-mass and friction randomization.
ConfigurationTraining RandomizationFeedback / Input
C1Physical interactionProprioception + tactile
C2Physical + SlewDRProprioception + tactile
C3 (ours)Physical + SlewDR + STATProprioception + tactile
Proprio-onlyPhysical interactionProprioception only
C3-TacOffC3Proprioception only †
Open-loop–None (recorded trajectory)
TABLE I: Evaluated configurations
 Full rotationsTime to drop (s)η\etaSpeed (HW)Speed (sim)Drop rate
Conditionmean ±\pm stdmin–maxmean ±\pm stdmin–maxmean ±\pm stdmean ±\pm stdmin–maxmean ±\pm stdsimHW
Prior hardware results:
Nagabandi et al. [ 8 ] †∼0.50\sim 0.50—————————
Yuan et al. [ 9 ] ‡∼3.00\sim 3.00————∼\sim 0.30————
Yang et al. [ 10 ] §∼0.50\sim 0.50—————————
Benchmark & ablations:
TABLE II: Hardware results are mean ±\pm std over 10 trials; simulation results are aggregated over 768 episodes. Dashes (—) denote metrics not evaluated or reported.
Fig. 5: Joint trajectories for the ring-finger MCP flexion joint and the thumb IP joint over four rotation cycles. Top row: the proprioception-only policy with and without the balls in the hand. Bottom row: C3-TacOff against C3. Each plot overlays the two conditions for a single joint.
Fig. 6: Mean evaluation return during training for configurations C1, C2, and C3, each run across three seeds for 250 M environment steps. The 100 mark corresponds approximately to retaining both balls without rotating them, so returns above it indicate that rotations have begun. C1 crosses it at around 80 M steps against approximately 160–170 M for C2 and C3, and reaches roughly twice their final return.
Fig. 7: Commanded and achieved joint positions on hardware for two configurations. Left column: C1. Right column: C2. Top row: the positions commanded by the policy. Bottom row: the positions measured on the physical hand over the same interval. Each panel plots two joints: the ring-finger MCP flexion joint and the thumb IP joint.
Fig. 8: Normalised binary-tactile activation for C3 policy in simulation (left) and hardware (right) . In simulation, activation is concentrated primarily around the middle and knuckle regions of the middle and ring fingers. On hardware, the dominant activation shifts proximally, with the proximal middle and ring-finger sensors accounting for a substantial amount of tactile activation.
More in this topic

In-Hand Manipulation

In-hand manipulation studies how robot hands reposition and reorient objects without releasing them.

See all 34 papers

Explore similar work

CardsList
Aug 30, 2026

Blind Dexterity: Whole-Body Humanoid Manipulation via Pure Proprioception

We present blind, whole-body manipulation skills on a Unitree G1 humanoid using only onboard proprioception, without cameras, markers, force-torque, or tactile sensors. Despite this minimal sensing, the trained policies exhibit surprising capability across qualitatively different tasks: push-resilient bipedal walking without IMU feedback, active soccer ball trapping with a foot, seeking and lifting a suitcase by its handle, and mounting a randomly positioned skateboard. We argue that these capabilities arise from a key underappreciated signal: the way the joint encoder readouts evolve under purposeful compliant contact, effectively forming a whole-body tactile channel. By generating contact-rich motions, the trained policies actively probe the environment; as a result, task-relevant object state (e.g., pose) becomes increasingly decodable from short proprioceptive histories. We expose this information using compact task-specific state estimators trained alongside, but fully separately from, the policies; their prediction errors decrease rapidly after informative contact. Our results indicate that joint encoder-based proprioception, combined with compliant actuation (now widely available on commercial robots and low-cost motors) is already a strong, practical substrate for whole-body dexterous manipulation and interactive perception, and therefore a natural foundation on which richer sensing can be layered.

Aditya Bhatt, Oleg Kaidanov, Puze Liu +1

Intelligent Autonomous Systems Lab, TU Darmstadt · German Research Center for AI (DFKI) · Tongji University, Shanghai Research Institute for Autonomous Intelligent Systems +1

Whole-Body Robotic ManipulationRobotic ManipulationHumanoid RoboticsContact-Rich Robotic ManipulationRobotic Perception
AskPDF
Jun 23, 2026

NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no-contact-no-worries/

Soham Patil, Avirup Das, Sourabh Bhosale +1

Robotics Research Center (RRC), International Institute of Information Technology (IIIT), Hyderabad, India · Department of Computer Science, The University of Manchester

Vision-Based Tactile SensingDexterous ManipulationIn-Hand ManipulationRobotic Tactile SensingMultimodal Sensor Fusion
AskPDF
Sep 25, 2026

See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands

In this work we study if a robotic hand using proprioception alone can grasp diverse objects with no visual observation. We present a modular dexterous grasping architecture that separates global arm motion from local contact control. An independently controlled arm guides the hand toward the object, while a reinforcement learning policy grasps and stabilizes it using only hand proprioceptive feedback. We call this a blind grasp reflex: grasping without images, object poses, or geometric observations. A learned stable-grasp score determines when the object is securely held, allowing the arm to begin post-grasp manipulation. This separation makes grasping a reusable hand-level skill that can be combined with independently designed arm controllers for various manipulation tasks. Experiments in simulation and on hardware demonstrate robust blind grasping across diverse objects and seamless composition with a range of arm controllers. Moreover, despite never observing contact geometry, the learned grasp score closely aligns with an independent physics-based measure of grasp stability. The resulting approach follows a simple principle: see to reach, feel to grasp. Project page: https://blindgraspreflex.github.io.

Alexander Alexiev, Tzu-Yuan Lin, Sang Min Kim +3

Department of Mechanical Engineering, Massachusetts Institute of Technology

Robotic GraspingRobotic ManipulationRL for Robotics
AskPDF