Compliant In-hand Rolling Manipulation Using Tactile Sensing

Huan Weng; Yifei Chen; Kevin M. Lynch

cs.ROMar 4, 2026

Compliant In-hand Rolling Manipulation Using Tactile Sensing

Authors: Huan Weng, Yifei Chen, Kevin M. Lynch

Organizations: Center for Robotics and Biosystems and the Department of Mechanical Engineering, Northwestern University, Evanston, IL 60208 USA · Northwestern Institute on Complex Systems (NICO)

Abstract

We investigate in-hand rolling manipulation using a multifingered robot hand, where each finger is compliant and equipped with a tactile fingertip providing contact location and wrench information. We derive the equations of motion for compliant quasistatic in-hand rolling manipulation and formulate a fingertip rolling manipulation controller for multiple fingers to achieve a desired object twist within a grasp. The contact mechanics are demonstrated in simulation and the controller is tested on an experimental robot system.

Figures & tables

Fig. 1: Most in-hand manipulation requires rolling, sliding, and/or spinning at finger contact patches, as illustrated in this example of three-fingered in-hand twisting of the lid of a jar.

Fig. 2: Three major categories of dexterous manipulation in a common example of tool use. The lower chopstick is held in a fixed grasp, the upper chopstick undergoes in-hand manipulation, and the chopsticks push the steamed bun (nonprehensile manipulation) before grasping it.

Fig. 3: A classification of types of dexterous manipulation. This paper focuses on quasistatic in-hand rolling manipulation of rigid bodies.

Fig. 4: Simplified model of a finger. The “anchor” is driven by high-impedance, effectively position-controlled, finger joints, typical of the highly-geared actuators used in many robot hands. The fingertip is mounted to the anchor by a 6-dof flexure, providing passive compliance and the possibility of force control for safe manipulation of rigid objects.

Fig. 5: The Barrett WAM robot arm and Allegro hand equipped with Visiflex tactile sensors.

Fig. 6: A fingertip displaced from the flexure’s relaxed position and making contact with an object, with defined frames. The fingertip’s pose when the flexure is relaxed is drawn in grey. All the frames at the contact point have their $\hat{z}$ -axes normal to the tangent plane. $\phi$ is the rotation angle about the $\hat{z}$ -axis of frame {l 1 }/{c 1 } from the $\hat{x}$ -axis of {l 1 }/{c 1 } to that of {l 2 }/{c 2 }.

Fig. 7: A compliant fingertip moving on the object surface while applying the contact force

{\bf f}_{\text{con}}

. A local surface patch and a coordinate frame are defined with the origin at the contact point and two axes

u_{1,x}

and

u_{1,y}

orthogonal to each other. The contact point velocity can be represented in that frame as

\dot{\bf u}_{1}

Frame	Definition
{w}	World frame.
{o}	Object frame, attached to the object.
{a}	Fingertip anchor frame connected to the robot hand palm through finger links (phalanges) and joints.
{f}	Fingertip frame, attached to the hemispherical fingertip, connected with {a} through the flexure.
{f 0 }	Fingertip rest frame, attached to {a} but coincident with {f} when the flexure is at rest.
{l 1 },{l 2 }	Stationary contact frames at the current contact points, attached to {o} and {f}, respectively, each with its $\hat{z}$ -axis pointing outwards and $\hat{x}$ , $\hat{y}$ -axes tangential to the local coordinate system of the surface patch.

TABLE I: Frame definitions

Variable	Description
${\bf p}^{\text{}}_{\text{ij}}$	Position vector pointing from the origin of frame {i} to that of {j}, ${\bf p}^{\text{}}_{\text{ij}}\in{\mathbb{R}}^{3}$ .
${\bf R}_{\text{ij}}$	Rotation matrix from frame {i} to {j}, ${\bf R}_{ij}\in SO(3)$ .
${\bf T}_{\text{ij}}$	Transformation matrix from frame {i} to {j}, ${\bf T}_{\text{ij}}:=\begin{bmatrix}{\bf R}_{\text{ij}}&{\bf p}^{\text{i}}_{\text{ij}}\\ {\bf 0}&1\end{bmatrix}\in SE(3)$ .
$[\text{Ad}_{{\bf T}_{\text{ij}}}]$	Adjoint representation of ${\bf T}_{\text{ij}}$ , $[\text{Ad}_{{\bf T}_{\text{ij}}}]:=\begin{bmatrix}{\bf R}_{\text{ij}}&{\bf 0}\\ {[{\bf p}^{\text{i}}_{\text{ij}}]}{\bf R}_{\text{ij}}&{\bf R}_{\text{ij}}\end{bmatrix}$ .
${\mathcal{X}}^{\text{}}_{\text{ij}}$	Exponential coordinates representation of ${\bf T}_{\text{ij}}$ , ${\mathcal{X}}^{\text{}}_{\text{ij}}\in{\mathbb{R}}^{6},{\bf T}_{\text{ij}}=\text{exp}([{\mathcal{X}}^{\text{i}}_{\text{ij}}])$ .
${\mathcal{V}}^{\text{}}_{\text{ij}}$	Twist of frame {j} relative to {i}, ${\mathcal{V}}^{\text{}}_{\text{ij}}:=\begin{bmatrix}{\boldsymbol{\omega}}^{\text{}}_{\text{ij}}\\ {\bf v}^{\text{}}_{\text{ij}}\end{bmatrix}\in{\mathbb{R}}^{6}$ .

TABLE II: Notation from [ 47 , 81 ]

Fig. 8: Feedback control of in-hand rolling manipulation.

Fig. 9: (Top left) Two planar fingers in initial contact with a disk. (Top right) When each finger anchor advances forward, the disk rotates in place, the flexure-mounted fingertips comply symmetrically, and the fingertips roll on the object. (Bottom) A simulation of the manual experiment predicts similar results. Flexures are not drawn in the simulation snapshots.

Component	Inputs and outputs	Rate (Hz)
OptiTrack vision tracking	Output: object pose	360
Tactile sensors	Output: contact locations and wrenches	60
Allegro hand and controller	Input: commanded joint positions. Output: finger configurations and Jacobians	333
WAM arm and controller	Input: all data from other components. Output: finger joint position commands	500

TABLE III: Hardware of the manipulation system

Fig. 10: Left: A cylindrical object with a central axis. Right: An axis that allows only vertical and rotational motions of the cylindrical object.

Fig. 11: Snapshots of in-hand rolling manipulation to rotate the cylinder.

Fig. 12: (Left) Schematic of the Visiflex tactile sensor. (Middle) The embedded camera detects the contact location on the dome and the deflection of the eight LED fiducials, which allows calculation of the contact wrench due to the known flexure stiffness. (Right) A color-adjusted version of the camera’s view.

Fig. 13: The inset shows the evolution of the contact point on the robot’s index fingertip during one execution of in-hand rolling manipulation.

Fig. 14: Experimental results of

19

runs of in-hand rotation of the cylinder, plotted as the mean and standard deviation. Trajectory tracking errors are small.

Fig. 15: The normal contact forces of all fingertips in an example run.

Fig. 16: The tangential contact forces of all fingertips in an example run.

Explore similar work

Mar 7, 2026

TacDexGrasp: Compliant and Robust Dexterous Grasping with Tactile Feedback

Multi-fingered hands offer great potential for compliant and robust grasping of unknown objects, yet their high-dimensional force control presents a significant challenge. This work addresses two key problems: (1) distributing forces across multiple contacts to counteract an object's weight, and (2) preventing rotational slip caused by gravitational torque when a grasp is distant from the object's center of mass. We address these challenges via tactile feedback and a Second-Order Cone Programming (SOCP)-based controller, without explicit torque modeling or slip detection. Our key insights are (1) rotational slip inevitably induces translational slip at some contact points for a multi-fingered grasp, and (2) the ratio of tangential to normal force at each contact is an effective early stability indicator. By actively constraining this ratio for each finger below the estimated friction coefficient, our controller maintains grasp stability against both translational and rotational slip. Real-world experiments on 12 diverse objects demonstrate the robustness and compliance of our approach.

Yubin Ke, Jiayi Chen, Hang Lv +2

Peking University · Galbot · Beijing Academy of Artificial Intelligence

Robotic Grasping Force-Controlled Robotic Manipulation Dexterous Manipulation

Ask PDF

Mar 29, 2026

Robotic Dexterous Manipulation via Anisotropic Friction Modulation using Passive Rollers

Controlling friction at the fingertip is fundamental to dexterous manipulation, yet remains difficult to realize in robotic hands. We present the design and analysis of a robotic fingertip equipped with passive rollers that can be selectively braked or pivoted to modulate contact friction and constraint directions. When unbraked, the rollers permit unconstrained sliding of the contact point along the rolling direction; when braked, they resist motion like a conventional fingertip. The rollers are mounted on a pivoting mechanism, allowing reorientation of the constraint frame to accommodate different manipulation tasks. We develop a constraint-based model of the fingertip integrated into a parallel-jaw gripper and analyze its ability to support diverse manipulation strategies. Experiments show that the proposed design enables a wide range of dexterous actions that are conventionally challenging for robotic grippers, including sliding and pivoting within the grasp, robust adaptation to uncertain contacts, multi-object or multi-part manipulation, and interactions requiring asymmetric friction across fingers. These results demonstrate the versatility of passive roller fingertips as a low-complexity, mechanically efficient approach to friction modulation, advancing the development of more adaptable and robust robotic manipulation.

Ethan Fisk, Taeyoon Lee, Shenli Yuan

Department of Mechanical Engineering, Northeastern University, Boston, MA 02115, USA · RAI Institute, Cambridge, MA 02142, USA

Robotic Manipulation Dexterous Manipulation Robotic Hand Design Contact-Rich Robotic Manipulation In-Hand Manipulation

Ask PDF

Jul 3, 2026

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation

Compliance is essential for dexterous manipulation, yet existing solutions often rely on external tactile or force sensors that are costly, fragile, and difficult to deploy on low-cost robot hands. We propose a proprioception-driven framework that learns contact-aware compliance cues from motor current and joint states. Since motor current is closely related to actuator torque, it provides an intrinsic signal for perceiving contact force, object resistance, and grasp stability without additional sensing hardware. Rather than estimating external wrenches or commanding torque, our method predicts a compliance reference position: an ideal joint-position target for a standard PD controller whose induced position error generates appropriate grasping force. This position-based formulation is compatible with mainstream teleoperation and policy-learning pipelines, while enabling the robot to adapt interaction forces from real-time proprioceptive feedback. Thus, motor current serves not only as a force proxy but also as a learnable proprioceptive contact signal for compliance reference prediction. Experiments on multiple dexterous hands and contact-rich tasks, including fragile object handling, sustained surface contact, thin-object retrieval, and dynamic load adaptation, show stable compliant grasping, safer and more efficient teleoperation, and improved downstream policy learning without external tactile or force sensors.

Chenyang Ma, Yunchao Yao, Zhenyu Wei +3

University of North Carolina at Chapel Hill

Robotic Control Dexterous Manipulation Contact-Rich Robotic Manipulation

Ask PDF