Compliant In-hand Rolling Manipulation Using Tactile Sensing

Huan Weng; Yifei Chen; Kevin M. Lynch

cs.ROMar 4, 2026

Compliant In-hand Rolling Manipulation Using Tactile Sensing

Authors: Huan Weng, Yifei Chen, Kevin M. Lynch

Organizations: Center for Robotics and Biosystems and the Department of Mechanical Engineering, Northwestern University, Evanston, IL 60208 USA · Northwestern Institute on Complex Systems (NICO)

Abstract

We investigate in-hand rolling manipulation using a multifingered robot hand, where each finger is compliant and equipped with a tactile fingertip providing contact location and wrench information. We derive the equations of motion for compliant quasistatic in-hand rolling manipulation and formulate a fingertip rolling manipulation controller for multiple fingers to achieve a desired object twist within a grasp. The contact mechanics are demonstrated in simulation and the controller is tested on an experimental robot system.

Figures & tables

Fig. 1: Most in-hand manipulation requires rolling, sliding, and/or spinning at finger contact patches, as illustrated in this example of three-fingered in-hand twisting of the lid of a jar.

Fig. 2: Three major categories of dexterous manipulation in a common example of tool use. The lower chopstick is held in a fixed grasp, the upper chopstick undergoes in-hand manipulation, and the chopsticks push the steamed bun (nonprehensile manipulation) before grasping it.

Fig. 3: A classification of types of dexterous manipulation. This paper focuses on quasistatic in-hand rolling manipulation of rigid bodies.

Fig. 4: Simplified model of a finger. The “anchor” is driven by high-impedance, effectively position-controlled, finger joints, typical of the highly-geared actuators used in many robot hands. The fingertip is mounted to the anchor by a 6-dof flexure, providing passive compliance and the possibility of force control for safe manipulation of rigid objects.

Fig. 5: The Barrett WAM robot arm and Allegro hand equipped with Visiflex tactile sensors.

Fig. 6: A fingertip displaced from the flexure’s relaxed position and making contact with an object, with defined frames. The fingertip’s pose when the flexure is relaxed is drawn in grey. All the frames at the contact point have their $\hat{z}$ -axes normal to the tangent plane. $\phi$ is the rotation angle about the $\hat{z}$ -axis of frame {l 1 }/{c 1 } from the $\hat{x}$ -axis of {l 1 }/{c 1 } to that of {l 2 }/{c 2 }.

Fig. 7: A compliant fingertip moving on the object surface while applying the contact force

{\bf f}_{\text{con}}

. A local surface patch and a coordinate frame are defined with the origin at the contact point and two axes

u_{1,x}

and

u_{1,y}

orthogonal to each other. The contact point velocity can be represented in that frame as

\dot{\bf u}_{1}

Frame	Definition
{w}	World frame.
{o}	Object frame, attached to the object.
{a}	Fingertip anchor frame connected to the robot hand palm through finger links (phalanges) and joints.
{f}	Fingertip frame, attached to the hemispherical fingertip, connected with {a} through the flexure.
{f 0 }	Fingertip rest frame, attached to {a} but coincident with {f} when the flexure is at rest.
{l 1 },{l 2 }	Stationary contact frames at the current contact points, attached to {o} and {f}, respectively, each with its $\hat{z}$ -axis pointing outwards and $\hat{x}$ , $\hat{y}$ -axes tangential to the local coordinate system of the surface patch.

TABLE I: Frame definitions

Variable	Description
${\bf p}^{\text{}}_{\text{ij}}$	Position vector pointing from the origin of frame {i} to that of {j}, ${\bf p}^{\text{}}_{\text{ij}}\in{\mathbb{R}}^{3}$ .
${\bf R}_{\text{ij}}$	Rotation matrix from frame {i} to {j}, ${\bf R}_{ij}\in SO(3)$ .
${\bf T}_{\text{ij}}$	Transformation matrix from frame {i} to {j}, ${\bf T}_{\text{ij}}:=\begin{bmatrix}{\bf R}_{\text{ij}}&{\bf p}^{\text{i}}_{\text{ij}}\\ {\bf 0}&1\end{bmatrix}\in SE(3)$ .
$[\text{Ad}_{{\bf T}_{\text{ij}}}]$	Adjoint representation of ${\bf T}_{\text{ij}}$ , $[\text{Ad}_{{\bf T}_{\text{ij}}}]:=\begin{bmatrix}{\bf R}_{\text{ij}}&{\bf 0}\\ {[{\bf p}^{\text{i}}_{\text{ij}}]}{\bf R}_{\text{ij}}&{\bf R}_{\text{ij}}\end{bmatrix}$ .
${\mathcal{X}}^{\text{}}_{\text{ij}}$	Exponential coordinates representation of ${\bf T}_{\text{ij}}$ , ${\mathcal{X}}^{\text{}}_{\text{ij}}\in{\mathbb{R}}^{6},{\bf T}_{\text{ij}}=\text{exp}([{\mathcal{X}}^{\text{i}}_{\text{ij}}])$ .
${\mathcal{V}}^{\text{}}_{\text{ij}}$	Twist of frame {j} relative to {i}, ${\mathcal{V}}^{\text{}}_{\text{ij}}:=\begin{bmatrix}{\boldsymbol{\omega}}^{\text{}}_{\text{ij}}\\ {\bf v}^{\text{}}_{\text{ij}}\end{bmatrix}\in{\mathbb{R}}^{6}$ .

TABLE II: Notation from [ 47 , 81 ]

Fig. 8: Feedback control of in-hand rolling manipulation.

Fig. 9: (Top left) Two planar fingers in initial contact with a disk. (Top right) When each finger anchor advances forward, the disk rotates in place, the flexure-mounted fingertips comply symmetrically, and the fingertips roll on the object. (Bottom) A simulation of the manual experiment predicts similar results. Flexures are not drawn in the simulation snapshots.

Component	Inputs and outputs	Rate (Hz)
OptiTrack vision tracking	Output: object pose	360
Tactile sensors	Output: contact locations and wrenches	60
Allegro hand and controller	Input: commanded joint positions. Output: finger configurations and Jacobians	333
WAM arm and controller	Input: all data from other components. Output: finger joint position commands	500

TABLE III: Hardware of the manipulation system

Fig. 10: Left: A cylindrical object with a central axis. Right: An axis that allows only vertical and rotational motions of the cylindrical object.

Fig. 11: Snapshots of in-hand rolling manipulation to rotate the cylinder.

Fig. 12: (Left) Schematic of the Visiflex tactile sensor. (Middle) The embedded camera detects the contact location on the dome and the deflection of the eight LED fiducials, which allows calculation of the contact wrench due to the known flexure stiffness. (Right) A color-adjusted version of the camera’s view.

Fig. 13: The inset shows the evolution of the contact point on the robot’s index fingertip during one execution of in-hand rolling manipulation.

Fig. 14: Experimental results of

19

runs of in-hand rotation of the cylinder, plotted as the mean and standard deviation. Trajectory tracking errors are small.

Fig. 15: The normal contact forces of all fingertips in an example run.

Fig. 16: The tangential contact forces of all fingertips in an example run.

Explore similar work

TacDexGrasp: Compliant and Robust Dexterous Grasping with Tactile Feedback
Mar 7, 2026Yubin Ke, Jiayi Chen, Hang Lv +2Robotic Grasping Force-Controlled Robotic Manipulation
Peking University · Galbot · Beijing Academy of Artificial Intelligence
Robotic Dexterous Manipulation via Anisotropic Friction Modulation using Passive Rollers
Mar 29, 2026Ethan Fisk, Taeyoon Lee, Shenli YuanRobotic Manipulation Dexterous Manipulation
Department of Mechanical Engineering, Northeastern University, Boston, MA 02115, USA · RAI Institute, Cambridge, MA 02142, USA
Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation
Jul 3, 2026Chenyang Ma, Yunchao Yao, Zhenyu Wei +3Robotic Control Dexterous Manipulation
University of North Carolina at Chapel Hill

Mar 7, 2026J/K move · Enter open · S save

TacDexGrasp: Compliant and Robust Dexterous Grasping with Tactile Feedback

Yubin Ke, Jiayi Chen, Hang Lv, Xiao Zhou+1

Peking University · Galbot · Beijing Academy of Artificial Intelligence

Robotic Grasping Force-Controlled Robotic Manipulation Dexterous Manipulation

Open paper AskPDF arXiv

Multi-fingered hands offer great potential for compliant and robust grasping of unknown objects, yet their high-dimensional force control presents a significant challenge. This work addresses two key problems: (1) distributing forces across multiple contacts to counteract an object's weight, and (2) preventing rotational slip caused by gravitational torque when a grasp is distant from the object's center of mass. We address these challenges via tactile feedback and a Second-Order Cone Programming (SOCP)-based controller, without explicit torque modeling or slip detection. Our key insights are (1) rotational slip inevitably induces translational slip at some contact points for a multi-fingered grasp, and (2) the ratio of tangential to normal force at each contact is an effective early stability indicator. By actively constraining this ratio for each finger below the estimated friction coefficient, our controller maintains grasp stability against both translational and rotational slip. Real-world experiments on 12 diverse objects demonstrate the robustness and compliance of our approach.

Abstract

Explore similar work

TacDexGrasp: Compliant and Robust Dexterous Grasping with Tactile Feedback

Robotic Dexterous Manipulation via Anisotropic Friction Modulation using Passive Rollers

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation