Tactus: Tactile & Pose Sensing Glove for Natural Dexterity Data Collection
Authors: Li Guan, Rachael Granberry, Sahar Aseeri, Sahar Rostami, Taylor Niehues, Sai Dhanooshi Marri, Mia Huang, Dingtian Zhang, Ziyi Kou, Jason Pursel, Eric Whitmire, Vatsal Mehta, Ashish Deshpande, Tristan Trutna, Reed Belcher, Emelia Black, Julian Blake, Roger Boldu, Sean Bonaventure, Einar Boyovich, Jordan Brennan, Elizabeth Casillas, David Christensen, Tyler Christensen, Nicholas Colonnese, Stuart Dealey, Gaige DeHaven, Katie Doran, Chris Doughty, Robert Drury, Silvia Espino, Gabriel Geyne, Beth Goodrich, Kyra Gould, Joe Grisso, Susana Guerrero, Andrew Gwozdziowski, Monica Ha, Ehsan Hajiesmaili, Kate Healy, Rose Hope, Michael Hurst, Claire Ikemiya, Daisy Isibor, Rick James, Yashasvi Joginapally, Samuel Johnson, Tyler Jorgensen, Kristy Jost, Ben Kamath, Slava Karulin, Alexander Keener, Woo Kim, Alison Krauskopf, Ankit Kumar, Joshua Lang, Lizzy Li, Cameron Lynch, Daniel Ma, Hector Martinon, Christina Matos, Keagan McCurdy, Lee Mincey, Stuart Nezlek, Tiffany Nguyen, James O'Hara, Jason Osborne, Jacky Pai, Roham Pardakhtim, David Perek, Alexandre Poulin, Nick Raines, Salwa Rasheed, Leah Resneck, Kevin Rhodes, Ergys Ristani, Douglas Roberts, Anik Sarker, Harrison Scarborough, Jonas Schmidtler, Brian Shieh, Benjamin Shuman, Garrett Simard, Forrest Smith, Udara Somarathna, Eliot Sosin, Jonathan Stewart, Kristine Strom, Adam Swett, Hal Talbot-Mantooth, Julia Tanner, Sai Sharan Thirunagari, Curtis Torrey, Simge Uzun, Eileen Wachs, Rylan Wasson, Henry Wei, Jesse White, Beau Woodbyrne, Antonio Woods, Noah Young, Chandler Yount, Joseph Zhang, Sean Keller
Collecting naturalistic human manipulation demonstrations with accurate contact force and hand pose is essential for robot dexterous manipulation policy and foundational world model learning - yet no existing wearable platform provides both calibrated fingertip force sensing and occlusion-resilient hand tracking in a form factor that preserves natural manipulation behavior. We present Tactus, a textile-integrated sensing glove that co-optimizes three design pillars: (1) a thin, conformal 3D-knit form factor (1.5 mm fingertip stackup) designed to minimize interference with natural manipulation, (2) strategically placed capacitive tactile sensors at manipulation-relevant regions informed by empirical contact studies, and (3) IMU-fused pose tracking that maintains accuracy through the hand-object occlusions where manipulation data is most valuable. Tactus includes a documented calibration pipeline for cross-glove consistency and per-session calibration for subject-dependent correction. To enable objective comparison, we introduce standardized benchmarking protocols for tactile force regression and hand pose estimation, and apply them to Tactus and two commercially available tactile gloves - to our knowledge the first quantitative cross-platform evaluation of this kind. Tactus resolves a minimum detectable force of 0.59 N at 95% confidence, 1.6 - 2.7x lower than the commercial baselines, and with 0.87 cross-participant consistency, while our pose tracking sustains 11.0 mm fingertip keypoint relative error during manipulation, when occlusion is heaviest. We show that these combined signals support object property estimation, slip detection, and representative recording of hand-object interaction - yielding demonstration data with the fidelity required for contact-aware manipulation policies and world models.
Figures & tables
Fig. 1: Form-factor of Tactus digital sensing gloves capable of high-fidelity tactile and pose sensing for natural hand-object manipulation data collection.
Fig. 2: Glove fit sets whether sensor contact means hand contact. A) An oversized glove lifts the sensor off the skin, so it touches first. A conformal glove tensions sensors against the hand. B) Compound curvature demands sensor multi-axial conformation.
Fig. 3: Knitted tactile capacitive sensor mechanism. The conductor and insulation layers form a capacitor whose capacitivity changes when compressed by external force.
Fig. 4: Sensor layout was informed by three datasets. A) A contact frequency map identifies where sensors are worth placing. B) Inking the flexed hand leaves skin folds un-inked, ruling out those regions because sensors there register self-contact. C) 3D scans of gloved- & bare-hands rule out glove tenting regions, e.g. the palm center, where sensors contact the world first.
Fig. 5: Two ways to widen population coverage. A) Increase the elasticity of the sensor. B) Reduce the sensor coverage area, which helps when the sensor is less stretchable than the base glove.
Fig. 6: Factory calibration rig design. A closed pneumatic chamber in which the glove, donned on a prosthetic hand, is inflated against from above by a thermoformed elastomeric membrane with regulated air pressure.
Method
Median full-glove force RMSE (N)
Median Relative Error
Factory cal.
2.50–5.22
40.6%–98.1%
Online cal.
1.46–1.82
27.7%–40.9%
TABLE I: Factory vs. Per-Session Online Calibration Accuracy
TABLE II: Wearables Task Coverage Performance vs bare-hand baseline
Metric
PPS
JQ
Tactus
MDF@95% (N) ↓
1.58
0.94
0.59
Fingertip force error (N) ↓
0.23
0.18
0.20
X-subj. consistency ↑
0.77
0.86
0.87
PRA ratio ↓
2.13
0.43
0.17
TABLE III: Tactile benchmark result. Left: testing rig, and pressing locations at the fingertip. Right: tactile gloves’ force regression performance comparison.
Tracking Mode
MKPE (mm) ↓
MKPE.T (mm) ↓
F.MKPE (mm) ↓
F.MKPE.T (mm) ↓
CV-barehand [ 19 ]
15.8
8.9
19.8
17.1
CV+IMU EKF
15.4
7.9
18.7
14.2
AVI-HT [ 32 ]
10.4
7.0
13.3
12.8
AVI-HT offline
9.7
6.4
12.1
11.0
TABLE IV: Hand Pose Benchmarks Performance
Model
Overall
Mirror
Bushy
Scaly
Bumpy
Braided
Ridged
Fuzzy
Fluffy
Hairy
Tactile
0.75
0.78
0.83
0.66
0.81
0.67
0.61
0.67
0.97
0.84
VLM
0.56
0.84
0.00
0.96
0.97
0.08
1.00
0.86
0.45
0.00
TABLE V: Surface roughness 9-class accuracy table. The tactile network vs VLM zero-shot performance.
Fig. 7: We design a rig to test controlled slip events with measured force and displacement. We can automatically label different stages of slip from the ground truth sensory signals.
Fig. 8: Multi-modalities from Tactus (Tactile and IMU) enable qualitatively better hand-object interaction recording compared to CV-only HaMeR hand pose and RGBD CV-only FoundationPose object pose. t1 : FoundationPose has hard time to pin-point the cup handle orientation for such close-to-symmetric object. t2 & t3 : HaMeR hand and FoundationPose object loses track completely due to occlusion. t4 : although HaMeR recovers both hands, the hands scaling / depth are incorrectly estimated.
Meta, Redmond, Washington, 98052, USA · Department of Biomedical Engineering, Northwestern University, Evanston, Illinois, 60208, USA · Work done at Meta