DenseHap: Dense Hand Haptic Estimation from Monocular Images

Junhan Xue; Rolandos Alexandros Potamias

cs.CVOct 19, 2026

DenseHap: Dense Hand Haptic Estimation from Monocular Images

Authors: Junhan Xue, Rolandos Alexandros Potamias

Organizations: Imperial College London

Abstract

As dexterous robotic manipulation advances, scalable annotations of human hand-object interactions are becoming increasingly important. Furthermore, previous hand contact estimation methods suffered from issues such as dorsal-side misclassifications and insufficient precision, while lacking a unified framework for integrating multiple hand to environment interaction modalities, including distance fields, contact thresholds, and pressure. To address that, we present DenseHap, a unified end-to-end approach to estimate hand distance field recognition, comprehensive hand contact detection, hand pressure analysis, and hand reconstruction from a single monocular image. By incorporating three distinctly defined heads, along with a pretrained 3D reconstructed skeleton and an enhanced dataset, DenseHap outperforms previous state of the art models on four evaluation datasets, improving the F1 score by 8.5 percentage points (a 16% relative improvement) on untrained datasets and by 3 percentage points (5% relative improvement) on in-distribution datasets. Furthermore, DenseHap has effectively improved short-range hand estimate and pressure evaluation. Code and models will be publicly available upon the paper's acceptance.

Figures & tables

Figure 2: Overview of DenseHap. A frozen WiLoR [ 37 ] backbone yields the MANO [ 39 ] mesh and a ViT feature grid; three parameter-disjoint vertex-query cross-attention trunks feed contact, distance, and pressure heads under triple supervision. Detached proximity signals from the distance and pressure heads are injected into the contact head through zero-initialized learnable gates.

A. Sensor-measured (high reliability)
Dataset	Size	Share	Type
PressureVisionDB [ 14 ]	33,514	2.88%	Pressure
B. Multi-view, MoCap (high reliability)
TACO [ 29 ]	153,097	13.17%	Manipulation
OakInk2 [ 61 ]	147,553	12.69%	Manipulation
ARCTIC [ 11 ]	147,238	12.66%	Manipulation

Table 1: Dataset audit details 18 retained datasets, including their share of the training pool and primary interaction types. It also lists excluded datasets with reasons for their omission. ContactPose and EPIC-Contact (bottom block) are held out for evaluation.

		DenseHap				HACO
Dataset	$N$	P	R	F1	F1 ⋆	P	R	F1	F1 ⋆
EPIC-Contact	2272	.453	.622	.511	.525	.386	.595	.428	.459
ContactPose	2192	.606	.812	.673	.675	.565	.680	.588	.599
MOW	92	.432	.926	.555	.570	.525	.607	.522	.567
ARCTIC	3000	.675	.792	.703	.713	.516	.787	.682	.684
Mean	–	.542	.788	.600	.621	.498	.667	.555	.578

Table 2: Dense contact estimation under the Skip protocol. We report per-vertex precision (P), recall (R), and F1 score; F1 denotes performance at the default threshold ( 0.52), while F1 ⋆ denotes performance after readout and operating-point selection.

Method	MOW	ContactPose	EPIC	ARCTIC	RICH	Hi4D
POSA-hand [ 19 ] $\dagger$	.408	–	–	–	.398	.315
ContactPrompt [ 26 ] $\dagger$	.526	–	–	–	–	–
DECO [ 48 ]	.395 †	.513	.492	.365	.454	.304 †
BSTRO [ 21 ]	.380 †	.155	.382	.290	.645	.420 †
HACO [ 25 ]	.522	.599	.459	.684	.781	.474
JointHand (ours)	.571 d	.675 d	.524 c	.713 d	.799 c	.424 c

Table 3: Dense hand contact estimation, per-vertex F1 at each method’s own best threshold (Skip protocol). Our result uses the distance head (d) per dataset; EPIC, RICH, and Hi4D use only the contact head (c) because they lack distance ground truth.

\dagger

denotes HACO-reported results. Other results were evaluated locally under the same protocol.

Figure 3: Qualitative comparison between the proposed and the baseline method across five datasets.

	H $\rightarrow$ O Distance MAE (mm) $\downarrow$
Method	All	$<10$	$10$ – $20$	$\geq 20$
InterField-SF [ 11 ]	9.47	7.92	7.62	10.51
Our distance head	11.49	3.70	4.12	16.21

Table 4: Continuous hand-to-object distance on ARCTIC. MAE is computed over 778 MANO vertices with distances capped at 100 mm.

Method	VolIoU $\uparrow$	MAE (Pa) $\downarrow$	C-IoU $\uparrow$
Image space, $480\times 384$ px
PressureVision	0.413	4246	0.558
Mesh space, MANO-778
Failed blank control, $\ell_{1}$ loss †	0.000	6306	0.000
Ours (full model), BCE + proxy	0.491	2840	0.597

Table 5: Hand pressure prediction on PressureVisionDB , using the same six test identities. Cross block results are reference due to different output spaces. Metrics use the support

\{\hat{p}>1\,\text{kPa}\}\cup\{p>1\,\text{kPa}\}

. † Failed blank control producing all-zero pressure. Caveat: all checkpoints saw these identities during training.

A. Each vertex sees only its own image patch
Contact head	MOW	HIC	RICH	Hi4D	Mean
MLP, 1 hidden layer	.462	.109	.527	.516	.404
MLP, 2 hidden layers	.464	.107	.527	.515	.403
MLP, 5 hidden layers	.502	.168	.551	.519	.435
Graph network on the mesh	.477	.153	.536	.517	.421
B. All vertices share one summary of the image

Table 6: Contact-head architectures , grouped by what each vertex’s prediction is conditioned on. MLP hidden layer widths are

64

256

and

128

; “more training data” is

14

datasets instead of

13

; “rebalanced sampling” draws each dataset in proportion to

\sqrt{N}

; “queries learned from scratch” replaces the MANO-seeded queries with DINOv3. Single threshold

t{=}0.52

for all datasets, per-sample F1, non-contacting samples scored as

0

; bold marks the best in each group.

(a) Cross-attention, Skip protocol, best $t$
Contact head	Params	MOW	HIC	RICH	Hi4D
Full size	25 M	.547	.348	.799	.699
Small	1 M	.550	.266	.778	.698
Tiny	60 k	.533	.265	.781	.701
spread	$400\times$	.017	.083	.021	.003
(b) Per-vertex MLP, non-contact scored $0$ , $t{=}0.52$

Table 7: Head capacity, architecture held fixed. Cross-attention widths are

1024/256/64

with

6/2/1

layers; MLP hidden widths are

64

256

and

128

. spread is the gap between the best and worst row. Protocol is constant within a panel but differs between them, so the two panels are not comparable cell by cell.

Variant	ContactPose	ARCTIC
HACO [ 25 ]	.588	.682
2D proj. + distance supervision	.653	.687
+ distance-conditioned contact	.664	.688
+ BCS weighting	.659	.686
Full: 3 trunks + proximity injection	.673	.713

Table 8: Ablation study, per-vertex F1 via distance binarization on ContactPose (zero-shot) and ARCTIC. Each row adds one component to the previous. All variants surpass HACO; the full model performs best on both datasets.

Method	Params	FPS	Latency (ms)
ViT-H backbone only	${\approx}600$ M	43.5	$22.99\pm 0.28$
WiLoR (recon. only)	${\approx}640$ M	31.8	$31.49\pm 0.35$
HACO (contact only)	${\approx}630$ M	30.0	$33.32\pm 1.11$
WiLoR $+$ HACO cascade	${\approx}1.2$ B	15.4	$64.81\pm 3.25$
DenseHap (ours)	${\approx}672$ M	26.1	$38.37\pm 4.93$

Table 9: Efficiency on a single GH200 (bf16, batch size 1, mean over 200 runs). Sharing one backbone pass makes DenseHap

{\approx}1.7\times

faster than the WiLoR

+

HACO cascade; its

{\approx}40

M trainable head is

{\sim}15\times

smaller than the backbone and adds only

6.9

ms over reconstruction-only WiLoR.

Hand-Object Interaction Detection

See all 11 papers

Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models
May 7, 2026Daniel Sungho Jung, Kyoung Mu Lee
IPAI · Dept. of ECE & ASRI, Seoul National University, Korea
HOPE: Hand-Object Pressure Estimation from Monocular Videos
Aug 6, 2026Subin Jeon, Byungjun Kim, Hanbyul JooTactile Sensing Hand-Object Interaction
Seoul National University · RLWRLD
ScaleHP: Estimating Hand Pose in Metric Space
Jun 24, 2026Ruitao Jing, Xingyu Chen, Hongyang Li +33D Hand Pose Estimation Human Pose Estimation
Tsinghua University · Visincept · International Digital Economy Academy (IDEA Research) +2

May 7, 2026J/K move · Enter open · S save

Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models

Daniel Sungho Jung, Kyoung Mu Lee

IPAI · Dept. of ECE & ASRI, Seoul National University, Korea

Open paper AskPDF arXiv

Dense hand contact estimation requires both high-level semantic understanding and fine-grained geometric reasoning of human interaction to accurately localize contact regions. Recently, multi-modal large language models (MLLMs) have demonstrated strong capabilities in understanding visual semantics, enabled by vision-language priors learned from large-scale data. However, leveraging MLLMs for dense hand contact estimation remains underexplored. There are two major challenges in applying MLLMs to dense hand contact estimation. First, encoding explicit 3D hand geometry is difficult, as MLLMs primarily operate on vision and language modalities. Second, capturing fine-grained vertex-level contact remains challenging, as MLLMs tend to focus on high-level semantics rather than detailed geometric reasoning. To address these challenges, we propose ContactPrompt, a training-free and zero-shot approach for dense hand contact estimation using MLLMs. To effectively encode 3D hand geometry, we introduce a detailed hand-part segmentation and a part-wise vertex-grid representation that provide structured, localized geometric information. To enable accurate and efficient dense contact prediction, we develop a multi-stage contact reasoning with part conditioning, progressively bridging global semantics and fine-grained geometry. Therefore, our method effectively leverages the reasoning capabilities of MLLMs to enable precise dense hand contact estimation. Surprisingly, the proposed approach achieves performance comparable to the previous state-of-the-art method trained on large-scale dense contact datasets, without requiring any task-specific training.

Abstract

Hand-Object Interaction Detection

Explore similar work

Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models

HOPE: Hand-Object Pressure Estimation from Monocular Videos

ScaleHP: Estimating Hand Pose in Metric Space