We propose, HaNDF, object-conditional neural distance fields as data-driven priors for modeling the plausible hand-object configurations, in the product manifold of hand articulations. While recent unconditional generative priors showed success in modeling human pose, hands are versatile articulated bodies often found in interaction with other objects. This makes it crucial to develop a generative prior which can model the plausible hand poses conditioned on the object under interaction. To this end, our HaNDF proposes a geometry-aware, object-conditioned neural distance field, representing the plausible articulations in the zero-level set of a conditional field. We leverage tools from Riemannian geometry to (i) represent plausible hands in the zero level-set of an object-induced neural field, and (ii) project a given pose onto this field. Using HaNDF, we can also optimize for a plausible condition, determining the object under interaction. Our extensive evaluations demonstrate that our HaNDF provides a flexible and powerful prior for hand-object interaction, suitable for applications such as pose refinement, reconstruction under occlusion, and physically plausible manipulation synthesis.
Figures & tables
Figure 1: We introduce HaNDF , an object-conditioned generative hand prior. HaNDF is a geometric neural distance field representing the plausible hand configurations under interaction, and enables numerous tasks from denoising and completion to recovering hands or even objects from image data.
Figure 2: Manifold of quaternions.
Figure 3: HaNDF jointly models the plausible configurations of joints given the object code.
Figure 4: Pose generation on HOI4D: FID vs. dNN (lower is better); bubble size encodes APD.
Figure 5: Hand-pose denoising on HOI4D under different noise levels ( σ∈ [0.2, 1.5] rad). HaNDF (ours, red) achieves the best Δq and m2m at every noise level, with margins growing under heavier corruption.
Figure 6: Qualitative pose denoising iterations for hand-object interaction on HOI4D. Starting from noisy initializations ( t=0 ), our model iteratively refines the hand poses. By t=1000 , the hand reaches a physically plausible configuration that closely matches the Ground Truth (GT).
Figure 7: Qualitative comparison of pose denoising across different methods on HOI4D. We evaluate our method (HaNDF) against HMP and H-NRDF on a variety of object geometries. Ground Truth (GT) is provided for reference.
Missing Fingers
Missing Finger Parts
Method
Δq↓
dNN ↓
APD ↑
FID ↓
Δq↓
dNN ↓
APD ↑
FID ↓
HMP Duran et al. (2024)
24.37 ∘
18.77 ∘
36.86 ∘
0.7407
16.60 ∘
14.45 ∘
26.79 ∘
0.9448
H-PoseNDF
20.80 ∘
16.40 ∘
32.57 ∘
1.0318
12.87 ∘
8.83 ∘
20.03 ∘
0.7267
H-NRDF
16.60 ∘
8.01 ∘
24.48 ∘
0.1558
8.47 ∘
4.52 ∘
12.17 ∘
0.0453
HaNDF
12.57 ∘
5.44 ∘
20.79 ∘
0.0596
6.60 ∘
3.57 ∘
12.23 ∘
0.0106
Table 1: Hand part completion across two settings: missing fingers and partly missing fingers on HOI4D dataset.
HOI4D Liu et al. (2022)
OakInk Yang et al. (2022)
DexYCB Chao et al. (2021)
Method
MPVPE ↓
MPJPE ↓
I.V. ↓
MPVPE ↓
MPJPE ↓
I.V. ↓
MPVPE ↓
MPJPE ↓
I.V. ↓
IHoI Ye et al. (2022)
19.42
20.12
41.44
14.75
15.17
1.66
29.85
30.73
1.26
EasyHOI Liu et al. (2025)
16.64
17.06
13.28
13.17
13.48
0.93
29.64
30.08
2.61
EasyHOI + HaNDF
13.98
14.33
11.40
12.85
13.12
0.89
28.31
28.78
2.43
AlignSDF Chen et al. (2022b)
10.40
10.60
5.39
10.34
10.50
0.82
16.82
17.19
0.99
AlignSDF + HaNDF
10.22
10.43
4.39
9.98
10.17
0.87
16.59
16.97
0.98
Table 2: In-the-wild hand recovery on HOI4D, OakInk, and DexYCB. For hand-focused, fair comparisons, the object ground truth is provided during inference. Lower is better.
Figure 8: Qualitative results for in-the-wild hand recovery on HOI4D. Our HaNDF consistently improves the reconstruction quality of both EasyHOI and AlignSDF, alleviating common failure modes such as lacking contact, hand-object penetration, and implausible hand rotation.
Method
Obj retrieval
Obj recovery
Δq↓
MPJPE ↓
MPVPE ↓
obj-CD ↓
obj-F@1% ↑
obj-F@2% ↑
AlignSDF Chen et al. (2022b)
✗
N/A
14.77
12.08
11.90
0.0909
0.0296
0.1187
AlignSDF + HaNDF
✗
N/A
9.60
11.74
11.52
0.0909
0.0296
0.1187
✓
K=1
9.59
11.50
11.30
0.0896
0.0449
0.1568
✓
K=8
9.57
11.45
11.25
0.1083
0.0387
0.1360
Table 3: Hand-object recovery. We evaluate different hand priors built upon the hand–object reconstruction results from AlignSDF baseline Chen et al. (2022b) on the HOI4D dataset. We condition HaNDF on the object predicted by AlignSDF. We also refine the object by retrieving the nearest neighbor in the object latent space (with PCA), possibly with further object recovery by optimization with different K , which show advanced performance. CD refers to Chamfer Distance. The best and second-best results are highlighted in bold and underlined text , respectively.
Backbone / conditioning
Field geometry
Δq(∘)
dNN (∘)
m2m (mm)
MLP, unconditional
Euclidean
8.29
5.88
9.54
MLP, unconditional
Riemannian
7.16
6.52
8.60
MLP + object condition
Euclidean
8.15
5.12
9.06
MLP + object condition
Riemannian
5.98
4.57
7.14
Transformer, unconditional
Riemannian
6.10
5.75
7.40
Transformer + object condition
Euclidean
10.27
8.09
9.92
Table 4: Ablation over backbone, conditioning, and field geometry on HOI4D.
Δq ( ∘ ) ↓
dNN ( ∘ ) ↓
m2m ↓
Noisy input
11.74
12.43
11.13
HaNDF w/ wrong condition
5.80
5.25
7.05
HaNDF w/o object token
6.10
5.75
7.40
HaNDF
5.52
4.96
6.58
Table 5: Object-conditioning ablation on hand denoising with noise level σ=0.8 rad on HOI4D dataset.
Method
# Steps
FID ↓
dNN ↓
Runtime (ms) ↓
LatentHOI Li et al. (2025)
1000
2.7843
6.41 ∘
64.5
HaNDF
200 †
0.5482
5.07 ∘
12
HaNDF
500
0.5466
4.77 ∘
29
HaNDF
1000
0.5297
4.66 ∘
59
Table 6: Comparison of inference costs between the proposed HaNDF and LatentHOI Li et al. (2025) for object-conditioned generation on HOI4D. Runtime is reported per sample. † With 200 steps, the applied step size is doubled.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Diagram of HaNDF’s transformer backbone, which combines hand, object, and distance tokens to predict the distance.
(θ′,z,j)∈DO∑fΦ(θ′∣z)−minθ∈DjdMH(θ′,θ)
Appendix
Algorithm 1 Training HaNDF
Model
FID ↓
APD ↑
dNN ↓
HMP [ 18 ]
4.9338
44.72 ∘
30.66 ∘
H-PoseNDF
2.685
21.48 ∘
30.8 ∘
H-NRDF
0.7769
20.86 ∘
10.70 ∘
G-HOP [ 51 ]
0.7316
9.34 ∘
10.45 ∘
LatentHOI [ 29 ]
2.7843
6.01 ∘
6.41 ∘
Text2HOI [ 7 ]
7.05
31.15 ∘
26.85 ∘
Appendix
Table 7: Quantitative evaluation of pose generation on HOI4D dataset.
σ=0.4 rad
σ=0.8 rad
σ=1.5 rad
Model
dNN ↓
Δq↓
m2m ↓
dNN ↓
Δq↓
m2m ↓
dNN ↓
Δq↓
m2m ↓
HMP [ 18 ]
7.29 ∘
5.33 ∘
5.90
11.23 ∘
10.26 ∘
10.37
17.64 ∘
17.76 ∘
16.38
H-PoseNDF
3.81 ∘
5.67 ∘
6.62
5.88 ∘
8.29 ∘
9.54
10.97 ∘
14.10 ∘
15.59
H-NRDF
5.57 ∘
4.86 ∘
5.89
6.52 ∘
7.16 ∘
8.60
7.66 ∘
10.15 ∘
12.49
HaNDF
4.74 ∘
3.67 ∘
4.38
4.96 ∘
5.52 ∘
6.58
5.20 ∘
8.00 ∘
10.08
Appendix
Table 8: Quantitative evaluation of hand denoising on HOI4D dataset across three representative noise levels. All methods receive identical noisy inputs.
Figure 10: Extended qualitative comparison of pose denoising across different methods on HOI4D. We provide further comparison cases to supplement the results in the main paper. HaNDF (ours) is evaluated against HMP and H-NRDF, with Ground Truth (GT) provided for reference.
Figure 11: Additional qualitative pose denoising iterations for hand-object interaction on HOI4D. We show two further examples of the refinement steps from t=0 to t=1000 , with Ground Truth (GT) provided for reference.
Object input
Setting
+Δq ( ∘ )
+dNN ( ∘ )
+m2m (mm)
Point noise σp
0.01
+0.00
+0.00
+0.02
0.03
+0.02
+0.06
+0.07
0.05
+0.05
+0.06
+0.09
Partial visibility
25% occluded
+0.03
+0.05
+0.06
50% occluded
+0.04
+0.07
+0.10
Point-cloud sparsity
75% subsample (control)
+0.00
+0.00
+0.01
Appendix
Table 9: Robustness to object-input perturbations. We evaluate robustness to three types of perturbations: point-wise noise at different noise levels σp , partial visibility, and point-cloud sparsity. The results demonstrate good robustness across these perturbations.
Method
Δq ( ∘ )
MPVPE (mm)
MPJPE (mm)
IntVol ( cm3 )
EasyHOI
20.82
16.64
17.06
13.28
EasyHOI + HaNDF
18.60
15.49
15.88
8.11
Improvement (%)
+10.6
+6.9
+6.9
+38.9
Appendix
Table 10: EasyHOI-based end-to-end experiment. We use EasyHOI-predicted objects to match to the nearest training object by shape, and then conditioning its latent on HaNDF to do the hand refinement.
Object encoder
Δq↓
dNN ↓
m2m (mm) ↓
HaNDF+ COD-VAE [ 15 ]
6.11 ∘
5.68 ∘
7.46
HaNDF+ Tencent-XR-3DGen [ 49 ]
5.70 ∘
5.17 ∘
6.78
HaNDF+ DeepSDF-style auto-decoder
5.52∘
4.96∘
6.58
Appendix
Table 11: Ablation on object conditioning representations. We compare three pretrained object encoders as the source of object tokens fed into HaNDF on HOI4D.
Variant
Δq↓
dNN ↓
Tokens(N)
Params
FLOPs (G)
2 layers
4.95 ∘
6.08 ∘
17
1.63
0.027
4 layers
5.52 ∘
4.96 ∘
17
3.21
0.054
8 layers
5.73 ∘
5.07 ∘
17
6.36
0.109
1 object tokens
5.52 ∘
4.96 ∘
17
3.21
0.054
64 object tokens
6.40 ∘
6.09 ∘
80
3.21
0.266
Appendix
Table 12: Ablation on transformer architecture design choices on HOI4D dataset.
Metric
Without obj. optim.
With obj. optim.
Gain
Δq(∘)
9.5985±0.0008
9.5920±0.0011
0.0065±0.0011
MPJPE
11.735±0.002
11.499±0.006
0.236±0.005
MPVPE
11.516±0.002
11.300±0.005
0.216±0.005
Appendix
Table 13: Hand reconstruction results for AlignSDF + HaNDF over five random seeds on HOI4D. The gain is computed as the error without object optimization minus the error with object optimization.
PCA dimension K
Δq(∘)↓
MPJPE (mm) ↓
MPVPE (mm) ↓
2
9.68
11.73
11.53
8
9.74
11.70
11.50
16
9.63
11.63
11.42
32 (default)
9.59
11.50
11.30
48
9.60
11.53
11.33
64 (full latent)
9.59
11.52
11.32
Appendix
Table 14: Effect of the PCA dimension on object retrieval on HOI4D. The 32-dimensional setting is used in Tab. 3 . Note that 64 dimensions correspond to the full latent space.
Iterations
10
50
100
200
350
500
750
1000
Δq(∘)↓
10.18
6.97
5.86
5.48
5.49
5.52
5.56
5.57
dNN(∘)↓
10.88
7.44
5.95
5.16
4.97
4.96
4.98
4.98
Time (ms/pose) ↓
0.6
3
6
12
21
29
44
59
Appendix
Table 15: Denoising accuracy and runtime for different numbers of Riemannian projection iterations. The default setting uses 500 iterations.
Model / Setting
Δq(∘^)↓
dNN(∘^)↓
m2m(mm)↓
Noisy input
11.78
11.46
11.25
HOI4D-HaNDF (unseen category)
8.67
7.50
9.28
OakInk-HaNDF (in-category)
5.01
3.77
6.58
Appendix
Table 16: Comparison of different models and settings.
Figure 12: Generalization to Unseen Object Categories (by testing HOI4D trained HaNDF on OakInk unseen Categories).
Method
Left
Right
Both-hand mean
HaMeR
10.00 / 9.84
10.38 / 10.38
10.19 / 10.11
+ right-hand-only prior (15 joints)
–
9.75 / 9.77
–
+ bimanual prior (30 joints)
9.13 / 8.98
9.97 / 9.99
9.55 / 9.49
Appendix
Table 17: Effect of hand priors on bimanual hand refinement using the OakInk2 dataset. Results are reported for the left hand, right hand, and both hands. The best results are shown in bold.