cs.CVNov 18, 2026

HOForce: Motion and Force Reconstruction of Hand-Object Interaction Using a Single-View RGB-D Video

Authors: Jiahui Wang, Wanqing Du, Yuanbin Zou, Zhefeng Xu, Feng Xu

Organizations: Tsinghua University · Parity Lab · Hangzhou Dianzi University

Abstract

Reconstructing hand-object interactions is an important task, and achieving it with a single camera makes large-scale capture practical but challenging due to heavy occlusions. We introduce HOForce to track hands and objects, and further recover interaction forces from single-view RGB-D video and known object properties. For two-hand manipulation, we jointly model fingertip forces of both hands and the contact forces between objects when two objects are presented. The reconstructed forces also improve motion tracking by giving pose constraints in the occluded regions. HOForce recovers hand and object motion more accurately on DexYCB and TACO and produces less hand-object penetration than interaction baselines on DexYCB. On our dataset of fingertip forces recorded with tactile sensors, HOForce reduces normal-force MAE by 24.5% relative to the strongest baseline. The recovered forces reproduce object motion in MuJoCo and transfer to other engines after adaptation.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction

    Sep 29, 2026Wenliang Guo, Zhanbo Huang, Yu Kong3D ReconstructionMonocular 3D Reconstruction

  2. OmniHOI: Dexterous Hand-Object Interaction from Monocular Human Video

    Oct 7, 2026Ting Mao, Yanming Shao, Ziheng Wang +4Hand-Object InteractionHand-Object Interaction Reconstruction

  3. Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstruction

    May 20, 2026Hao Xu, Yilin Liu, Yinqiao Wang +2Human-Object Interaction3D Reconstruction