cs.CVNov 18, 2026

TOMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting

Authors: Guanzhong Sun, Junyi Ma, Yuxuan Wu, Erhang Zhang, Chenyang Xu, Yanzi Miao, Hesheng Wang

Organizations: School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China. · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China. · Shanghai Innovation Institute, Shanghai, China. · Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai, China.

Abstract

Forecasting future hand movements from egocentric videos plays an important role in understanding human intentions and hand-object interactions. However, different manipulation intentions can lead to substantially different hand trajectories even when interacting with the same object. Existing methods have not sufficiently explored the intrinsic relationships between articulated hand movements and object semantics to capture such operation-dependent motion patterns. Therefore, we propose TOMAS, a topology-aware operation modeling and adaptive staging framework for egocentric hand forecasting. Specifically, we develop a topology-aware hand-object reasoning mechanism that associates hand skeletal structures and articulation dynamics with object semantics and spatial relationships to jointly infer potential interaction targets and latent manipulation operations. The resulting object-operation representations provide interaction conditions for future hand trajectory prediction, with supervision adapted to the annotations available in each dataset. Furthermore, we introduce an adaptive staging mechanism that organizes future hand movements into ordered latent stages with learnable temporal boundaries. Stage-specific geometric subgoals are then incorporated to guide trajectory refinement according to the underlying manipulation patterns. Experimental results on multiple publicly available benchmarks demonstrate that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting.

Figures & tables

Explore similar work

CardsList
  1. EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting

    Aug 23, 2026Wen Wang, Ruibing Hou, Hong Chang +2Human Motion PredictionRobotic Manipulation