cs.CVNov 4, 2026

TouchéWorld tech report : Towards Online Unified Camera--Hand Exploration in Generative Worlds

Authors: Jiaao Yu, Yanjian Zhang, Haoyang Jiang, Guillaume Wisniewski, Nadi Tomeh, Shijie Jia

Organizations: East China Normal University · Université Sorbonne Paris Nord, LIPN, CNRS · Université Paris Cité, LLF, CNRS · Dalian Jiaotong University

Abstract

Recent video world models allow users to explore generated environments through camera navigation. Continuous hand motion provides a fine-grained action channel for interacting with the people and objects encountered during exploration. We present TouchéWorld, a framework that integrates gesture-driven interaction into a camera-controlled video world model. Given an initial image and a scene-description prompt, the model generates video conditioned on temporally aligned camera and hand controls. We construct joint camera--hand training data from heterogeneous egocentric videos through annotation completion, consistency-based filtering, and temporal alignment. Building on a pretrained camera-controlled world model, we combine spatial hand-skeleton conditioning with numerical hand-motion features through parameter-efficient adaptation. Causal adaptation and few-step distillation support chunk-level online generation, allowing users to update their controls based on generated visual feedback. We further introduce TouchéBench to evaluate hand-motion following, camera--hand coordination, online interaction, and object response. Together, these components establish a framework for studying continuous exploration and hand-driven interaction within generative worlds.

Figures & tables

Explore similar work

CardsList
  1. Hy-Nomas: Towards Free-form Interactive World Generation

    Oct 27, 2026Qiaoling Chen, Jiaxiang Cheng, Xueqing Deng +35Interactive World ModelsVideo Generation