Jun 11, 2026 · cs.ROJ/K move · Enter open · S save
Zhiyuan Zhang, Pokuang Zhou, Kaidi Zhang, Adeesh Desai+5
Contact-rich manipulation requires world models to capture complex interaction dynamics from heterogeneous visual and tactile observations, yet the representation properties that enable reliable predictive planning remain poorly understood. We present ContactWorld, a systematic study of vision-tactile representations across 12 contact-rich manipulation tasks. Through controlled evaluation within a unified world-model and planning framework, we find that representations preserving spatial structure and temporal continuity consistently support more accurate prediction and stronger planning performance. Point-cloud observations increase average success from 20.7% and 22.0% with wrist- and front-view RGB, respectively, to 32.1%. Tactile sensing provides further gains only when its representation is compatible with the visual modality, with point clouds and tactile force fields achieving the highest overall success rate of 36.1%. These advantages become more pronounced at increasing goal offsets, where prediction errors and contact uncertainty accumulate. Controlled representation studies and real-world experiments across four manipulation tasks further support these trends. Together, our results establish spatial structure, temporal continuity, and cross-modal compatibility as key principles for designing vision-tactile world models for contact-rich robotic manipulation.