Contact-Aware Imitation Learning Through Contact Factorization

Jiho Hong; Daeun Song; Sanghyun Kim; Mingyo Seo

cs.ROOct 7, 2026

Contact-Aware Imitation Learning Through Contact Factorization

Authors: Jiho Hong, Daeun Song, Sanghyun Kim, Mingyo Seo

Organizations: Kyung Hee University · Ewha Womans University · University of Central Florida

Abstract

Generalizable contact-rich manipulation requires robots to preserve intended task behavior while adapting its physical realization to changing contact conditions. However, interaction forces can vary substantially with small changes in surface geometry, orientation, and friction, making policies trained directly on raw force measurements difficult to transfer beyond demonstrated conditions. We introduce FACE, a contact-factorized imitation learning framework that separates intended task behavior from environment-dependent contact factors. Our representation expresses interaction forces in normalized, contact-relative coordinates, while a learned contact-normal estimator and an online friction estimator infer the local contact normal and effective friction scale. Together, these estimators enable force observations to be encoded and policy outputs to be decoded into physical motion and force commands during execution. In this way, FACE adapts execution to current contact conditions while preserving the intended task behavior, without updating the policy parameters. We evaluate FACE on real-robot contact-rich manipulation under unseen variations in surface properties and geometry, demonstrating robust generalization across contact conditions through controlled comparisons with variants that adapt prior approaches to our setting. Videos and additional materials can be found on the project page: https://rcilab.khu.ac.kr/face.

Figures & tables

Fig. 1: Overview of FACE . FACE addresses contact-rich manipulation under uncertain physical conditions by separating the demonstrated interaction from contact-dependent force magnitude and direction. A contact-normal estimator trained through interactions with nominal object instances is combined with an online friction estimator during execution. This adaptation adjusts interaction forces online while preserving the intended behavior.

Fig. 2: Model architecture and contact-aware online adaptation. Visual and wrench observations are encoded into normalized interaction coordinates that remove contact-dependent force variation. The policy predicts normalized pose and force actions, which are decoded using online estimates of contact activation, surface normal, and friction to restore the appropriate force scale and direction. The resulting contact-aware references are tracked by a compliance controller, enabling robust execution across varying contact conditions.

Fig. 3: Hardware setup. ( a ) The handheld demonstration interface integrates an actuated parallel-jaw gripper, a wrist-mounted force/torque sensor, and an iPhone for visual-inertial tracking. ( b ) During data collection, the human demonstrator directly operates the interface, enabling intuitive demonstration of contact-rich interactions while synchronously recording motion, visual observations, and contact forces. ( c ) During robot deployment, the handle is removed and the same gripper and sensing assembly is mounted on the robot, reducing end-effector mismatch between human demonstration and execution.

Fig. 4: Success and failure modes in the marking task. Representative outcomes of the marking task. A successful trial produces a continuous target trajectory, while failures arise from insufficient contact force, excessive contact force, or loss of contact during execution.

Fig. 5: Contact factorization and online contact factor estimation. ( a ) Tangential loading $\lvert f_{\mathrm{tan}}\rvert$ varies across contact conditions, while the factorized coordinate $c=\log\rho$ , with $\rho=f_{\mathrm{tan}}/(\bar{\mu}f_{\mathrm{nor}})$ , remains centered near zero during steady sliding. For each material, the four violins correspond, from left to right, to flat, convex A (uphill), convex A (downhill), and convex B. ( b ) The residual $c$ exhibits a consistent temporal structure across geometries and materials, with larger deviations during initial contact and escape. ( c ) Online contact-normal estimation remains accurate on both seen and unseen geometries. ( d ) The normalized friction estimate $\hat{\mu}/\bar{\mu}$ converges toward unity during steady sliding for seen and unseen materials. Cotton and convex B are excluded from estimator training; lines show per-condition medians and shaded regions indicate interquartile ranges.

Fig. 6: Timelapse of deploying FACE across unseen objects. Despite substantial variations in surface geometry, material properties, and contact conditions, FACE adapts the interaction online and consistently executes the intended marking trajectory, demonstrating robust generalization beyond the conditions observed during training.

Method	Paper Cup					Egg
Method	Normal	Horizontal (mild)	Horizontal (severe)	Vertical (mild)	Vertical (severe)	Egg
Visual Observation Only	0% ( 0% , 80% , 20% , 0% )	0% ( 0% , 0% , 100% , 0% )	0% ( 40% , 0% , 60% , 0% )	0% ( 40% , 0% , 60% , 0% )	0% ( 20% , 20% , 60% , 0% )	0% ( 0% , 0% , 96% , 4% )
Force as Observation	0% ( 80% , 0% , 20% , 0% )	20% ( 40% , 0% , 20% , 20% )	0% ( 0% , 0% , 100% , 0% )	0% ( 100% , 0% , 0% , 0% )	0% ( 100% , 0% , 0% , 0% )	0% ( 0% , 0% , 0% , 100% )
No Force-Factorization	80% ( 0% , 0% , 0% , 20% )	60% ( 40% , 0% , 0% , 0% )	0% ( 0% , 0% , 100% , 0% )	80% ( 20% , 0% , 0% , 0% )	40% ( 40% , 0% , 20% , 0% )	0% ( 0% , 0% , 0% , 100% )
No Learned Contact-Normal Estimation	40% ( 40% , 0% , 20% , 0% )	0% ( 100% , 0% , 0% , 0% )	0% ( 100% , 0% , 0% , 0% )	0% ( 0% , 0% , 100% , 0% )	0% ( 0% , 0% , 100% , 0% )	4% ( 0% , 56% , 0% , 40% )
Full Method	100% ( 0% , 0% , 0% , 0% )	80% ( 0% , 0% , 20% , 0% )	60% ( 0% , 0% , 40% , 0% )	60% ( 40% , 0% , 0% , 0% )	60% ( 0% , 0% , 40% , 0% )	68% ( 24% , 8% , 0% , 0% )

TABLE I: Quantitative results across contact geometry variations. We vary the local surface geometry using folds and wrinkles with different orientations (horizontal or vertical) and deformation severities (mild or severe) across paper cups and eggs. Success rates and failure-mode breakdowns are reported as percentages over 5 trials for each method and condition. Higher success rates are better. Failure rates are shown in parentheses in the following order: Insufficient Force (purple), Excessive Force (orange), Contact Loss (green), and Incorrect Motion (red); lower values are better.

Method	Normal	PVC Tape	Plastic Shell	Wet Plastic Shell
Visual Observation Only	0% ( 0% , 80% , 20% , 0% )	0% ( 50% , 10% , 40% , 0% )	10% ( 30% , 0% , 60% , 0% )	20% ( 10% , 0% , 70% , 0% )
Force as Observation	0% ( 80% , 0% , 20% , 0% )	0% ( 30% , 0% , 0% , 70% )	10% ( 90% , 0% , 0% , 0% )	0% ( 100% , 0% , 0% , 0% )
No Force-Factorization	80% ( 0% , 0% , 0% , 20% )	40% ( 0% , 60% , 0% , 0% )	50% ( 0% , 50% , 0% , 0% )	60% ( 0% , 0% , 0% , 40% )
No Learned Contact-Normal Estimation	30% ( 50% , 0% , 20% , 0% )	30% ( 30% , 10% , 10% , 20% )	10% ( 20% , 0% , 70% , 0% )	10% ( 20% , 0% , 70% , 0% )
Full Method	100% ( 0% , 0% , 0% , 0% )	100% ( 0% , 0% , 0% , 0% )	100% ( 0% , 0% , 0% , 0% )	90% ( 10% , 0% , 0% , 0% )

TABLE II: Quantitative results across contact surface variations. We vary the contact material from the nominal surface (Normal) to PVC tape and a smooth plastic shell, with the wet plastic shell introducing an additional low-friction surface condition. Success rates and failure-mode breakdowns are reported as percentages over 10 trials for each method and condition. Higher success rates are better. Failure rates are shown in parentheses in the following order: Insufficient Force (purple), Excessive Force (orange), Contact Loss (green), and Incorrect Motion (red); lower values are better.

Explore similar work

ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching
May 11, 2026Shuoheng Zhang, Yifu Yuan, Hongyao Tang +7Force Multimodal Action Distributions
Tianjin University · Shanghai AI Lab · Huawei Noah’s Ark Lab
Wrench-ACT: Enhancing Robot Policies for Contact Rich Behavior Using Direct Wrench Control
Sep 29, 2026Johannes Hechtl, Yannik Blei, Simon Ball +5Robotic Manipulation Policies Contact Wrench Estimation
Siemens Research and Predevelopment · Department of Computer Science and Artificial Intelligence, University of Technology Nuremberg, Germany
TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
Jun 9, 2026Yujie Zang, Yuhang Zheng, Xian Nie +7Tactile World Model Tactile
TARS Robotics · National University of Singapore · Shanghai Jiao Tong University +2

May 11, 2026, cs.ROJ/K move · Enter open · S save

ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching

Shuoheng Zhang, Yifu Yuan, Hongyao Tang, Yan Zheng+6

Tianjin University · Shanghai AI Lab · Huawei Noah’s Ark Lab

Force Multimodal Action Distributions Conditional Flow Matching Multimodal Fusion

Open paper AskPDF arXiv

Existing imitation learning methods enable robots to interact autonomously with the physical environment. However, contact-rich manipulation tasks remain a significant challenge due to complex contact dynamics that demand high-precision force feedback and control. Although recent efforts have attempted to integrate force/torque sensing into policies, how to build a simple yet effective framework that achieves robust generalization under multimodal observations remains an open question. In this paper, we propose ForceFlow, a force-aware reactive framework built upon flow matching. For contact-stage policy design, we investigate force signal fusion mechanisms and adopt an asymmetric multimodal fusion architecture that treats force as a global regulatory signal, combined with a joint prediction paradigm that enhances the policy's understanding of instantaneous force and historical information, thereby achieving deep coupling between force and motion. For task-level hierarchical decomposition, we divide manipulation into a vision-dominant approach stage (VLM-based pointing for target localization) and a touch-dominant interaction stage (force-driven contact execution), with a Vision-to-Force (V2F) handover mechanism that explicitly decouples spatial generalization from contact regulation. Experimental results across six real-world contact-rich tasks demonstrate that ForceFlow achieves a 37% success rate improvement over the strong baseline ForceVLA while maintaining significantly lower cost. Moreover, ForceFlow exhibits accurate force signal prediction and demonstrates superior performance in contact force self-regulation and zero-shot out-of-distribution (OOD) generalization.

Abstract

Explore similar work

ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching

Wrench-ACT: Enhancing Robot Policies for Contact Rich Behavior Using Direct Wrench Control

TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation