Imp-ACT: Adaptive Impedance Control and Action Chunking with Transformers to Learn Contact-Rich Manipulation from Demonstrations

Luca Zanetti; Doganay Sirintuna; Idil Ozdamar; Pietro Balatti; Heng Zhang; Arash Ajoudani

cs.ROSep 25, 2026

Imp-ACT: Adaptive Impedance Control and Action Chunking with Transformers to Learn Contact-Rich Manipulation from Demonstrations

Authors: Luca Zanetti, Doganay Sirintuna, Idil Ozdamar, Pietro Balatti, Heng Zhang, Arash Ajoudani

Organizations: Human-Robot Interfaces and Interaction Laboratory, Istituto Italiano di Tecnologia, Genoa, Italy · Safe AI Lab, Carnegie Mellon University

Abstract

Contact-rich manipulation requires robots to balance accurate motion tracking with compliant interaction, yet most visual-action policies leave compliance fixed at the controller level. We present Imp-ACT, a methodologically grounded and practical approach to incorporating direction-dependent Cartesian stiffness modulation directly into demonstration collection, without manual stiffness selection or offline target reconstruction. During teleoperation, a self-tuning impedance controller adapts stiffness along the instantaneous direction of motion while maintaining compliance in orthogonal directions. The adapted stiffness is applied and recorded alongside visual observations and motion commands, capturing motion and compliance under the same dynamics. We implement this pipeline using Action Chunking with Transformer (ACT) to predict end-effector pose, gripper action, and motion-direction stiffness from visual, proprioceptive, and wrench observations. The performance of Imp-ACT is evaluated on wiping and plug insertion using both success rate and quantitative measures of contact behavior. Compared with fixed low- and high-stiffness baselines, Imp-ACT achieves comparable or higher success while maintaining low interaction forces. In wiping, it reduces contact-force vibration by approximately $29\times$ relative to the compliant baseline and $180\times$ relative to the stiff baseline. In plug insertion, it reduces forces orthogonal to the insertion direction by $43\%$ relative to the better fixed-stiffness baseline. These results highlight the benefit of maintaining sufficient stiffness along the direction needed for task execution while preserving compliance in other directions to limit contact forces and accommodate environmental constraints.

Figures & tables

Fig. 1 : Imp-ACT turns intuitive human demonstrations into compliant, adaptive robot skills. By learning from human motion while an online Cartesian impedance controller generates direction-dependent stiffness, it unites natural kinematic teaching with interaction-aware control.

Fig. 2 : Overview of (a) Demonstrations and (b) Deployment phases, where $\bm{\Psi}_{B}$ denotes the base frame. During demonstration collection, images from the cameras, measured end-effector pose $\bm{X}_{a}$ , external interaction wrench $\bm{F}_{ext}$ , and the gripper state $G_{s}$ are recorded as observations. The corresponding action targets comprise the desired end-effector pose $\bm{X}_{d}$ , the gripper command $G_{c}={open,close}$ , and the applied stiffness coefficient $k_{st}$ . During deployment, Imp-ACT predicts these action targets, including the stiffness coefficient $\tilde{k}_{st}$ .

Panel of Figure 3 — Fig. 3 : Cartesian stiffness ellipsoids during wiping (top) and plug insertion (bottom). The proposed approach increases the predicted stiffness $\tilde{k}_{st}$ along the instantaneous motion direction (magenta arrow), while maintaining the two orthogonal directions at $200$ N/m. The principal stiffness direction follows the task motion, lying approximately along the wiping direction or insertion axis. Each panel reports the time (upper left) and $\tilde{k}_{st}$ (lower right). Across the illustrated episodes, $\tilde{k}_{st}$ ranges from $200$ – $642$ N/m for wiping and from $200$ – $329$ N/m for insertion.

Fig. 4 : Predicted Cartesian stiffness during one wiping episode. $K_{x}$ , $K_{y}$ and $K_{z}$ are the stiffness commanded w.r.t $\bm{\Psi}_{B}$ , and $\tilde{k}_{st}$ (dashed) the inferred stiffness along the direction of motion. The pink region marks the gripper closing on the wiper while the yellow one marks contact with the board.

Fig. 5 : Task outcome and contact metrics for wiping (top) and plug insertion (bottom). Grey-background metrics are better when higher; white-background metrics are better when lower and are computed over successful episodes only. Bold text indicates the best result, including ties.

Explore similar work

Feb 15, 2026

TDC: Sim-to-Real Transferable Directional Compliance for Contact-Rich Manipulation

Contact-rich manipulation requires robots to regulate both motion and interaction forces, yet achieving adaptive compliance remains a fundamental challenge. Learning from real-world data is costly and risky, while simulation-based approaches struggle with the sim-to-real gap in contact dynamics; existing sim-to-real methods either require real-world adaptation or sacrifice adaptive compliance by relying on isotropic compliant controllers. Our key insight is that force regulation decomposes into a time-varying but simulation-transferable directional component and a dynamics-sensitive but manually tunable magnitude component. We instantiate this directional component as two policy outputs, a task frame and a control mode vector, predicted by a visuomotor policy adapted from a pre-trained VLA model and trained via imitation learning on automatically generated simulation demonstrations. At deployment, an admittance controller integrates these predictions with human-specified stiffness and target wrench values to realize adaptive compliance. Our approach achieves adaptive compliance using only simulation data and can benefit from large-scale VLA pre-training. Extensive real-world experiments on four contact-rich tasks, microwave opening, peg-in-hole insertion, whiteboard wiping, and door opening, demonstrate strong task success rates and robustness to external disturbances. Project page: https://yifei-y.github.io/project-pages/TDC/.

Yifei Yang, Anzhe Chen, Zhenjie Zhu +6

Zhejiang University · Zhejiang Humanoid Robot Innovation Center

Contact-Rich Robotic Manipulation Sim-to-Real Transfer Impedance Control Force-Controlled Robotic Manipulation

Ask PDF

Mar 14, 2026

Stiffness Copilot: An Impedance Policy for Contact-Rich Teleoperation

In teleoperation of contact-rich manipulation tasks, selecting robot impedance is critical but difficult. The robot must be compliant to avoid damaging the environment, but stiff to remain responsive and to apply force when needed. In this paper, we present Stiffness Copilot, a vision-based policy for shared-control teleoperation in which the operator commands robot pose and the policy adjusts robot impedance online. To train Stiffness Copilot, we first infer direction-dependent stiffness matrices in simulation using privileged contact information. We then use these matrices to supervise a lightweight vision policy that predicts robot stiffness from wrist-camera images and transfers zero-shot to real images at runtime. In a human-subject study, Stiffness Copilot achieved safety comparable to using a constant low stiffness while matching the efficiency of using a constant high stiffness.

Yeping Wang, Zhengtong Xu, Pornthep Preechayasomboon +4

All authors are with Meta Reality Labs Research · Yeping Wang is with University of Wisconsin-Madison, Madison, WI, USA · Zhengtong Xu is with Purdue University, West Lafayette, IN, USA

Robot Teleoperation Contact-Rich Robotic Manipulation Impedance Control Vision-Based Robot Control

Ask PDF

Jul 16, 2026

Beyond Implicit Force: Evaluating Explicit Force-Torque Proxies in Action Chunking with Transformers

Contact-rich manipulation requires policies to infer interaction state from signals that are often weakly observable through vision and kinematics alone. Action Chunking with Transformers (ACT) has shown strong performance in fine-grained manipulation, but many deployments collect demonstrations through leader-follower teleoperation, where tracking error between commanded leader motion and executed follower motion implicitly encodes contact, resistance, and constraint violation. This paper examines whether ACT's apparent force-awareness depends on this hidden interaction cue. We introduce an observation-centric ACT variant that predicts future follower joint states instead of leader commands, thereby removing the teleoperation-induced discrepancy signal while preserving the rest of the learning pipeline. We then evaluate whether simple joint-torque proxies, derived from onboard motor current or joint effort, can recover contact-aware behavior without external force/torque sensors. Across four real-world tasks spanning surface following, insertion, stiffness discrimination, and force-based stopping, removing the implicit cue leads to severe failures in force-critical phases. In contrast, torque-augmented policies recover robust contact behavior and improve the base ACT policy. These results demonstrate that, on real hardware, the implicit teleoperation cue is a recoverable source of force-awareness, where torque signals are available, a simple proxy matches, surpasses, or further enhances it.

King Hang Wong, Lingqiao Liu, Feras Dayoub

Australian Institute for Machine Learning (AIML), Adelaide University, Adelaide, SA 5005, Australia

Contact-Rich Robotic Manipulation Robotic Manipulation Action Chunking

Ask PDF