TDC: Sim-to-Real Transferable Directional Compliance for Contact-Rich Manipulation

Yifei Yang; Anzhe Chen; Zhenjie Zhu; Kechun Xu; Yunxuan Mao; Yufei Wei; Lu Chen; Rong Xiong

cs.ROFeb 15, 2026

TDC: Sim-to-Real Transferable Directional Compliance for Contact-Rich Manipulation

Authors: Yifei Yang, Anzhe Chen, Zhenjie Zhu, Kechun Xu, Yunxuan Mao, Yufei Wei, Lu Chen, Rong Xiong, +1 more

Organizations: Zhejiang University · Zhejiang Humanoid Robot Innovation Center

Abstract

Contact-rich manipulation requires robots to regulate both motion and interaction forces, yet achieving adaptive compliance remains a fundamental challenge. Learning from real-world data is costly and risky, while simulation-based approaches struggle with the sim-to-real gap in contact dynamics; existing sim-to-real methods either require real-world adaptation or sacrifice adaptive compliance by relying on isotropic compliant controllers. Our key insight is that force regulation decomposes into a time-varying but simulation-transferable directional component and a dynamics-sensitive but manually tunable magnitude component. We instantiate this directional component as two policy outputs, a task frame and a control mode vector, predicted by a visuomotor policy adapted from a pre-trained VLA model and trained via imitation learning on automatically generated simulation demonstrations. At deployment, an admittance controller integrates these predictions with human-specified stiffness and target wrench values to realize adaptive compliance. Our approach achieves adaptive compliance using only simulation data and can benefit from large-scale VLA pre-training. Extensive real-world experiments on four contact-rich tasks, microwave opening, peg-in-hole insertion, whiteboard wiping, and door opening, demonstrate strong task success rates and robustness to external disturbances. Project page: https://yifei-y.github.io/project-pages/TDC/.

Figures & tables

Figure 1: Task frames and per-axis control modes (the directional component of force regulation) visualized across four contact-rich tasks. Rotational axes are omitted for three tasks where they remain in low-stiffness position control. This directional component evolves with task geometry, making manual specification impractical, yet it is governed by geometry rather than contact dynamics and thus sim-to-real transferable. TDC trains a policy on simulation data alone to predict it, achieving adaptive compliance.

Figure 2: Overview of the proposed framework. (Top) Real objects are scanned into simulated scenes, where a privileged expert generates demonstrations and retains only successful rollouts. (Bottom right) A visuomotor policy built on a pretrained VLA model jointly predicts end-effector poses, task frame orientations, and control mode vectors; new output heads are randomly initialized while original weights are inherited. (Bottom left) At deployment, an admittance controller combines policy predictions with operator-tuned stiffness and force setpoints for adaptive compliance.

Figure 3: Ground-truth task frame and control mode vector.

	MO		PH		WW		DO		Overall
	SR-ND	SR-D	SR-ND	SR-D	SR-ND	SR-D	SR-ND	SR-D	SR-ND	SR-D
Pi0-Real	-	-	30	40	60	20	-	-	45	30
E2VLA-Real	-	-	35	20	90	60	-	-	63	40
ForceVLA-Real	-	-	15	40	100	80	-	-	58	60
ACP-Real	-	-	45	20	60	0	-	-	53	10
Pi0-Sim+L-A	30	60	35	60	35	60	80	100	45	70

Table 1: Success rate (sr) comparison. SR-ND: sr without disturbance; SR-D: sr with disturbance. “-” indicates an unavailable real-data baseline, not a zero success rate.

Table 5

Figure 6: Sensitivity analysis: (a) stiffness

k

across four tasks; (b) target force magnitude on WW and PH; (c) high-to-low stiffness ratio on MO with

k_{\mathrm{low}}

set to 50/10. In (a) and (b), task curves and error bars are slightly offset horizontally for visibility; ticks indicate the actual parameter values.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Parameter	Distribution
Background	HDR image	$\mathcal{U}\{1,...,27\}$
Lighting	light intensity	$\mathcal{N}(1000,200^{2})$
Lighting	color temperature	$\mathcal{N}(6500,500^{2})$
Global Camera	focal length offset	$\mathcal{U}(-10,10)$
	x offset (m)	$\mathcal{U}(-0.02,0.02)$
	y offset (m)	$\mathcal{U}(-0.02,0.02)$

Appendix

Table 6: Shared domain randomization parameters applied to all four tasks.

\mathcal{U}

and

\mathcal{N}

denote uniform and Gaussian distributions, respectively.

Task	Parameter	x (m)	y (m)	z (m)	roll (deg)	pitch (deg)	yaw (deg)
MO	microwave pose	(-0.1, 0.1)	(-0.2, 0.2)	(-0.1, 0.1)	—	—	(-45, 45)
MO	robot initial pose	(-0.2, 0.2)	(-0.3, 0.3)	(-0.2, 0.2)	(-45, 45)	(-45, 45)	(-45, 45)
PH	hole pose	(-0.2, 0.2)	(-0.2, 0.2)	(-0.1, 0.1)	—	—	(-45, 45)
	peg grasp pose	—	—	(-0.005, 0.005)	—	—	—
	robot initial pose	(-0.2, 0.2)	(-0.3, 0.3)	(-0.3, 0.2)	(-45, 45)	(-45, 45)	(-45, 45)
WW	whiteboard pose	(-0.05, 0.05)	(-0.05, 0.05)	(-0.1, 0.1)	(-20, 20)	—	(-15, 15)

Appendix

Table 7: Task-specific randomization parameters. All offsets are uniformly sampled and applied to default values. “—” indicates no randomization along that axis.

Figure 7: The real-world hardware settings.

Panel of Figure 8 — Figure 8: Force profiles of TDC during real-world execution. Each panel shows RGB images with the predicted task frame overlaid (top) and the corresponding three-axis force in the task frame (bottom). Purple dashed lines mark the image capture instants. Yellow shaded regions indicate disturbance periods. (a)–(c) illustrate three types of disturbances on whiteboard wiping: the board is lowered, raised, and tilted, respectively. (d) shows a full peg insertion. (e)–(f) and (g)–(h) exhibit undisturbed and disturbed trials for microwave opening and door opening, respectively.

Explore similar work

ConCent: Contact-Centric Real-to-Sim-to-Real Learning from One Demonstration
Jun 29, 2026Heecheol Kim, Namiko Saito, Katsushi Ikeuchi +1Sim-To-Real Reinforcement Learning Sim-To-Real Gap
Microsoft Research Asia - Tokyo · The University of Tokyo
VIDP: Variable Impedance Diffusion Policy for Compliant Robot Manipulation from Diverse Demonstrations
Aug 6, 2026Hisham Khalil, Neil Fernandes, Thomas M. Kwok +2Impedance Control Robotic Manipulation Policies
Department of Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, Canada · School of Computer Science and the Department of Electrical and Computer Engineering, McGill University, Montreal, QC, Canada
Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation
May 27, 2026Jiahe Pan, Stelian Coros, Jitendra Malik +1Sim-To-Real Reinforcement Learning Contact-Rich Dexterous Manipulation
1ETH Zürich · 2UC Berkeley

Jun 29, 2026, cs.ROJ/K move · Enter open · S save

ConCent: Contact-Centric Real-to-Sim-to-Real Learning from One Demonstration

Heecheol Kim, Namiko Saito, Katsushi Ikeuchi, Yasuyuki Matsushita

Microsoft Research Asia - Tokyo · The University of Tokyo

Sim-To-Real Reinforcement Learning Sim-To-Real Gap Contact-Rich Manipulation Robotic Manipulation Concentration

Open paper AskPDF arXiv

Sim-to-real policy transfer -- deploying policies trained in simulation in the real world -- is a promising paradigm for scaling robot manipulation without large-scale real-world data. However, transferring simulation-trained policies remains challenging due to discrepancies in contact dynamics -- particularly in contact-rich tasks where subtle differences can alter task outcomes entirely. Because interaction between the manipulated object and the environment is mediated through contact, task success depends on accurately reproducing task-relevant contacts. Accordingly, in manipulation, contact-centric fidelity -- reproducing both the contact event sequence (when, where, and how contacts occur) and the local contact dynamics (how forces and motions evolve at each contact) -- is a necessary condition for task success. Based on this insight, we propose a contact-centric real-to-sim-to-real RL framework that uses task-relevant contact event sequences extracted from real demonstrations as the learning objective. We approximate objects as groups of primitives and optimize their contact geometry in simulation so that the resulting local contact dynamics explain the observed state transitions. The contact event sequence is automatically extracted by replaying the demonstration. This sequence serves as a structured reward signal, guiding the policy toward physically plausible contact regimes validated in reality and preventing exploitation of unrealistic simulator contacts. The signal is obtained automatically, requiring no per-task reward design. Experiments on contact-rich manipulation tasks demonstrate more stable and robust sim-to-real policy transfer compared to unconstrained RL baselines.

Abstract

Explore similar work

ConCent: Contact-Centric Real-to-Sim-to-Real Learning from One Demonstration

VIDP: Variable Impedance Diffusion Policy for Compliant Robot Manipulation from Diverse Demonstrations

Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation