Force-Driven Hook Disassembly via Constraint-Aware Object-Centric Reinforcement Learning

Nadeen Elhadad; Takuya Kiyokawa; Kensuke Harada

cs.ROOct 28, 2026

Force-Driven Hook Disassembly via Constraint-Aware Object-Centric Reinforcement Learning

Authors: Nadeen Elhadad, Takuya Kiyokawa, Kensuke Harada

Organizations: N. Elhadad, T. Kiyokawa, and K. Harada are with the Department of Systems Innovation, Graduate School of Engineering Science, The University of Osaka, 1-3 Machikaneyama, Toyonaka, Osaka, Japan. · K. Harada is with the Industrial Cyber-physical Systems Research Center, The National Institute of Advanced Industrial Science and Technology (AIST), 2-3-26 Aomi, Koto-ku, Tokyo, Japan.

Abstract

The growing volume of end-of-life products has intensified the need for automated disassembly, yet hook-type connections remain challengin: their orientations vary, they deform under load, and their geometry is hidden, so vision alone cannot identify a feasible non-destructive disassembly direction. This work proposes a constraint-aware reinforcement learning framework that infers feasible extraction directions online from contact forces. The policy is trained in a robot-agnostic, object-centric simulation on a simplified peg-in-hole pair and deployed zero-shot on a physical robot, where it transfers to real connections with unseen, multi-stage constraint structures. Because instantaneous contact forces are noisy and locally ambiguous, the policy is coupled with a particle filter belief that provides the policy with temporally integrated directional information and reduces redundant exploratory motion. The approach is validated on four real-world tasks, from peg-in-hole pairs to the inner propeller and front cover of an end-of-life air-conditioning unit. The full method achieves success rates of 100%, 100%, 90%, and 75% with the highest path efficiency on every task, and raises the success rate on the air-conditioning cover from 45% to 75% relative to a policy trained without the belief. These results indicate that force-driven reinforcement learning can enable disassembly without product-specific models or demonstrations.

Figures & tables

Fig. 1 : CAD render of the target outdoor air-conditioning unit viewed from two angles. The front cover is retained by two asymmetric constraint mechanisms: two downward-facing hooks on the left edge and two lip-type fasteners on the right edge. The cover material exhibits mild elastic compliance, permitting local elastic deformation that influences the feasible extraction path.

Fig. 2 : Training Environment: Simulated contact interaction under misaligned and aligned extraction actions. When the commanded action direction (cyan arrow) deviates from the groundtruth extraction axis (left), the peg engages the surrounding constraint geometry, producing visible contact forces (gold contact markers). On the other hand, when the action direction is more aligned to the groundtruth axis (right), contact decreases until it disappears at complete alignment. This illustrates that contact forces carry direct information about the feasible motion direction, which the proposed method exploits during learning.

Fig. 3 : Overview of the proposed framework. Left: During training, the SAC policy receives state observations and belief estimates and is shaped by a multi-component reward. Center: The particle filter maintains a belief over the feasible extraction direction, updated from force-motion observations (Section III-C ). Right: During deployment, a single-system condition is adopted where the gripper and grasped object are assumed to move as a rigid composite body and the wrist-measured forces equal the constraint force on the object [ 18 ] . Using proprioceptive information, force sensor data, and particle filter belief specifics, the policy drives a Cartesian impedance controller on the physical robot.

Parameter	Training	Deployment
Particles $N$	$64$	$256$
Force update rate	per action	per action
Evidence	force	force + motion
Force model	$\hat{f}^{\,a}$	mean of $\hat{f}^{\,a}$ , $\hat{f}^{\,p}$
Force normalization	fixed, $F_{\max}=200$ N	adaptive, $F_{\text{ref,min}}=15$ N
Dead-band $F_{\text{db}}$	–	$0.5$ N

TABLE I : Particle filter parameters used during training and deployment.

Term	Sign	Purpose	GT	Value
$r_{\text{prog}}$	$\pm$	Encourage efficient extraction progress; penalize regression.	✓	$k_{\text{prog}}=200,\ \beta=0.05$
$r_{a_{GT}}$	$+$	Encourage motion along the true joint axis.	✓	$k_{\text{GT}}=2.0$
$r_{a_{bel}}$	$+$	Reward actions aligned with the particle-filter belief.		$k_{\text{bel}}=2.0,\ c_{\text{th}}=0.6$
$r_{\text{force}}$	$-$	Penalize excessive contact forces.		$\lambda_{1}=0.5,\ \lambda_{2}=0.01,\ F_{\text{thresh}}=60\text{N}$
$r_{\text{stag}}$	$-$	Penalize stagnation over multiple steps.		$k_{\text{stag}}=0.01,\ M=20$ steps
$r_{\text{time}}$	$-$	Encourage shorter episodes.		$0.02$

TABLE II : Summary of reward components used during reinforcement learning.

Fig. 4 : Particle filter belief evolution on the star-shaped peg-in-hole task for three representative trials (fast, typical, and slow). (a) Angle between the belief $\bar{\mathbf{p}}$ and the true extraction axis. (b) Belief confidence $c$ , with the reward threshold $c_{\text{th}}$ dashed. (c) Displacement along the true extraction axis. Dotted vertical lines mark task completion. In each trial, extraction accelerates as the belief error falls and the confidence rises, while early phases with an inaccurate belief produce little progress.

Fig. 5 : Sequential two-axis disassembly task. From left to right: (a) initial insertion configuration, (b) transition point where the extraction direction changes, (c) continued extraction along the second orthogonal axis, and (d) complete removal of the peg from the fixture.

Task	Trials	Success [%]	Time [s]	Peak Force [N]	Path Eff. [%]	Direction Reversals
Star Peg-in-Hole	20	100	1.98	25.65	47	49.2
Sequential Two-Axis	20	100	2.07	33.14	82	17.4
Propeller	20	90	8.41	36.84	29	138.89
Front Cover	20	75	19.52	26.13	31	261.87

TABLE III : Real-world results of the proposed method (RL + PF) across all four tasks.

Fig. 6 : EoL air-conditioning unit front cover disassembly task. From left to right: (a) initial insertion configuration, (b) initial upward motion, (c) hook release, (d) distance increase, (e) full lip disengagement, and (f) full disassembly.

Policy	Task	Trials	Success [%] $\uparrow$	Peak Force [N] $\downarrow$	Time [s] $\downarrow$	Path Eff. [%] $\uparrow$	Direction Reversals $\downarrow$
RL + PF (ours)	Star Peg-in-Hole	20	100	25.65	1.98	47	49.2
	Sequential Two-Axis	20	100	33.14	2.07	82	17.4
	Propeller	20	90	36.84	8.41	29	138.89
	Front Cover	20	75	26.13	19.52	31	261.87
RL Only (PF ablation)	Star Peg-in-Hole	20	100	37.90	2.35	16	71.8
	Sequential Two-Axis	20	80	42.92	11.14	17	219.5

TABLE IV : Comparison of the full method (RL + PF) with the RL-only ablation and the force-following (FF) baseline across all real-world tasks. Bold indicates the best value per task.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,699 papers

Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly
Apr 22, 2026Yongqiang Zhao, Xuyang Zhang, Zhuo Chen +3RL for Robotics Contact-Rich Robotic Manipulation
Department of Engineering, King’s College London, Strand, London, WC2R 2LS, United Kingdom · Department of Informatics, King’s College London, Strand, London, WC2R 2LS, United Kingdom
PACER: Process-Aware VLA Policy Learning from Human Corrections for Robotic Disassembly
Oct 13, 2026Chang Liu, Sibo Tian, Litian Gong +4Robot Policy Learning Robotic Manipulation
Texas A&M University · University of California, Riverside · Georgia Institute of Technology
FORGE-plus: Force-Budgeted Recovery for Contact-Rich Assembly with a Frozen LLM Supervisor
Jul 23, 2026Kyupaeck Jeff Rah, Midum OhReinforcement Learning Language Model-Based Control
Independent Researcher

Apr 22, 2026J/K move · Enter open · S save

Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly

Yongqiang Zhao, Xuyang Zhang, Zhuo Chen, Matteo Leonetti+2

Department of Engineering, King’s College London, Strand, London, WC2R 2LS, United Kingdom · Department of Informatics, King’s College London, Strand, London, WC2R 2LS, United Kingdom

RL for Robotics Contact-Rich Robotic Manipulation Robotic Manipulation Robot Imitation Learning Robotic Assembly

Open paper AskPDF arXiv

Peg-in-hole (PiH) assembly is a fundamental yet challenging robotic manipulation task. While reinforcement learning (RL) has shown promise in tackling such tasks, it requires extensive exploration. In this paper, we propose a novel visual-tactile skill learning framework for the PiH task that leverages its inverse task, i.e., peg-out-of-hole (PooH) disassembly, to facilitate PiH learning. Compared to PiH, PooH is inherently easier as it only needs to overcome existing friction without precise alignment, making data collection more efficient. To this end, we formulate both PooH and PiH as Partially Observable Markov Decision Processes (POMDPs) in a unified environment with shared visual-tactile observation space. A visual-tactile PooH policy is first trained; its trajectories, containing kinematic, visual and tactile information, are temporally reversed and action-randomized to provide expert data for PiH. In the policy learning, visual sensing facilitates the peg-hole approach, while tactile measurements compensate for peg-hole misalignment. Experiments across diverse peg-hole geometries show that the visual-tactile policy attains 6.4% lower contact forces than its single-modality counterparts, and that our framework achieves average success rates of 87.5% on seen objects and 77.1% on unseen objects, outperforming direct RL methods that train PiH policies from scratch by 18.1% in success rate. Demos, code, and datasets are available at https://sites.google.com/view/pooh2pih.

Abstract

Robotic Manipulation

Explore similar work

Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly

PACER: Process-Aware VLA Policy Learning from Human Corrections for Robotic Disassembly

FORGE-plus: Force-Budgeted Recovery for Contact-Rich Assembly with a Frozen LLM Supervisor