Manifold Generative Annealing for Contact Constraint-aware Impedance Control with Reinforcement Learning Prior

Zhilin He; Guoquan Qi; Changliu Liu

cs.ROOct 15, 2026

Manifold Generative Annealing for Contact Constraint-aware Impedance Control with Reinforcement Learning Prior

Authors: Zhilin He, Guoquan Qi, Changliu Liu

Organizations: Carnegie Mellon University · Institute of Automation, Chinese Academy of Sciences

Abstract

Contact-rich robotic tasks require coordinated motion and impedance under uncertain interaction dynamics and physical constraints. While recent methods on learned policies provide useful interaction strategies, they may fail when contact conditions differ from those encountered during training. Model-based generative methods show promising receding-horizon trajectories optimization results, but often lack structured behavioral guidance and refinement that jointly accounts for constraint geometry and actual contact responses. To this end, we propose Manifold Generative Annealing (MGA), a joint model-free and model-based generative framework for motion--impedance optimization. MGA first organizes pretrained reinforcement learning policies into structured sequence proposals, retaining task-specific interaction strategies without committing to their execution. It then evaluates these proposals alongside exploratory samples through model-based rollouts for refinement or separate execution selection. Crucially, realization-aware manifold refinement aligns updates from the refinement bank with both task constraints and locally achievable physical responses, allowing learned experience to guide online adaptation under changing contact conditions. Experiments cover surface scanning, peg insertion, and humanoid pushing, with improved safety success rate of impedance control under nominal and sensing-shift conditions.

Figures & tables

Figure 1: Left: Overview of MGA : (1) offline policy training, (2) model-based refinement, and (3) force-adaptive deployment with feedback. Right: an open-circle start-to-goal toy: (A) Gaussian-initialized refinement, (B) RL prior, (C) MGA w/o LRC, and (D) MGA . The prior supplies path structure. LRC reduces maximum median-path drift: $0.165$ (B), $0.129$ (C), $0.008$ (D). Arrows mark full-path maxima of distance to the realized manifold. C/D differ only in LRC.

Panel of Figure 2 — Figure 2: The environments are (a) Surface Scanning over rigid (purple) and soft (green) surfaces with varied geometries and hybrid hard–soft patterns, (b) Peg Insertion into a fixed socket under nominal conditions (ID), perturbed socket poses (Pose-OOD), and biased force/torque sensing (Sensing-OOD), and (c) Humanoid Push for constant $15$ -N force regulation, fixed-stance pushing to the green line, and box unjamming with yaw correction.

Panel of Figure 3 — Figure 3: Left: Scanning panels show a (a) bumpy rigid surface replay, (b) hybrid stripe rigid-soft pattern execution coverage and their (c) force–stiffness response, and (d) compares MGA and w/o LRC on a soft curved surface. Right panels show Pose-OOD Peg Insertion (e) contact progression replay, (f) unfiltered wrench utilization, and (g) logged plan selection for RL prior.

Figure 4: Panels (a–c) show force regulation, pushing, and unjamming. Panel (d) shows 10 paired position–yaw trajectories. Crosses mark safety-triggered stops. Force transients are retained in (a).

Figure 5: Left shows the hardware setup. Middle shows table, foam, and phantom scanning from top to bottom, with force references of $3$ , $0.8$ , and $3$ N (Actual: $2.99\pm 0.01$ , $0.82\pm 0.02$ , $2.94\pm 0.06$ N, respectively). Right plots lateral offset $e$ (mm) against base frame height $z$ (mm) during insertion.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Real-world contact fixtures: the rigid table region, compliant foam, and curved phantom (left), the phantom profile (center), and the pipe used for clearance insertion (right). Dimensions describe the fixtures rather than the commanded path lengths.

Method		Policy Role	Model-based	Horizon Rollout	Constraint Treatment	Realization Geometry	LRC	Learned Reliability
Baselines	ISSA	Nominal action	✗	✗	Implicit safe-set filter	✗	✗	✗
	ATACOM	Tangent action	✗	✗	Augmented constraint manifold	✗	✗	✗
	MPPI	✗	✓	✓	Reward penalties	✗	✗	✗
	DIAL	✗	✓	✓	Reward penalties	✗	✗	✗
	PegasusFlow	✗	✓	✓	Reward penalties	✗	✗	✗
Variants	MGA	Structured proposals	✓	✓	Feasibility cost + local correction	✓	✓	✓

Appendix

Table 2: The table compares the configured simulation baselines and MGA variants. Entries describe the task-adapted implementations, with ✓ and ✗ denoting presence and absence of the indicated mechanism. Model-based denotes use of an explicit dynamics model to predict and optimize candidate trajectories. Black-box safety queries and local constraint geometry do not qualify. Realization geometry and LRC denote response-conditioned sequence refinement and its post-transport residual correction, not action-level constraint handling.

Surface	$F_{n}^{\star}$ (N)	Mean $F_{n}$ (N)	MAE (N)	RMSE (N)
Table	$3.0$	$2.987\pm 0.014$	$0.391\pm 0.054$	$0.506\pm 0.094$
Foam	$0.8$	$0.817\pm 0.017$	$0.176\pm 0.006$	$0.222\pm 0.012$
Phantom	$3.0$	$2.940\pm 0.064$	$0.378\pm 0.013$	$0.454\pm 0.015$

Appendix

Table 3: Surface-scanning results report the mean

\pm

sample standard deviation over five trials per material. Metrics use unsmoothed compensated feedback and physical timestamps, including recovery intervals. Complete-session outcomes are reported separately in the text.

Figure 7: The plots compare terminal progress and contact-valid coverage across all scanning conditions with matched settings and seeds. Large markers show suite means, and small markers retain individual executions. Held-out surfaces are not included again in the Hard and Soft groups.

Method	Operational success	Zero-violation success	Coverage (%)
MGA	$120/130$	$119/130$	$96.03\pm 1.38$
w/o RL prior	$120/130$	$117/130$	$97.10\pm 1.26$
w/o LRC	$87/130$	$86/130$	$84.75\pm 6.19$

Appendix

Table 4: Scanning success is evaluated under two force-violation allowances. Both criteria require coverage

\geq 90\%

and settled contact loss

\leq 10\%

. Counts cover all

13\times 10

executions. Coverage is reported as the mean and population standard deviation of the ten equal-suite seed means.

Figure 8: Matched seed-level observations compare coverage, force tails, and deformation for MGA and variants without the learned prior or LRC under compliant and hybrid contact. Force utilization is normalized by the

60

-N limit, and deformation is reported in physical units.

Figure 9: The Soft Convex panels show native MGA context and matched executions. Visit strips count reference waypoints within $5$ mm of contact-valid executed samples in 3-D. The error trace measures executed-point distance to the reference polyline, also in 3-D. The boundary marks the same tolerance.

Figure 10: Insertion outcomes are reported across ten seeds per method and setting. All eight methods use the same frozen evaluation conditions. Safe completion, completion with a wrench violation or jam, and incomplete insertion are reported separately. No trial is discarded.

Figure 11: Depth and wrench utilization for ten seeds per variant and setting, with seed $0$ emphasized. All recorded steps are retained. The depth and $\rho=1$ thresholds do not subsume pose tolerances or the three-step hold.

Setting	Method	Completed	$F_{xy}^{\max}$ (N)	$F_{z}^{\max}$ (N)	F/T violation (%)
ID	MGA	$10/10$	$15.85\pm 0.13$	$14.51\pm 1.18$	$0.00\pm 0.00$
	w/o RL prior	$9/10$	$15.14\pm 1.93$	$15.04\pm 1.12$	$0.00\pm 0.00$
	w/o LRC	$5/10$	$10.46\pm 6.14$	$7.98\pm 5.05$	$0.00\pm 0.00$
Pose-OOD	MGA	$10/10$	$20.05\pm 3.09$	$18.00\pm 3.48$	$0.62\pm 1.25$
	w/o RL prior	$5/10$	$17.17\pm 3.02$	$16.73\pm 2.46$	$0.00\pm 0.00$
	w/o LRC	$3/10$	$12.46\pm 9.05$	$10.01\pm 8.23$	$0.78\pm 1.88$

Appendix

Table 5: The table reports contact outcomes for MGA and both controlled ablations. Completed counts denote raw task completion. Loads and violation rates are mean

\pm

population standard deviation over all ten trials, including failures. None of these three variants records a jam.

Figure 12: Pose-OOD replay and all ten recorded executions for MGA and w/o RL prior, with seed $0$ emphasized. Circles mark endpoints and diamonds first completion. Load exceedances remain visible. The shaded positional target uses lateral error $\leq 1.2$ mm and depth $\geq 32$ mm, separately from the orientation, hold-time, and full-window safety conditions.

Figure 13: The plot compares impact and steady-state error for

15

N Force Regulation. Small points denote seeds, and large markers denote method means. All methods share physical sampling.

Figure 14: Unjamming traces end at completion or safety-triggered rejection, without absorbing padding. Rejected actions are not executed.

Figure 15: The panels show simulated Unjamming behavior for MGA, w/o RL prior, and w/o LRC. Native imagery provides task context. The plotted motion and force are from the matched executions, ending at completion or rejection without absorbing padding. Force is synchronized with the $20$ -ms control-step pose records. Physical impact and safety analyses elsewhere retain the finer $4$ -ms samples. The position–yaw target is a task-completion condition, not a complete safety certificate.

Setting	Scanning	Peg Insertion	Humanoid Push
Policy optimizer	PPO	PPO	PPO
Hidden layers	$4\times 32$	$2\times 16$	$2\times 16$
Recorded training budget	$2$ M	$200$ k	$200$ k
Selected PPO checkpoint	$1$ M	$200$ k	Final
ATACOM proposal expert	Yes	No	No
Selected ATACOM checkpoint	$500$ k	–	–

Appendix

Table 6: The table lists frozen proposal settings, with budgets measured in training environment steps. The ATACOM rows describe a proposal expert. The separate baseline is described in Appendix D .

Setting	Scanning	Peg Insertion	Humanoid Push
Evaluation intervals	$100$	$64$	$100$
Control interval (s)	$0.02$	$0.02$	$0.02$
Dense-grid intervals ( $H_{\mathrm{sample}}$ )	$16$	$8$	$16$
Node intervals ( $H_{\mathrm{node}}$ )	$4$	$4$	$4$
Refinement-bank budget $M$	$64$	$64$	$64$
Initial refinement iterations	$10$	$4$	$10$

Appendix

Table 7: The table lists the recorded MGA inference settings. Peg insertion retains the full

64

-step window after completion. Other tasks use their documented termination rules. Sampling temperature enters

\beta_{k}=\tau\sigma_{J,k}

and is not a physical contact parameter.

Recorded setting	Gaussian	PPO	ATACOM	Integration
Scanning	$48$	$8$	$8$	Replacement bank
Peg Insertion: all settings	$64$	$8$	$0$	Additive expert
Humanoid Push	$64$	$8$	$0$	Additive expert

Appendix

Table 8: The table lists the recorded stochastic proposal counts. Deterministic anchors and execution-revalidation queries are additional.

Setting	Scanning	Peg Insertion	Humanoid Push
Linear feasibility coefficient	$2$	$2$	$2$
Quadratic feasibility coefficient	$200$	$80$	$200$
Normalized geometric metric	$I$	$I$	$I$
LRC iterations per correction	$1$	$1$	$1$
LRC gain $\eta$	$1$	$1$	$1$
LRC regularization $\varepsilon$	$10^{-6}$	$10^{-6}$	$10^{-6}$

Appendix

Table 9: The table lists the evaluation and normalized correction settings. Task modulation, trust maps, and execution checks are detailed in Appendix H . The operator alone does not certify safety.

Figure 16: MGA scans the four rigid surfaces: plane, cylinder, convex, and bumpy. Five recorded states per row show the executed robot and tool motion from the same seed, with a fixed camera and common rendering style. The contact inset identifies the rigid regime without a finite-stiffness color assignment.

Figure 17: MGA scans the four compliant surfaces. The frames show recorded robot motion against the nominal rendered surface. Mesh appearance alone does not measure deformation, which is quantified separately in Fig. 8 . Contact-parameter insets distinguish the compliant setting from the corresponding rigid geometry.

Figure 18: The frames show MGA executions across three planar hybrid stiffness maps: alternating stripes, hard center, and soft center. Material variation changes the contact response without introducing a curved surface. Insets show the nominal task-chart stiffness field. It is evaluated at the commanded surface coordinate, not inferred from the displayed end-effector position. All frames are states from the recorded executions.

Figure 19: MGA scans the held-out rigid and compliant surfaces. The rigid example is retained despite failing the scanning success criterion, so visual coverage of the benchmark does not select only successful cases.

Figure 20: Five chronological pose replays show MGA under ID, Pose-OOD, and Sensing-OOD. Native robot views replay recorded joint configurations without re-simulation. Quantitative plots use the saved task-coordinate signals. The fixed examples all complete safely. The statistical comparisons retain every trial, including the two unsafe Pose-OOD executions.

Figure 21: The frames show MGA in Force Regulation, Fixed-stance Push, and Unjamming. Five frames span each execution before terminal padding. Goal markings appear only in the corresponding displacement tasks. These are not walking pushes.

Contact-Rich Robotic Manipulation

Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.

See all 410 papers

Constraint-Decomposed Impedance Control from Generative Policy for Articulated Object Manipulation
Oct 12, 2026Kyungseo Park, Byeongdo Lim, Jungbin Lim +2Impedance Control Robotic Manipulation
Seoul National University · Department of Mechanical Engineering, KAIST
Constraint-Grounded Reinforcement Learning for Variable Impedance Control in Contact-Rich Robotic Insertion
Sep 11, 2026Lin He, Min DengRobotic Insertion Impedance Control
Department of Civil and Environmental Engineering, The University of Tennessee, Knoxville, TN 37996 USA
Impedance Cloning: Learning Equilibrium Point Parameters for Contact-Rich Manipulation
Sep 25, 2026Hayato Takahashi, Ryoga Oishi, Yuki Kasuga +1Contact-Rich Robotic Manipulation Force-Controlled Robotic Manipulation
Graduate School of Science and Engineering, Saitama University, Saitama, Japan

Oct 12, 2026J/K move · Enter open · S save

Constraint-Decomposed Impedance Control from Generative Policy for Articulated Object Manipulation

Kyungseo Park, Byeongdo Lim, Jungbin Lim, Minseok Choi+1

Seoul National University · Department of Mechanical Engineering, KAIST

Impedance Control Robotic Manipulation Articulated Object Manipulation

Open paper AskPDF arXiv

Articulated object manipulation tasks such as opening doors, tightening bolts, and pulling drawers impose kinematic constraints on a robot end-effector's motion that neural policies may fail to satisfy. Unlike hybrid force-position control that strictly separates force and position control into orthogonal subspaces, impedance control modulates a mechanical impedance system in the form of a virtual inertia, damping, and stiffness. However, selecting task-appropriate impedance gains remains challenging for robot policies. In this paper, we address this problem for the case of manipulating articulated objects. Our constraint-decomposed impedance control method uses the sampling diversity of a generative policy to estimate the feasible motion directions in twist space. The key idea is to infer the feasible and constrained subspaces of twists from an ensemble of trajectories generated by the policy from the same observation. Principal component analysis (PCA) of the end-effector twists determines the feasible twist subspace, which is then used to form 6x6 impedance gain matrices. For many common articulated objects, the feasible twist subspace is independent of the object configuration, so the subspace and resulting gains are estimated once and held fixed during manipulation, minimizing computational overhead. For less common cases where this independence does not hold, we instead estimate a configuration-independent twist subspace of higher dimension that contains the local feasible subspaces, allowing the same fixed-gain strategy to be used. Experiments across simulated and real-world single- and multi-DoF objects show that the proposed method consistently reduces constraint-violating contact loads relative to isotropic impedance control while maintaining task execution.

Abstract

Contact-Rich Robotic Manipulation

Explore similar work

Constraint-Decomposed Impedance Control from Generative Policy for Articulated Object Manipulation

Constraint-Grounded Reinforcement Learning for Variable Impedance Control in Contact-Rich Robotic Insertion

Impedance Cloning: Learning Equilibrium Point Parameters for Contact-Rich Manipulation