OpenHEART: Opening Heterogeneous Articulated Objects with a Legged Manipulator

Seonghyeon Lim; Hyeonwoo Lee; Seunghyun Lee; I Made Aswin Nahrendra; Hyun Myung

cs.ROMar 6, 2026

OpenHEART: Opening Heterogeneous Articulated Objects with a Legged Manipulator

Authors: Seonghyeon Lim, Hyeonwoo Lee, Seunghyun Lee, I Made Aswin Nahrendra, Hyun Myung

Organizations: School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Republic of Korea. · KRAFTON, Seoul, 06142, Republic of Korea.

Abstract

Legged manipulators offer high mobility and versatile manipulation. However, robust interaction with heterogeneous articulated objects, such as doors, drawers, and cabinets, remains challenging because of the diverse articulation types of the objects and the complex dynamics of the legged robot. Existing reinforcement learning (RL)-based approaches often rely on high-dimensional sensory inputs, leading to sample inefficiency. In this paper, we propose a robust and sample-efficient framework for opening heterogeneous articulated objects with a legged manipulator. In particular, we propose Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization. Additionally, Articulation Information Estimator (ArtIEst) is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object. The proposed framework was deployed to manipulate various heterogeneous articulated objects in simulation and real-world robot systems. Videos can be found on the project website: https://openheart-icra.github.io/OpenHEART/

Figures & tables

Figure 1: We present a framework for opening heterogeneous articulated objects with a legged manipulator. The robot can open diverse articulated objects, such as (a) a revolute cabinet with a vertical handle and (b) a prismatic drawer with a horizontal handle without any object-specific models.

Figure 2: The framework is hierarchically structured, with a high-level planner and (a) a low-level controller. SAFE is introduced to efficiently represent the object shape, while mitigating overfitting. (b) ArtIEst estimates the articulation information by adaptively mixing exteroception and proprioception. The history encoder extracts features from proprioception history.

Figure 3: Defined articulation information (pink solid arrow) for objects with (a) a revolute joint and (b) a prismatic joint. The expected opening trajectory is shown as a blue dashed arrow.

Reward	Equation	Weight
Opening	$\xi^{\text{norm}}_{t}$	6.5
EE approach	$\begin{cases}\text{exp}(-4\\|\textbf{p}^{\text{EE}}_{t}-\textbf{p}^{\text{H}}_{t}\\|^{2}),&\text{if }\xi^{\text{norm}}_{t}<0.2\\ 1,&\text{otherwise}\end{cases}$	1.9
EE alignment	$\text{cos}^{2}\theta_{t}$	0.75
Grasping	$\mathbb{I}\{\text{grasp}\}$	1.0
Command rate	$60\lambda_{t}\cdot\\|\textbf{c}_{t}-\textbf{c}_{t-1}\\|^{2}$	-0.1
EE command rate	$30\lambda_{t}\cdot\\|\textbf{c}^{\text{EE}}_{t}-\textbf{c}_{t-1}\\|^{2}$	-0.1

Table 1: Reward function elements.

\textbf{p}^{\text{EE}}_{t}

and

\textbf{p}^{\text{H}}_{t}

denote the position of the end-effector and handle, respectively.

\theta_{t}

is the angle between the dorsal-palmar axis of EE and the longer side of the handle.

\mathbb{I}\{\cdot\}

is the indicator function, which returns one if the condition is satisfied, and zero otherwise.

Figure 4: Demonstration of a heterogeneous articulated object opening in the simulation. The objects differ in their handle shape, dimension, and opening direction.

Figure 5: (a) Learning curve of opening reward. Ours shows the highest performance compared with the baselines. Saliency maps are shown for the object with (b) revolute joint and (c) prismatic joint. Object appearances are depicted at the top line, the saliency maps from Point cloud-based policy , and Ours are shown in the middle and below each subfigure.

Figure 6: Estimation error transition of exteroception-based, proprioception-augmented, and mixed estimations during the manipulation. The progress of the opening motion is indicated by the object’s open angle normalized using the joint limits. (a) The appearance of the object can be inferred to be opened in either a rightward or upward direction, while the correct direction is upward, leading to visual ambiguity in exteroception-based estimation. (b) In contrast, no visual ambiguity exists in this case.

Method	Estimation direction error (rad) $\downarrow$
Method	Before contact	During contact	Entire episode
Ours	0.2293	0.0687	0.1701
Ours w/o prop. est.	0.2623	0.2228	0.2482
Ours w/ mono. est.	0.2574	0.0606	0.1827

Table 2: ArtIEst evaluation. The estimation errors are separately measured before and during the contact.

Figure 7: Real-world demonstration of opening a drawer with a prismatic joint and a horizontally elongated handle. Although the first grasp pose (2.1 s) was not stable, the robot autonomously retrieved the handle (3.0 s) and successfully opened the drawer.

Method	Success rate (%) $\uparrow$		Test/Train (%) $\uparrow$
Method	Train set	Test set	Test/Train (%) $\uparrow$
Ours	79.35	79.02	99.35
Ours w/o sampling	77.56	70.23	92.92
Center-based teacher [ 47 ]	62.43	50.60	81.05
Point cloud-based policy	56.58	41.39	73.15

Table 3: Success rate across training and test sets. The proposed method shows the highest performance retention.

Articulated Object Manipulation

Articulated object manipulation studies how robots perceive and operate objects with movable parts such as doors, drawers, tools, and lids.

See all 34 papers

Learning to Explore Hidden Kinematics for Articulated Object Manipulation
Sep 29, 2026Ruiyao Liu, Boshu Lei, Zhuoyang Pan +1RL for Robotics Active Perception
GRASP Lab, University of Pennsylvania, Philadelphia, PA 19104, USA
HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions
May 11, 2026Zhenhao Shen, Zeming Yang, Yue Chen +5Robotic Grasping Robot Foundation Models
1Peking University · 2Tianjin University
LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation
Apr 9, 2026Jingjing Wang, Zhengdong Hong, Chong Bao +3Generalization in Robotic Manipulation Robotic Manipulation
State Key Lab of CAD&CG, Zhejiang University

Sep 29, 2026J/K move · Enter open · S save

Learning to Explore Hidden Kinematics for Articulated Object Manipulation

Ruiyao Liu, Boshu Lei, Zhuoyang Pan, Kostas Daniilidis

GRASP Lab, University of Pennsylvania, Philadelphia, PA 19104, USA

RL for Robotics Active Perception Active Vision for Robot Manipulation Robotics Articulated Object Manipulation Policy Learning+1

Open paper AskPDF arXiv

The kinematics of an articulated object is often ambiguous from vision alone. Interaction resolves the ambiguity, and active perception methods exploit this by searching for the single action that most sharpens a belief over the kinematic parameters at each step. Such greedy search cannot be extended over a horizon without forward models of the contact and inertial dynamics, which are themselves unknown. We instead amortize action selection into training. We maintain a belief distribution over joint type and parameters, initialized from a generative prior and updated by Bayesian filtering on the observed part motion. To condition the policy on this belief, we render it as a per-point articulation flow field, the motion that the current posterior predicts for every point on the object. Carrying the inductive bias of articulated motion, this representation generalizes better than a latent encoding of the belief or flow tracked from observation. We train the policy with reinforcement learning, rewarding the entropy that each interaction removes from the posterior, so that informative exploration becomes learned behavior rather than a search at every step. Our method outperforms previous approaches across door and drawer manipulation on the PartManip benchmark, and reaches 61.7% success on ArticuRiddle, a new dataset of objects whose appearance implies the wrong articulation, against 44.4% for the best previous method. Project Website: https://hiddenkinematics.github.io/

Abstract

Articulated Object Manipulation

Explore similar work

Learning to Explore Hidden Kinematics for Articulated Object Manipulation

HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions

LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation