Learning to Explore Hidden Kinematics for Articulated Object Manipulation

Ruiyao Liu; Boshu Lei; Zhuoyang Pan; Kostas Daniilidis

cs.ROSep 29, 2026

Learning to Explore Hidden Kinematics for Articulated Object Manipulation

Authors: Ruiyao Liu, Boshu Lei, Zhuoyang Pan, Kostas Daniilidis

Organizations: GRASP Lab, University of Pennsylvania, Philadelphia, PA 19104, USA

Abstract

The kinematics of an articulated object is often ambiguous from vision alone. Interaction resolves the ambiguity, and active perception methods exploit this by searching for the single action that most sharpens a belief over the kinematic parameters at each step. Such greedy search cannot be extended over a horizon without forward models of the contact and inertial dynamics, which are themselves unknown. We instead amortize action selection into training. We maintain a belief distribution over joint type and parameters, initialized from a generative prior and updated by Bayesian filtering on the observed part motion. To condition the policy on this belief, we render it as a per-point articulation flow field, the motion that the current posterior predicts for every point on the object. Carrying the inductive bias of articulated motion, this representation generalizes better than a latent encoding of the belief or flow tracked from observation. We train the policy with reinforcement learning, rewarding the entropy that each interaction removes from the posterior, so that informative exploration becomes learned behavior rather than a search at every step. Our method outperforms previous approaches across door and drawer manipulation on the PartManip benchmark, and reaches 61.7% success on ArticuRiddle, a new dataset of objects whose appearance implies the wrong articulation, against 44.4% for the best previous method. Project Website: https://hiddenkinematics.github.io/

Figures & tables

Fig. 1: Visual appearance can be misleading. We change a drawer’s joint from prismatic to revolute without altering its appearance. Both methods open the original drawer, but on the modified one the baseline is misled by the geometry, whereas ours refines its belief through interaction and adapts.

Fig. 2: Pipeline Overview. Given the initial observation, we use a generative prior to initialize a set of particles, where each particle represents a hypothesis of the object’s articulation model. At each interaction step, each hypothesis is converted into a per-point flow field. These flow fields are weighted-averaged over the particles. The flow field is first concatenated with the current point cloud as $O_{t}^{\prime}$ and then fused with the encoded robot’s proprioceptive state as input to the actor. The predicted action is executed in the environment, and the observed motion is used to update the particle weights through a Bayes filter. The information reward, defined as the reduction in belief entropy from before to after the update, is combined with the task reward to train the policy using PPO.

Fig. 3: Flow field updates during interaction at time $t$ . As the robot interacts with the object, hypotheses inconsistent with the observed motion are suppressed and the posterior gradually concentrates. The flow is updated accordingly, providing the policy with increasingly accurate motion cues.

Fig. 4: Overview of ArticuRiddle. We construct challenging objects through (a) handle-pose edits and (b) joint-parameter edits that preserve appearance, both testing manipulation under ambiguous geometric cues.

Category	Method	Train			Val			ArticuRiddle
Category	Method	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$
Door	AKM [ 4 ]	79.0 $\pm$ 1.9	48.2 $\pm$ 1.3	55.9 $\pm$ 2.5	84.1 $\pm$ 1.6	29.6 $\pm$ 5.6	33.3 $\pm$ 5.9	64.0 $\pm$ 3.5	30.7 $\pm$ 6.1	32.7 $\pm$ 5.8
	H-SAUR [ 3 ]	81.7 $\pm$ 1.9	40.3 $\pm$ 3.6	46.6 $\pm$ 4.6	85.2 $\pm$ 2.7	35.1 $\pm$ 5.4	43.2 $\pm$ 8.1	68.1 $\pm$ 1.9	39.0 $\pm$ 2.9	50.3 $\pm$ 2.9
	Ours	82.7 $\bm{\pm}$ 1.3	62.4 $\bm{\pm}$ 1.3	72.7 $\bm{\pm}$ 2.1	87.8 $\bm{\pm}$ 2.2	59.2 $\bm{\pm}$ 5.1	70.1 $\bm{\pm}$ 3.0	78.4 $\bm{\pm}$ 5.2	57.7 $\bm{\pm}$ 5.6	65.4 $\bm{\pm}$ 3.3
Drawer	AKM [ 4 ]	75.9 $\pm$ 1.6	42.1 $\pm$ 1.9	49.7 $\pm$ 3.8	79.2 $\pm$ 4.4	41.5 $\pm$ 4.0	52.5 $\pm$ 6.5	54.1 $\pm$ 4.7	35.8 $\pm$ 2.2	41.7 $\pm$ 1.8
	H-SAUR [ 3 ]	73.0 $\pm$ 2.0	43.0 $\pm$ 2.3	49.7 $\pm$ 0.4	73.5 $\pm$ 1.9	38.1 $\pm$ 0.3	54.3 $\pm$ 7.1	59.0 $\pm$ 1.3	29.0 $\pm$ 5.5	36.2 $\pm$ 3.3

TABLE I: Articulation estimation on PartManip and ArticuRiddle.

Method	Door		Drawer		ArticuRiddle
Method	Train	Val	Train	Val	ArticuRiddle
H-SAUR [ 3 ]	21.8 $\pm$ 1.2	27.0 $\pm$ 1.3	33.5 $\pm$ 1.8	32.7 $\pm$ 1.5	44.4 $\pm$ 2.2
AKM [ 4 ]	25.3 $\pm$ 2.0	19.0 $\pm$ 1.6	26.7 $\pm$ 1.1	22.6 $\pm$ 1.4	27.0 $\pm$ 2.5
Wang et al. [ 35 ]	37.4 $\pm$ 2.5	39.7 $\pm$ 5.4	44.2 $\pm$ 3.3	27.8 $\pm$ 10.6	18.2 $\pm$ 3.9
PartManip [ 10 ]	44.5 $\pm$ 0.7	40.6 $\pm$ 2.7	67.1 $\pm$ 2.6	74.3 $\pm$ 4.3	32.4 $\pm$ 1.4
Obs-flow	59.4 $\pm$ 1.9	43.6 $\pm$ 3.6	76.7 $\pm$ 1.8	88.0 $\pm$ 1.1	53.8 $\pm$ 3.4

TABLE II: Manipulation success rate.

Fig. 5: Reward alignment and opening progress on the training set. Top : episodes collected with our trained policies are scored by three different rewards; bars give the fraction with a final axis error above $10^{\circ}$ in the lowest- (hatched) and highest-scoring (solid) quarter. The results suggest optimizing the entropy-reduction return may lead to more accurate estimates. Bottom : median opening over assets for the policy trained with each reward; the dashed line marks the success threshold. Error bars and bands are bootstrapped $95\%$ confidence intervals, over episodes (top) and over assets (bottom).

Variant	Door		Drawer		ArticuRiddle
Variant	Train	Val	Train	Val	ArticuRiddle
w/o RL	57.6 $\pm$ 1.2	40.2 $\pm$ 2.4	82.7 $\pm$ 3.0	87.7 $\pm$ 1.0	48.1 $\pm$ 1.1
$\delta q$	63.9 $\pm$ 1.5	42.9 $\pm$ 4.2	84.4 $\pm$ 1.0	90.4 $\pm$ 2.3	55.2 $\pm$ 2.8
$D_{\mathrm{KL}}$	64.4 $\pm$ 1.1	42.9 $\pm$ 3.2	87.1 $\pm$ 3.9	93.9 $\pm$ 1.3	58.6 $\pm$ 4.6
$\Delta H$ (Ours)	68.2 $\bm{\pm}$ 1.3	51.3 $\bm{\pm}$ 1.9	89.0 $\bm{\pm}$ 3.0	95.4 $\bm{\pm}$ 1.5	61.7 $\bm{\pm}$ 1.1

TABLE III: Reward ablation on manipulation success rate.

Fig. 6: Real-world manipulation examples. Each row shows the object followed by point-cloud visualizations at successive interaction steps. The rendered flow field shows how observed part motion drives the articulation belief to converge toward the true joint.

Object	Axis ( ∘ ) $\downarrow$	Pivot (cm) $\downarrow$	SR (%) $\uparrow$
Door 1	2.5	2.2	60.0
Door 2	3.1	2.6	70.0
Drawer 1	1.7	–	90.0
Drawer 2	3.6	–	80.0

TABLE IV: Real-world articulation estimation errors and manipulation success rates. A trial succeeds when the door opens by more than

30^{\circ}

or the drawer by more than

20\%

of its opening range.

Explore similar work

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation
Jul 27, 2026Yaping Li, Zhaxizhuoma, Qiaojun Yu +3Articulated Kinematics Articulated Objects
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University
PokeNet: Learning Kinematic Models of Articulated Objects from Human Observations
Feb 2, 2026Anmol Gupta, Weiwei Gu, Omkar Patil +2Articulated Kinematics Articulated Objects
School of Computation and AI, ASU, Tempe · AI Institute, Seoul National University, Seoul, South Korea
Query-Conditioned Articulation Estimation from a Single Image
Oct 1, 2026Abdelrhman Werby, Fabio Scaparro, Kai O. ArraCategory-Level Object Pose Estimation Articulated Kinematics
Socially Intelligent Robotics Lab, Institute for Artificial Intelligence, University of Stuttgart, Germany

Jul 27, 2026, cs.ROJ/K move · Enter open · S save

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation

Yaping Li, Zhaxizhuoma, Qiaojun Yu, Jia Zeng+2

The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University

Articulated Kinematics Articulated Objects Kinematics

Open paper AskPDF arXiv

Articulated object manipulation requires an understanding of kinematic structure that is difficult and costly to learn from robot demonstrations alone. We introduce the Kinematic-Aware Articulation Interface (KAI), a structured intermediate representation that captures the kinematic structure of articulated objects. By embedding interpretable geometric and kinematic priors into policy learning, KAI provides a strong inductive bias aligned with the underlying structure of articulated motion. This design effectively improves sample efficiency, with gains particularly pronounced in low-data regimes: across six simulation tasks, our method achieves an average success rate of 82.9%, matching or surpassing baseline performance while using only half the demonstration data. Our method also exhibits robust generalization to unseen backgrounds and visual distractors, transferring from a single clean training environment to cluttered real-world scenes. KAI's action-agnostic design further enables co-training with human interaction videos to enhance real-world robustness: under diverse visual distractions, our method with video co-training achieves over 70% average success rate.

Abstract

Explore similar work

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation

PokeNet: Learning Kinematic Models of Articulated Objects from Human Observations

Query-Conditioned Articulation Estimation from a Single Image