Learning to Explore Hidden Kinematics for Articulated Object Manipulation

Ruiyao Liu; Boshu Lei; Zhuoyang Pan; Kostas Daniilidis

cs.ROSep 29, 2026

Learning to Explore Hidden Kinematics for Articulated Object Manipulation

Authors: Ruiyao Liu, Boshu Lei, Zhuoyang Pan, Kostas Daniilidis

Organizations: GRASP Lab, University of Pennsylvania, Philadelphia, PA 19104, USA

Abstract

The kinematics of an articulated object is often ambiguous from vision alone. Interaction resolves the ambiguity, and active perception methods exploit this by searching for the single action that most sharpens a belief over the kinematic parameters at each step. Such greedy search cannot be extended over a horizon without forward models of the contact and inertial dynamics, which are themselves unknown. We instead amortize action selection into training. We maintain a belief distribution over joint type and parameters, initialized from a generative prior and updated by Bayesian filtering on the observed part motion. To condition the policy on this belief, we render it as a per-point articulation flow field, the motion that the current posterior predicts for every point on the object. Carrying the inductive bias of articulated motion, this representation generalizes better than a latent encoding of the belief or flow tracked from observation. We train the policy with reinforcement learning, rewarding the entropy that each interaction removes from the posterior, so that informative exploration becomes learned behavior rather than a search at every step. Our method outperforms previous approaches across door and drawer manipulation on the PartManip benchmark, and reaches 61.7% success on ArticuRiddle, a new dataset of objects whose appearance implies the wrong articulation, against 44.4% for the best previous method. Project Website: https://hiddenkinematics.github.io/

Figures & tables

Fig. 1: Visual appearance can be misleading. We change a drawer’s joint from prismatic to revolute without altering its appearance. Both methods open the original drawer, but on the modified one the baseline is misled by the geometry, whereas ours refines its belief through interaction and adapts.

Fig. 2: Pipeline Overview. Given the initial observation, we use a generative prior to initialize a set of particles, where each particle represents a hypothesis of the object’s articulation model. At each interaction step, each hypothesis is converted into a per-point flow field. These flow fields are weighted-averaged over the particles. The flow field is first concatenated with the current point cloud as $O_{t}^{\prime}$ and then fused with the encoded robot’s proprioceptive state as input to the actor. The predicted action is executed in the environment, and the observed motion is used to update the particle weights through a Bayes filter. The information reward, defined as the reduction in belief entropy from before to after the update, is combined with the task reward to train the policy using PPO.

Fig. 3: Flow field updates during interaction at time $t$ . As the robot interacts with the object, hypotheses inconsistent with the observed motion are suppressed and the posterior gradually concentrates. The flow is updated accordingly, providing the policy with increasingly accurate motion cues.

Fig. 4: Overview of ArticuRiddle. We construct challenging objects through (a) handle-pose edits and (b) joint-parameter edits that preserve appearance, both testing manipulation under ambiguous geometric cues.

Category	Method	Train			Val			ArticuRiddle
Category	Method	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$	Acc. $\uparrow$	ASR ${}_{10}\uparrow$	ASR ${}_{20}\uparrow$
Door	AKM [ 4 ]	79.0 $\pm$ 1.9	48.2 $\pm$ 1.3	55.9 $\pm$ 2.5	84.1 $\pm$ 1.6	29.6 $\pm$ 5.6	33.3 $\pm$ 5.9	64.0 $\pm$ 3.5	30.7 $\pm$ 6.1	32.7 $\pm$ 5.8
	H-SAUR [ 3 ]	81.7 $\pm$ 1.9	40.3 $\pm$ 3.6	46.6 $\pm$ 4.6	85.2 $\pm$ 2.7	35.1 $\pm$ 5.4	43.2 $\pm$ 8.1	68.1 $\pm$ 1.9	39.0 $\pm$ 2.9	50.3 $\pm$ 2.9
	Ours	82.7 $\bm{\pm}$ 1.3	62.4 $\bm{\pm}$ 1.3	72.7 $\bm{\pm}$ 2.1	87.8 $\bm{\pm}$ 2.2	59.2 $\bm{\pm}$ 5.1	70.1 $\bm{\pm}$ 3.0	78.4 $\bm{\pm}$ 5.2	57.7 $\bm{\pm}$ 5.6	65.4 $\bm{\pm}$ 3.3
Drawer	AKM [ 4 ]	75.9 $\pm$ 1.6	42.1 $\pm$ 1.9	49.7 $\pm$ 3.8	79.2 $\pm$ 4.4	41.5 $\pm$ 4.0	52.5 $\pm$ 6.5	54.1 $\pm$ 4.7	35.8 $\pm$ 2.2	41.7 $\pm$ 1.8
	H-SAUR [ 3 ]	73.0 $\pm$ 2.0	43.0 $\pm$ 2.3	49.7 $\pm$ 0.4	73.5 $\pm$ 1.9	38.1 $\pm$ 0.3	54.3 $\pm$ 7.1	59.0 $\pm$ 1.3	29.0 $\pm$ 5.5	36.2 $\pm$ 3.3

TABLE I: Articulation estimation on PartManip and ArticuRiddle.

Method	Door		Drawer		ArticuRiddle
Method	Train	Val	Train	Val	ArticuRiddle
H-SAUR [ 3 ]	21.8 $\pm$ 1.2	27.0 $\pm$ 1.3	33.5 $\pm$ 1.8	32.7 $\pm$ 1.5	44.4 $\pm$ 2.2
AKM [ 4 ]	25.3 $\pm$ 2.0	19.0 $\pm$ 1.6	26.7 $\pm$ 1.1	22.6 $\pm$ 1.4	27.0 $\pm$ 2.5
Wang et al. [ 35 ]	37.4 $\pm$ 2.5	39.7 $\pm$ 5.4	44.2 $\pm$ 3.3	27.8 $\pm$ 10.6	18.2 $\pm$ 3.9
PartManip [ 10 ]	44.5 $\pm$ 0.7	40.6 $\pm$ 2.7	67.1 $\pm$ 2.6	74.3 $\pm$ 4.3	32.4 $\pm$ 1.4
Obs-flow	59.4 $\pm$ 1.9	43.6 $\pm$ 3.6	76.7 $\pm$ 1.8	88.0 $\pm$ 1.1	53.8 $\pm$ 3.4

TABLE II: Manipulation success rate.

Fig. 5: Reward alignment and opening progress on the training set. Top : episodes collected with our trained policies are scored by three different rewards; bars give the fraction with a final axis error above $10^{\circ}$ in the lowest- (hatched) and highest-scoring (solid) quarter. The results suggest optimizing the entropy-reduction return may lead to more accurate estimates. Bottom : median opening over assets for the policy trained with each reward; the dashed line marks the success threshold. Error bars and bands are bootstrapped $95\%$ confidence intervals, over episodes (top) and over assets (bottom).

Variant	Door		Drawer		ArticuRiddle
Variant	Train	Val	Train	Val	ArticuRiddle
w/o RL	57.6 $\pm$ 1.2	40.2 $\pm$ 2.4	82.7 $\pm$ 3.0	87.7 $\pm$ 1.0	48.1 $\pm$ 1.1
$\delta q$	63.9 $\pm$ 1.5	42.9 $\pm$ 4.2	84.4 $\pm$ 1.0	90.4 $\pm$ 2.3	55.2 $\pm$ 2.8
$D_{\mathrm{KL}}$	64.4 $\pm$ 1.1	42.9 $\pm$ 3.2	87.1 $\pm$ 3.9	93.9 $\pm$ 1.3	58.6 $\pm$ 4.6
$\Delta H$ (Ours)	68.2 $\bm{\pm}$ 1.3	51.3 $\bm{\pm}$ 1.9	89.0 $\bm{\pm}$ 3.0	95.4 $\bm{\pm}$ 1.5	61.7 $\bm{\pm}$ 1.1

TABLE III: Reward ablation on manipulation success rate.

Fig. 6: Real-world manipulation examples. Each row shows the object followed by point-cloud visualizations at successive interaction steps. The rendered flow field shows how observed part motion drives the articulation belief to converge toward the true joint.

Object	Axis ( ∘ ) $\downarrow$	Pivot (cm) $\downarrow$	SR (%) $\uparrow$
Door 1	2.5	2.2	60.0
Door 2	3.1	2.6	70.0
Drawer 1	1.7	–	90.0
Drawer 2	3.6	–	80.0

TABLE IV: Real-world articulation estimation errors and manipulation success rates. A trial succeeds when the door opens by more than

30^{\circ}

or the drawer by more than

20\%

of its opening range.

Explore similar work

Jul 27, 2026cs.RO

KAI: A Kinematic-Aware Interface for Data-Efficient Articulated Object Manipulation

Articulated object manipulation requires an understanding of kinematic structure that is difficult and costly to learn from robot demonstrations alone. We introduce the Kinematic-Aware Articulation Interface (KAI), a structured intermediate representation that captures the kinematic structure of articulated objects. By embedding interpretable geometric and kinematic priors into policy learning, KAI provides a strong inductive bias aligned with the underlying structure of articulated motion. This design effectively improves sample efficiency, with gains particularly pronounced in low-data regimes: across six simulation tasks, our method achieves an average success rate of 82.9%, matching or surpassing baseline performance while using only half the demonstration data. Our method also exhibits robust generalization to unseen backgrounds and visual distractors, transferring from a single clean training environment to cluttered real-world scenes. KAI's action-agnostic design further enables co-training with human interaction videos to enhance real-world robustness: under diverse visual distractions, our method with video co-training achieves over 70% average success rate.

Yaping Li, Zhaxizhuoma, Qiaojun Yu +3

The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University

Articulated Kinematics Articulated Objects Kinematics

Ask PDF

Feb 2, 2026cs.RO

PokeNet: Learning Kinematic Models of Articulated Objects from Human Observations

Articulation modeling enables robots to learn joint parameters of articulated objects for effective manipulation which can then be used downstream for skill learning or planning. Existing approaches often rely on prior knowledge about the objects, such as the number or type of joints. Some of these approaches also fail to recover occluded joints that are only revealed during interaction. Others require large numbers of multi-view images for every object, which is impractical in real-world settings. Furthermore, prior works neglect the order of manipulations, which is essential for many multi-DoF objects where one joint must be operated before another, such as a dishwasher. We introduce PokeNet, an end-to-end framework that estimates articulation models from a single human demonstration without prior object knowledge. Given a sequence of point cloud observations of a human manipulating an unknown object, PokeNet predicts joint parameters, infers manipulation order, and tracks joint states over time. PokeNet outperforms existing state-of-the-art methods, improving joint axis and state estimation accuracy by an average of over 27% across diverse objects, including novel and unseen categories. We demonstrate these gains in both simulation and real-world environments.

Anmol Gupta, Weiwei Gu, Omkar Patil +2

School of Computation and AI, ASU, Tempe · AI Institute, Seoul National University, Seoul, South Korea

Articulated Kinematics Articulated Objects Kinematics Human Agency

Ask PDF

Oct 1, 2026cs.RO

Query-Conditioned Articulation Estimation from a Single Image

Enabling robots to estimate the kinematic parameters of articulated objects unlocks a wide range of capabilities for interaction and manipulation. The estimation has to happen from the information the robot currently observes, often just a single RGB image of an object it has never seen before. Current single-image approaches couple articulation part segmentation with articulation estimation, making their predictions vulnerable to missed detections and incorrect part associations, and they regress metric 3D geometry that a single view fixes only up to scale. We present QueryArt, a model that estimates articulation parameters from a single RGB image, a 2D query point, and camera intrinsics. QueryArt is trained to estimate the 3D articulation geometry relative to the queried point and in units of its depth, which keeps its target identifiable from the image alone. A single depth measurement at the query point then supplies the scale and recovers the metric parameters. We train QueryArt on a curated mixture of synthetic and real-world articulation datasets. We evaluate QueryArt on several benchmarks and compare it against recent baselines. QueryArt outperforms recent baselines on most articulation metrics, including on out-of-distribution data. To demonstrate the model's capabilities in real-world settings, we evaluate QueryArt on a mobile manipulator across 57 manipulation trials spanning 16 object parts and five viewpoint classes, achieving a 70.2% success rate. We provide code and videos at: https://abwerby.github.io/queryart/

Abdelrhman Werby, Fabio Scaparro, Kai O. Arra

Socially Intelligent Robotics Lab, Institute for Artificial Intelligence, University of Stuttgart, Germany

Category-Level Object Pose Estimation Articulated Kinematics Art Robot Systems 3D Geometry

Ask PDF