ViewStruct: View-Grounded Articulation and Connectivity Inference

Wenjing Bian; Ranran Huang; Andreas Geiger

cs.CVOct 12, 2026

ViewStruct: View-Grounded Articulation and Connectivity Inference

Authors: Wenjing Bian, Ranran Huang, Andreas Geiger

Organizations: University of Tübingen, Tübingen AI Center · Imperial College London

Abstract

Interacting with unfamiliar objects requires understanding how their parts connect and move. We introduce ViewStruct, a framework for jointly predicting connectivity and articulation in objects and scenes. Given an RGB image with aligned part segmentation and depth, optionally supplemented by images or video, ViewStruct predicts rigid-body groupings, attachments, and joint parameters. Its view-grounded representation links predictions to observed regions and expresses joint geometry in a gravity-aligned, view-relative frame, without requiring category-specific canonical orientations. A pretrained vision--language model is adapted to predict structure in seconds, without requiring observed articulation motion or per-instance optimisation. Experiments demonstrate improved structural prediction over evaluated baselines and transfer to unseen categories, asset sources, and real-world human-interaction videos, despite task-specific training exclusively on synthetic data. Simulated manipulation experiments further demonstrate the utility of predicted joints for generating robot motion trajectories.

Figures & tables

Figure 1: ViewStruct predicts part connectivity and articulation from visual observations. Given a reference RGB image with aligned part segmentation and depth, optionally supplemented by images or video, it predicts structure in seconds. Part identifiers associate predictions with observed regions, while joint geometry is expressed in a gravity-aligned, view-relative frame. Examples show object- and scene-level predictions, transfer to real-world interactions, and predicted trajectories used for simulated manipulation.

Figure 2: Overview of ViewStruct. Given a reference RGB image, aligned segmentation and depth maps, and a text prompt specifying an indexed part inventory, ViewStruct uses a VLM to predict part connectivity and articulation parameters in the reference-view frame. Additional views or video can provide complementary evidence, but observed articulation motion is not required.

Figure 3: Qualitative comparison on PhysX-Mobility. Predicted motion is overlaid on the input geometry: hinges in red, slides in blue, and axes in yellow. Dimmed panels indicate missing outputs.

Method	Parent $\uparrow$	Edge $F_{1}\uparrow$	Joint $F_{1}\uparrow$	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Range $\uparrow$	matched	all joints	objects	Time (s) $\downarrow$
	Connectivity		Articulation					Success $\uparrow$
Qwen3-VL-4B (zero-shot)	19.2	19.9	31.5	66.0	41.2	0.391	30.5	1.1	0.2	1.0	–
PhysX-Anything	61.0	67.9	85.9	96.2	94.4	0.118	26.3	81.6	64.3	40.2	267
PhysX-Omni	70.1	73.6	85.9	97.8	95.8	0.096	26.5	85.3	66.6	38.9	241
URDF-Anything+	17.3	14.7	18.6	73.0	58.5	0.269	34.1	14.0	1.5	2.3	64
ViewStruct	92.3	92.7	96.8	96.6	89.9	0.084	72.3	81.5	78.2	52.5	6.5

Table 1: Structure prediction on PhysX-Mobility. All methods receive a single reference image. ViewStruct additionally uses part segmentation and depth, either supplied or estimated by MoGe in the last row; zero-shot Qwen3-VL receives the same inputs. PhysX-Anything and PhysX-Omni use only the image, while URDF-Anything+ also receives a whole-object mesh. Comparisons with SINGAPO and Particulate appear in Table 6 . Success is measured over matched joints, over all reference joints (missed joints counted as failures), and per object (the fraction of objects whose reference joints are all recovered correctly; an object with many keys or buttons counts once). Pivot errors are in normalised asset units, inference times in seconds, and all other metrics in percentages. Evaluation protocols are detailed in Appendix B .

Method	Parse $\uparrow$	Recall $\uparrow$	Joint $F_{1}\uparrow$	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Range $\uparrow$	Success $\uparrow$	Success-all $\uparrow$
OPDMulti	100.0	15.7	25.8	73.9	49.4	0.440	50.0	11.8	1.8
PhysX-Omni	100.0	30.3	40.4	66.7	49.7	0.400	8.1	24.4	7.4
ViewStruct	98.9	94.7	95.2	77.6	62.1	0.331	72.2	26.2	24.8
$+2$ views with parts moved	97.7	94.8	95.2	80.5	64.4	0.337	74.1	28.0	26.6

Table 2: Scene-level articulation prediction on GRScenes. Pivot thresholds use per-object normalisation. Success over matched and all reference joints distinguishes parameter accuracy from coverage. Pivot error is in asset units; other entries are percentages. The last row uses the same model and reference views with two additional room renderings showing moved parts, one from the reference viewpoint and one from another; other rows use a single view. Parent and edge scores are omitted because the predominantly static attachment structure allows a prediction with no joints to score

99.9\%

( Section B.3 ).

SimReady objects ( $242$ )
Method	Joint $F_{1}\uparrow$	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Range $\uparrow$	Success $\uparrow$	Success-all $\uparrow$
Qwen3-VL-4B (zero-shot)	17.9	46.7	39.3	0.382	24.6	0.0	0.0
PhysX-Anything	61.3	62.0	48.7	0.327	15.9	14.9	6.9
URDF-Anything+	33.0	65.7	52.1	0.239	31.1	2.8	0.6
PhysX-Omni	61.1	62.8	52.6	0.380	18.1	18.0	8.2
ViewStruct	95.0	75.0	67.1	0.115	31.5	33.3	31.5

Table 3: Object-level generalisation to SimReady and held-out GRScenes categories. Metrics, units and columns follow Table 1 : success is measured over matched joints and over all reference joints (missed joints counted as failures).

		Type acc. $\uparrow$		Axis error $\downarrow$		Hinge pos. $\downarrow$	$n$
Method	Visual input	prism.	revol.	prism.	revol.	(m)
ArtiPoint	Full RGB-D sequence	75.3	97.3	20.5 ∘ ( 4.4 ∘ )	20.9 ∘ (10.3 ∘ )	0.19 ( 0.12 )	338
OPDMulti	Initial RGB only	52.2	72.0	44.3 ∘ (32.5 ∘ )	42.8 ∘ (30.9 ∘ )	0.42 (0.37)	307
ViewStruct	Initial RGB-D only	69.3	77.2	36.7 ∘ (17.0 ∘ )	37.6 ∘ (4.0 ∘ )	0.31 (0.22)	342
ViewStruct	Initial RGB-D + 2 frames	81.5	91.2	27.3 ∘ (14.2 ∘ )	27.0 ∘ (1.8 ∘ )	0.27 (0.18)	341
ViewStruct	Initial RGB-D + 2 frames + clip	86.0	93.0	28.6 ∘ (15.8 ∘ )	21.3 ∘ ( 1.6 ∘ )	0.23 (0.18)	342

Table 4: Evaluation on Arti4D real-world videos. All ViewStruct variants receive an initial RGB-D observation with Mobile-SAM regions; additional RGB frames and clips provide motion evidence without camera poses. ArtiPoint uses the full RGB-D sequence, camera poses, and reference interaction cues; OPDMulti uses the initial RGB image alone and returns no detection for

35

interactions. Metrics follow Werby et al. (2025) : type accuracy (%) for each reference joint type, undirected axis error in degrees, and hinge-position error as the common-perpendicular distance in metres. Errors are reported as means, with medians in parentheses;

n

denotes the number of evaluated outputs.

Method	Attempted	Opens	+ retry
GT joints (oracle)	100.0	99.9	99.9
PhysX-Anything	57.2	50.3	52.3
URDF-Anything+	11.3	4.4	4.9
PhysX-Omni	56.5	48.9	51.2
ViewStruct	91.3	74.6	80.3

Table 5: Compliant manipulation on PhysX-Mobility. Success (%) over all reference joints; missing predictions count as failures. “Attempted” reports prediction coverage, “

+

retry” allows one reversal of the motion direction. Per-joint-type results are in Table 12 .

Figure 6: Panda manipulation in GRScenes. The arm follows ViewStruct predictions while objects retain reference mechanisms. Cyan arrows indicate sliding; yellow bars mark hinge axes.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Hinge-pivot equivalence. Displacement along a shared axis leaves the rotation unchanged; the scoring convention is given in Equation 8 .

Structure Joint Parse

\uparrow

Parent

\uparrow

Edge

F_{1}\uparrow

Pres.

F_{1}\uparrow

Type

\uparrow

Axis

\uparrow

Pos.

\downarrow

Range IoU

\uparrow

Qwen3-VL-4B (zero-shot) 81.4 19.2 19.9 31.5 66.0 41.2 0.391 30.5 PhysX-Anything 100.0 61.0 67.9 85.9 96.2 94.4 0.118 26.3 PhysX-Omni 100.0 70.1 73.6 85.9 97.8 95.8 0.096 26.5 URDF-Anything+ 99.2 17.3 14.7 18.6 73.0 58.5 0.269 34.1 Particulate ¶ 100.0 37.8 45.3 61.0 97.5 94.8 0.036 54.1 ViewStruct (PhysX only) 98.7 91.1 91.4 97.2 96.2 86.5 0.087 69.8 ViewStruct 99.7 92.3 92.7 96.8 96.6 89.9 0.084 72.3 ViewStruct (MoGe depth) 99.5 92.3 91.0 96.5 95.6 89.3 0.094 69.1 StorageFurniture subset (

65

objects; the only category shared with SINGAPO’s taxonomy) SINGAPO † 100.0 29.8 30.4 51.2 84.1 86.6 0.569 45.5 PhysX-Anything 100.0 38.4 26.8 79.8 90.2 79.5 0.190 18.5 Particulate ¶ 100.0 32.7 27.8 96.4 100.0 100.0 0.014 93.5 ViewStruct 100.0 87.8 86.4 98.2 98.2 96.3 0.036 82.7

Articulation failure attribution Type

\downarrow

Axis

\downarrow

Origin

\downarrow

Limit

\downarrow

Success

\uparrow

Succ. obj.

\uparrow

Qwen3-VL-4B (zero-shot) 34.0 35.6 28.7 0.6 1.1 1.0 PhysX-Anything 3.8 3.0 10.7 0.9 81.6 40.2 PhysX-Omni 2.2 3.1 9.1 0.3 85.3 38.9 URDF-Anything+ 27.0 25.0 33.0 1.0 14.0 2.3 Particulate ¶ 2.5 3.9 4.5 20.9 68.2 51.0 ViewStruct (PhysX only) 3.8 11.1 8.1 0.6 76.5 53.0 ViewStruct 3.4 8.1 6.2 0.8 81.5 52.5 ViewStruct (MoGe depth) 4.4 8.0 8.6 0.7 78.3 48.7

Appendix

Table 6: Detailed PhysX-Mobility comparison. Full-set and StorageFurniture-subset diagnostics. † SINGAPO:

52/65

training-overlap objects; ¶ Particulate:

81/387

. Scores are percentages except normalised pivot error. Part-matching conventions are described in the text.

Method	Parent $\uparrow$	Edge $F_{1}\uparrow$	Joint $F_{1}\uparrow$	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Range $\uparrow$	Success $\uparrow$	Succ. obj. $\uparrow$
Qwen3-VL-4B (zero-shot)	56.0	52.3	55.3	69.0	42.9	0.352	37.2	0.0	0.0
URDF-Anything+	48.6	47.5	46.5	74.5	47.0	0.443	37.0	0.5	0.4
Particulate	82.9	81.2	87.0	88.1	80.1	0.150	79.9	63.7	57.2
ViewStruct	99.7	99.9	99.8	93.2	88.6	0.100	86.1	70.6	66.3

Appendix

Table 7: GRScenes objects from seen categories (

306

test instances). Single-view results; percentages except normalised pivot error.

Gravity error	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Success $\uparrow$	Succ. obj. $\uparrow$
none	96.6	89.9	0.084	81.5	52.5
$1^{\circ}$	96.6	90.0	0.084	81.6	52.5
$2^{\circ}$	96.6	90.0	0.084	81.5	52.2
$5^{\circ}$	96.6	88.5	0.087	79.8	52.5
$10^{\circ}$	96.6	82.5	0.096	70.7	45.7
$20^{\circ}$	96.6	29.8	0.120	25.6	8.3

Appendix

Table 8: Cost of a wrong gravity direction (PhysX-Mobility,

387

objects, single view). The reported predictions rescored as if the assumed gravity were tilted by

\delta

about a random horizontal axis.

Recipe	Parent $\uparrow$	Edge $F_{1}\uparrow$	Joint $F_{1}\uparrow$	Type $\uparrow$	Axis $\uparrow$	Pivot $\downarrow$	Range $\uparrow$	Success $\uparrow$	Succ. obj. $\uparrow$
ViewStruct	92.3	92.7	96.8	96.6	89.9	0.084	72.3	81.5	52.5
camera frame instead of gravity-aligned	91.1	90.6	96.9	96.1	86.6	0.082	65.7	77.6	54.5
without decomposition augmentation	88.6	88.4	96.2	94.8	82.8	0.096	66.3	71.5	50.4

Appendix

Table 9: Ablations on PhysX-Mobility (

387

objects, single view, strict origin metric). Each row retrains the reported recipe with one change. Metrics and units follow Table 1 .

Structure
	Qwen3-VL-4B	Qwen3-VL-8B	ViewStruct
	(zero-shot)	(zero-shot)	(single view)
parse success $\uparrow$	81.4%	100.0%	99.7%
parent acc (canon) $\uparrow$	19.2%	38.7%	92.3%
parent acc (rerooted) $\uparrow$	22.3%	29.3%	92.6%
edge $F_{1}\uparrow$	19.9%	27.3%	92.7%

Appendix

Table 10: Backbone scaling versus adaptation. Zero-shot Qwen3-VL and ViewStruct on

387

PhysX-Mobility objects with identical input and output formats.

	Success $\uparrow$		Success obj. $\uparrow$		Axis $\uparrow$		Origin fail $\downarrow$
	all	visible	all	visible	all	visible	all	visible
Qwen3-VL-4B (zero-shot)	1.1	1.2	1.0	1.0	41.2	41.7	28.7	30.1
PhysX-Anything	81.6	82.1	40.2	36.9	94.4	94.5	10.7	10.5
PhysX-Omni	85.3	86.0	38.9	35.3	95.8	96.0	9.1	9.0
URDF-Anything+	14.0	11.4	2.3	1.8	58.5	57.7	33.0	34.9
Particulate (GT mesh)	68.2	68.9	51.0	46.3	94.8	95.4	4.5	5.0

Appendix

Table 11: Visibility diagnostic on PhysX-Mobility. All-part and visible-part scores (%). The video row uses a separate clip set; the URDF-Anything+ export differs from the main comparison, as detailed in the text.

Figure 8: Articulation predictions on SimReady. The image baselines of Figure 3 on an unseen asset source. Red: hinged motion; blue: sliding motion; yellow: axes. Dimmed panels indicate missing outputs.

Figure 9: Additional GRScenes scene-view predictions. Reference, PhysX-Omni, OPDMulti and ViewStruct on the same views, with the same panels as Figure 5 . Each method’s own joints move the same reference meshes.

		Open as predicted			With one retry
Method	Attempted	All	Hinge	Slide	All	Hinge	Slide
GT joints (oracle)	100.0	99.9	100.0	99.9	99.9	100.0	99.9
PhysX-Anything	57.2	50.3	52.1	49.6	52.3	58.0	49.9
URDF-Anything+	11.3	4.4	12.8	1.0	4.9	14.0	1.2
PhysX-Omni	56.5	48.9	51.6	47.8	51.2	58.0	48.4
ViewStruct	91.3	74.6	56.7	81.8	80.3	74.1	82.7

Appendix

Table 12: Compliant manipulation on PhysX-Mobility. Opening success (%) over all reference joints, including missing predictions as failures. “Attempted” denotes prediction coverage (%). One retry reverses the trajectory direction.

Figure 10: Additional Panda manipulation trials. Four successful openings and two failures under reference mechanisms with assisted grasps. Outcomes are discussed in the text.

Articulated Object Reconstruction

Articulated object reconstruction recovers 3D part geometry, joints, and motion from images, video, or depth for digital twins and robot interaction.

See all 60 papers

DailyArt: Discovering Articulation from Single Static Images via Latent Dynamics
Apr 9, 2026Hang Zhang, Qijian Tian, Jingyu Gong +4Articulated Object Reconstruction
East China Normal University · Shanghai AI Lab · Shanghai Jiao Tong University +1
Track, Articulate, Act: Generating Articulation from Casual Human Videos
Sep 16, 2026Jiaming Zhang, Homanga BharadhwajArticulated Object Reconstruction Articulated Object Manipulation
Department of Computer Science, Johns Hopkins University
Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control
Jun 12, 2026Ruining Li, Yuxin Yao, Matt Zhou +5Articulated Object Reconstruction 3D Part Segmentation
University of Oxford · University of Cambridge · 3Nanyang Technological University

Apr 9, 2026J/K move · Enter open · S save

DailyArt: Discovering Articulation from Single Static Images via Latent Dynamics

Hang Zhang, Qijian Tian, Jingyu Gong, Daoguo Dong+3

East China Normal University · Shanghai AI Lab · Shanghai Jiao Tong University+1

Articulated Object Reconstruction

Open paper AskPDF arXiv

Articulated objects are essential for embodied AI and world models, yet inferring their kinematics from a single closed-state image remains challenging because crucial motion cues are often occluded. Existing methods either require multi-state observations or rely on explicit part priors, retrieval, or other auxiliary inputs that partially expose the structure to be inferred. In this work, we present DailyArt, which formulates articulated joint estimation from a single static image as a synthesis-mediated reasoning problem. Instead of directly regressing joints from a heavily occluded observation, DailyArt first synthesizes a maximally articulated opened state under the same camera view to expose articulation cues, and then estimates the full set of joint parameters from the discrepancy between the observed and synthesized states. Using a set-prediction formulation, DailyArt recovers all joints simultaneously without requiring object-specific templates, multi-view inputs, or explicit part annotations at test time. Taking estimated joints as conditions, the framework further supports part-level novel state synthesis as a downstream capability. Extensive experiments show that DailyArt achieves strong performance in articulated joint estimation and supports part-level novel state synthesis conditioned on joints. Project page is available at https://rangooo123.github.io/DaliyArt.github.io/.

Abstract

Articulated Object Reconstruction

Explore similar work

DailyArt: Discovering Articulation from Single Static Images via Latent Dynamics

Track, Articulate, Act: Generating Articulation from Casual Human Videos

Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control