cs.CVNov 17, 2026

PhyVGGT: Feed-Forward Articulated Object Reconstruction from Monocular Videos

Authors: Qi Shao, Zhihao Cao, Shuhao Zhai, Anh Nguyen, Hong Liang, Baoru Huang, Liang Li

Organizations: Harbin Engineering University, Harbin, China · University of Liverpool, Liverpool, U.K. · ETH Zurich, Zurich, Switzerland · National University of Singapore, Singapore · 

Abstract

Reconstructing an articulated object with its 3D movable parts, joint parameters and unseen articulation states from an uncalibrated monocular video is essential for robot manipulation and interactive digital twins. Existing per-scene optimisation methods require minutes to hours and external depth or camera poses, whereas feed-forward 3D foundation models reconstruct dense point maps in one pass without segmenting movable parts or inferring joints. Even on these monocular point maps, regressing or fitting continuous joints directly fails on casual videos, since cross-frame attention blurs moving part boundaries in deep layers, underestimates motion along the viewing ray and leaves continuous axes ill-conditioned under small opening angles. To address these challenges, we propose PhyVGGT, a single-pass framework on a frozen 3D foundation model that anchors moving-part matching and closed-form joint estimation to the undistorted geometry of the stationary base. First, our Part and Motion Decoder fuses intermediate per-frame tokens with global tokens and references part queries to a static-base query to segment 3D parts, align point maps on the static base and predict learned Bingham priors. Second, our Kinematic Inference Layer fits an orthogonal Manhattan frame to static surface normals, selects the joint axis in closed form under a Bingham posterior combining pairwise motion scatter with the learned priors, and solves a linear system for the hinge pivot to drive unseen states. Trained solely on synthetic data, PhyVGGT achieves state-of-the-art monocular performance across five benchmarks, outperforming FAMOS and monocular ArtiPoint, surpassing posed RGB-D ArtiPoint in median error, and running in 6.8 s per 48-frame video against hours of per-scene optimisation in Articulat3D. Project page: https://phyvggt.github.io.

Figures & tables

Explore similar work

CardsList
  1. MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstruction

    Mar 19, 2026Haitian Li, Haozhe Xie, Junxiang Xu +3Articulated Object ReconstructionMonocular 3D Reconstruction

  2. Track, Articulate, Act: Generating Articulation from Casual Human Videos

    Sep 16, 2026Jiaming Zhang, Homanga BharadhwajArticulated Object ReconstructionArticulated Object Manipulation

  3. Track2Art: Articulated Object Model Recovery with Visual-Geometric Track Representations

    Sep 23, 2026Xiaotong Li, Yixiong Jing, Junsheng Ding +4Articulated Object Reconstruction3D Part Segmentation