FACT: Fidelity-Aware Construction of Articulated Twins

Kuixiang Shao; Chuansen Nie; Yinuo Bai; Jiayuan Gu; Jingyi Yu

cs.ROSep 29, 2026

FACT: Fidelity-Aware Construction of Articulated Twins

Authors: Kuixiang Shao, Chuansen Nie, Yinuo Bai, Jiayuan Gu, Jingyi Yu

Organizations: ShanghaiTech University Shanghai, China

Abstract

Visually plausible articulated assets may still fail during contact interactions or exhibit inaccurate motion. We present FACT (Fidelity-Aware Construction of Articulated Twins), an agentic framework that progressively constructs articulated twins to improve geometry, contact, and dynamic fidelity. The agent drives an evidence--diagnosis--revision loop on a shared editable representation, selecting measurements and model edits using quantitative feedback, while numerical tools execute and validate the updates. It reconstructs editable articulated geometry from images through feature planning, targeted measurements, and diagnostic refinement. On this reference, it repairs collision proxies through task-aware local repartitioning before fidelity-constrained compression. Finally, it constructs response models from passive-response videos, using simulation residuals to guide model revision and constrained physical parameter fitting. Experiments show that FACT improves geometric reconstruction over baselines, enables more reliable interaction with simpler collision proxies, and better reproduces held-out physical responses than direct parameter inference.

Figures & tables

Figure 1: Geometry, contact, and dynamics. FACT reconstructs editable articulated objects, refines collisions to preserve free space for interaction, and calibrates motion dynamics from video.

Figure 2: FACT overview. The agent drives an evidence–diagnosis–revision loop on a shared articulated representation. It plans geometric features and selects measurements, directs task-aware collision repair followed by fidelity-constrained compression, and constructs and revises response models while guiding constrained parameter fitting.

Image inputs
Method	CD $\downarrow$	SD-P95 $\downarrow$	F@1% $\uparrow$	F@2% $\uparrow$	Avg. SIoU $\uparrow$	Min. SIoU $\uparrow$
Codex (GPT-6)	1.1238	3.6060	0.6148	0.8237	0.8813	0.8310
Articraft (GPT-6)	1.3117	3.3719	0.5687	0.7615	0.8597	0.8068
FACT (GPT-6; original)	1.0821	2.7884	0.6838	0.8391	0.9381	0.9166
FACT (GPT-6; parts)	1.0317	2.9972	0.6937	0.8519	0.9317	0.9051
Rodin (original)	1.6840	1.9756	0.5663	0.7206	0.9393	0.9164

Table 1: Quantitative comparison of geometry fidelity. Gray rows show upstream Rodin meshes for reference; bold marks the best reconstruction results within each setting.

Figure 3: Geometry fidelity. FACT better preserves local structures and dimensions, while baselines can miss or distort fine details despite similar overall shapes.

Condition	CD $\downarrow$	SD-P95 $\downarrow$	F@1% $\uparrow$	F@2% $\uparrow$	Avg. SIoU $\uparrow$	Min. SIoU $\uparrow$
Codex	0.5402	1.0572	0.8783	0.9291	0.9888	0.9790
+Feature planning	0.1414	0.6836	0.9730	0.9950	0.9903	0.9813
+Measurement	0.0970	0.4878	0.9739	0.9926	0.9950	0.9915
+Refinement ( FACT )	0.0874	0.3740	0.9899	0.9990	0.9902	0.9861

Table 2: Cumulative reconstruction ablation on oracle meshes. Workflow stages are added sequentially to Codex (GPT-6); the final row is full FACT .

Method	IoU (%) $\uparrow$	SCR (%) $\downarrow$	TFSO (%) $\downarrow$	ISR (%) $\uparrow$	Step time (ms) $\downarrow$	$H\downarrow$	$V\downarrow$
Body	54.17	83.18	42.82	15.79	0.1035	76.1	9039.6
Part	68.75	79.52	21.35	28.42	0.1013	223.6	22123.7
Component	72.37	72.48	17.85	33.68	0.0956	306.2	21190.2
DiagSearch	74.50	59.78	16.15	42.28	0.0988	326.0	22043.4
FACT	79.43	0.00	6.79	99.47	0.0627	229.2	14493.7

Table 3: Quantitative comparison of contact fidelity. FACT achieves the lowest SCR and highest ISR with fewer hulls and lower physics-step time than DiagSearch.

Panel of Figure 4 — Figure 4: Contact fidelity and proxy complexity. Body, Part, and Component sweep CoACD thresholds; DiagSearch sweeps hull budgets. Failed settings are omitted; stars denote FACT .

Figure 6: Held-out motion prediction. Time-aligned frames (left) and joint trajectories (right) compare real observations with simulated responses from FACT and the baselines for the oven, fridge, and drawer.

Method	VRMSE $\downarrow$	TMSE $\downarrow$	OTR@10% $\downarrow$
GPT-5.6 Sol	0.5181	0.6763	40.52
GPT-6 Astra	0.5853	1.1535	31.73
Gemini-3.8-Flash	0.5566	0.1711	27.05
Qwen3.8-Max	0.6101	0.4236	36.49
Kimi-K3	0.8541	0.6995	46.97
PhysX-Anything	0.7165	0.7549	32.01

Table 4: Dynamic prediction on held-out clips. FACT outperforms direct parameter inference across all three motion metrics.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Asset ID	Source	Kinematic bodies
bakingchamber	V	2
bedsidetable	V	3
brimnes_cabinet	V	3
cabinet_1	V	3
drawer_kit_3	V	6
grand_piano	A	8

Appendix

Table 5: Reference dataset for the geometry and contact stages. V: ArtVIP; A: commercial artist asset.

Object	Observed motion	Kinematic bodies	Duration (s)	Clips
Chest	Lid with passive stays	4	25.84	2
Drawer	Lower drawer	4	21.93	6
Trashcan	Coupled lid and pedal	3	18.91	2
Cabinet	Middle door	4	17.08	3
Fridge	Upper door	4	41.70	7
Oven	Vertical-axis door	2	32.47	9

Appendix

Table 6: Real-object inventory and recording coverage.

Method	Configuration
CAD-Recode	256 input points; 10 generated programs; CD-based selection among executable candidates
MeshCoder	16,384 normalized input points; generated Blender code executed without manual repair
PrimitiveAnything	10,000 surface points with normals sampled from the normalized input mesh; official autoregressive inference pipeline
SuperFlex	4,096-point input point cloud; feed-forward deformable-superquadric checkpoint; no object-specific optimization

Appendix

Table 7: Inference configurations for learned geometry baselines.

Part	Before	After	Revision
Handle trim	0.037068	0.009469	Remove rear closure
Handle	0.087434	0.009444	Remeasure; replace extrusion with open loft
Dark surround	0.114964	0.002874	Remove concave mating surface

Appendix

Table 8: Local baking-chamber refinement. Surface distances are the maximum of the two directed part P95 values, divided by

D_{k}

Stage / samples	$H\downarrow$	$V\downarrow$	IoU $\uparrow$	ROI occ. $\downarrow$	False/free $\downarrow$
$C^{0}$ / development	420	19,641	95.6736	24.4141	93/93
$C^{a}$ / development	364	14,449	99.7027	16.9789	0/93
$C^{\rm final}$ / development	282	12,385	99.5943	17.0410	0/93
$C^{\rm final}$ / final	282	12,385	99.5784	16.9522	0/603

Appendix

Table 9: Microwave refinement trace from the initial proxy

C^{0}

to the repaired anchor

C^{a}

and compressed proxy

C^{\rm final}

. IoU and ROI occupancy are percentages; fidelity constraints are enforced per body and region rather than on aggregate values. False/free reports proxy-induced collision poses among reference-free samples.

Object	Response model and fitted parameters
Chest	Gravity-driven linkage with Coulomb lid resistance, $\tau_{\mathrm{res}}=-\tau_{c}\operatorname{Sign}(v)$ .
	Fit: $\tau_{c}=0.56757~\mathrm{N\,m}$ .
Drawer	Viscous and Coulomb resistance, $\ddot{q}=-\beta v-\alpha\operatorname{Sign}(v)$ .
	Fit: $\beta=4.04196~\mathrm{s}^{-1}$ , $\alpha=0.375197~\mathrm{m/s^{2}}$ .
Trashcan	Coupled lid–pedal mechanism with viscous lid resistance, $\tau_{\mathrm{res}}=-bv$ .
	Fit: $b=0.00790463~\mathrm{N\,m\,s/rad}$ .

Appendix

Table 10: Agent-constructed response models for dynamics calibration. Reported values are the final fitted parameters.

Object	Parameter bounds	Initial-state bounds
Chest	$\tau_{c}\in[0.05,1.5]$ , $b\in[0,0.12]$ .	$q_{0}:\pm 1.3^{\circ}$ ; $v_{0}:\pm r_{v}$ , with $v_{0}\leq 0$ .
Drawer	$\beta\in[0.01,30]$ , $\alpha\in[0,10]$ .	$q_{0}:\pm 0.004\,\mathrm{m}$ ; $v_{0}:\pm\max\!\left(0.3\,\mathrm{m/s},0.35\|\widetilde{v}_{0}\|\right)$ .
Trashcan	$b\in[0,0.04]$ , $\tau_{c}\in[0,0.035]$ .	$q_{0}:\pm 3^{\circ}$ ; $v_{0}:\pm 57.3^{\circ}/\mathrm{s}$ , with $v_{0}\leq 0$ .
Cabinet	$k_{c}\in[0.001,2]$ , $b_{c}\in[0.001,3]$ , $k_{o}\in[0.1,40]$ , $b_{o}\in[0.005,3]$ , $q_{o}\in[86^{\circ},97^{\circ}]$ .	$q_{0}:\pm 2^{\circ}$ ; $v_{0}:\pm 20^{\circ}/\mathrm{s}$ .
Fridge	$A\in[0.1,200]$ , $w\in[0.015,0.25]$ , $f\in[0.01,1.5]$ .	$q_{0}:\pm 1.5^{\circ}$ ; $v_{0}:\pm 8^{\circ}/\mathrm{s}$ .
Oven	$a\in[0.03,8]$ , $b\in[0,8]$ , $e\in[0.05,0.95]$ , $\delta\in[0,4^{\circ}]$ .	$q_{0}:\pm 2^{\circ}$ , with $q_{0}\geq 0$ ; $v_{0}:\pm\max\!\left(20^{\circ}/\mathrm{s},0.25\|\widetilde{v}_{0}\|\right)$ .

Appendix

Table 11: Candidate parameter and initial-state bounds considered during calibration and model revision. Final selected models may retain only a subset of the listed parameters.

Explore similar work

Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints
Mar 12, 2026Lijun Guo, Haoyu Zhao, Xingyue Zhao +5Monocular Video Articulated Objects
Wuhan University · Hong Kong Embodied AI Lab · The Chinese University of Hong Kong +3
Articulated Object Reconstruction from Rest-State Observation
Jul 30, 2026Daeun Lee, Jaeah Lee, Woosung Kim +23D Models State-Tracking
Seoul National University, Republic of Korea
GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects
Aug 20, 2025Licheng Shen, Saining Zhang, Honghan Li +4Geometric Reconstruction Object Articulation
BAAI · AIR, THU · NTU +2

Mar 12, 2026, cs.CVJ/K move · Enter open · S save

Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints

Lijun Guo, Haoyu Zhao, Xingyue Zhao, Rong Fu+4

Wuhan University · Hong Kong Embodied AI Lab · The Chinese University of Hong Kong+3

Monocular Video Articulated Objects

Open paper AskPDF arXiv

Building high-fidelity digital twins of articulated objects from visual data remains a central challenge. Existing approaches depend on multi-view captures of the object in discrete, static states, which severely constrains their real-world scalability. In this paper, we introduce Articulat3D, a novel framework that constructs such digital twins from casually captured monocular videos by jointly enforcing explicit 3D geometric and motion constraints. We first propose Motion Prior-Driven Initialization, which leverages 3D point tracks to exploit the low-dimensional structure of articulated motion. By modeling scene dynamics with a compact set of motion bases, we facilitate soft decomposition of the scene into multiple rigidly moving groups. Building on this initialization, we introduce Geometric and Motion Constraints Refinement, which enforces physically plausible articulation through learnable kinematic primitives parameterized by a joint axis, a pivot point, and per-frame motion scalars, yielding reconstructions that are both geometrically accurate and temporally coherent. Extensive experiments demonstrate that Articulat3D achieves state-of-the-art performance on synthetic benchmarks and real-world casually captured monocular videos, significantly advancing the feasibility of digital twin creation under uncontrolled real-world conditions. Our project page is available at https://maxwell-zhao.github.io/Articulat3D/.

Abstract

Explore similar work

Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints

Articulated Object Reconstruction from Rest-State Observation

GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects