Door opening and traversal is a long-horizon loco-manipulation task that requires precise handle interaction and coordinated base-arm control. We present Video2DoorTraversal, a single-video real-to-sim-to-real framework for wheel-legged mobile manipulators. Given one RGB video of a real door, DoorTwin reconstructs an instance-aligned, articulated, and simulation-ready door twin with realistic geometry and appearance. A simulation-in-the-loop agent converts the recovered articulation into a parameterized skill program and iteratively refines failed rollouts to generate physically executable demonstrations. These demonstrations are used to train ArticuACT, a dual-depth policy that predicts coordinated base, arm, and gripper commands using robot-centric camera conditioning and interaction-aware supervision. With all perception and policy inference running onboard, the system achieves a 96.57% average success rate across five real doors and an 80.95% zero-shot success rate on structurally similar unseen doors, while completing the full approach, opening, and traversal sequence in approximately 13s on average. Project Page: https://video2doortraversal.github.io/.
Figures & tables
Fig. 1: From a single RGB video, Video2DoorTraversal enables real-world wheel-legged door opening and traversal.
Method
Real-to-Sim Input
Mobile Robot
Articulated Manipulation
Agentic Expert
Sim-to-Real
Perception Input
Door Traversal
Human2Sim2Robot
✓ Human RGB video + scans
✗
✗
✗
✓
6D object pose
✗
X-SIM
✓ Human RGB video + scans
✗
✗
✗
✓
RGB
✗
SIMPACT
✓ Single RGB image
✗
✗
✓
✓
Initial RGB
✗
UniDoorManip
✗
✓
✓
✗
✓
Point cloud
✗
Teacher-student
✗
✓
✓
✗
✓
Given handle position
✓
Ours
✓ Single RGB video
✓
✓
✓
✓
Depth
✓
TABLE I: System-level comparison with representative real-to-sim-to-real manipulation and door-traversal methods.
Fig. 2: Overview of Video2DoorTraversal. Given a single RGB video, DoorTwin reconstructs a metrically grounded, articulated, and simulation-ready door asset through geometric grounding, articulation-aware generation, reference-view critique, and appearance refinement. A simulation-in-the-loop agent then instantiates and refines a parameterized skill program to collect randomized expert trajectories. Finally, ArticuACT combines dual-view depth observations with robot-centric Plücker ray maps to predict coordinated base, arm, and gripper commands for real-world door opening and traversal.
Fig. 3: Qualitative comparison of articulated door assets.
Fig. 4: Reference-view rendering and critic refinement iterations.
Method
Geometry
Appearance
VLM Score ↑
SS ↑
mIoU ↑
PSNR ↑
SSIM ↑
LPIPS ↓
PhysX-Omni
65.75
0.635
17.51
0.726
0.484
11.54
Articraft
89.03
0.880
16.64
0.702
0.470
28.75
Articulate Anything
86.62
0.831
16.03
0.667
0.565
11.59
DoorTwin (Ours)
94.95
0.972
18.53
0.692
0.408
56.74
TABLE II: Comparison of Geometry and Appearance Quality.
Method
Door Opening Success ↑
Traversal Success ↑
Replay
14.06%
14.06%
OA Replay
53.13%
53.13%
DoorGym
12.50%
12.50%
UniDoorManip
74.22%
50.78%
Vanilla ACT
67.58%
64.84%
Diffusion Policy (DP)
64.45%
64.45%
TABLE III: Simulation Results on the Door-Traversal Task.
Fig. 5: Left: ablation study of the proposed policy components and command spaces. Right: scaling performance of our method.
Method
Success Rate ↑
Avg. Agent Loops ↓
Rule-based
74.38%
0
Ours w/o Simulation Rollout
48.13%
0
Ours w/ VLM Fix–Log
85.00%
2.8
Ours Full
85.63%
2.5
TABLE IV: Ablation Study of the Agentic Trajectory Generation.
Fig. 6: Snapshots of real-world experiments with interaction state.