cs.CVNov 5, 2026

XM4D: 4D Reconstruction through Certifiable Geometric Optimization

Authors: Haoyu Han, Jack Benarroch Jedlicki, Jiahui Lei, Heng Yang

Organizations: Harvard University · UC Berkeley

Abstract

Accurate 4D reconstruction remains challenging: jointly estimating camera motion, scene geometry, and non-rigid deformation entails highly nonconvex optimization, making global optimality difficult to guarantee. We present XM4D, a framework for accurate and efficient 4D reconstruction built upon certifiably globally optimal geometric optimization. By separating the estimation of camera motion from non-rigid deformation, we formulate both stages as structured quadratically constrained quadratic programs (QCQP) that admit semidefinite programming (SDP) relaxations and verifiable global optimality certificates. For camera motion estimation, we lift static point tracks into 3D using foundational depth prediction models and formulate a scaled bundle adjustment problem that jointly estimates camera poses, depth scales, and static landmarks. By analytically eliminating translations and landmarks, we reduce the problem to a QCQP over scaled rotations. For non-rigid reconstruction, we represent dynamic motion using an embedded deformation graph (EDG) and show that its tracking, rigidity, and temporal regularization objectives similarly reduce to a rotation-only QCQP. Both problems admit SDP relaxations that we solve efficiently with a GPU-accelerated Burer-Monteiro solver based on Riemannian optimization. The recovered geometry further initializes dynamic Gaussian splatting for photometric refinement. Extensive experiments across seven benchmarks demonstrate accurate camera motion estimation on both static and dynamic scenes, efficient optimization that scales to thousands of frames, and state-of-the-art novel-view rendering quality.

Figures & tables

Explore similar work

CardsList
  1. Complet4R: Geometric Complete 4D Reconstruction

    Mar 28, 2026Weibang Wang, Kenan Li, Zhuoguang Chen +24D ReconstructionDynamic Scene Reconstruction

  2. SM4RT: Learning Structured Motion Geometry for 4D Reconstruction

    Jul 24, 2026Shing Ho J. Lin, Wenzhao Zheng, Dong Zhuo +33D ViTs4D Reconstruction

  3. SM4RT: Cascaded Feed-forward Model for 4D Reconstruction

    Nov 5, 2026Pang-Chi Lo, Jhao-Syun Lai, Tsung-Wei Ke4D ReconstructionDynamic Scene Reconstruction