Dynamic scene reconstruction from casual videos has seen recent remarkable progress. Numerous approaches have attempted to overcome the ill-posedness of the task by distilling priors from 2D foundational models and by imposing hand-crafted regularization on the optimized motion. However, these methods struggle to reconstruct scenes from extreme novel viewpoints, especially when highly articulated motions are present. In this paper, we present DRoPS, a novel approach that leverages a static pre-scan of the dynamic object as an explicit geometric and appearance prior. While existing state-of-the-art methods fail to fully exploit the pre-scan, DRoPS leverages our novel setup to effectively constrain the solution space and ensure geometrical consistency throughout the sequence. The core of our novelty is twofold: first, we establish a grid-structured and surface-aligned model by organizing Gaussian primitives into pixel grids anchored to the object surface. Second, by leveraging the grid structure of our primitives, we parameterize motion using a CNN conditioned on those grids, injecting strong implicit regularization and correlating the motion of nearby points. Extensive experiments demonstrate that our method significantly outperforms the current state of the art in rendering quality and 3D tracking accuracy.
Figures & tables
Figure 2: DRoPS Overview . (a) We organize our canonical Gaussians into structured pixel grids that reside on virtual cameras surrounding the object, each pixel encoding the parameters of its back-projected 3D Gaussian ( Sec. 3.1 ). (b) To reconstruct the dynamic sequence, we model the object deformation with Deep Motion Prior Φθ – a CNN that maps canonical positions μj and timestep encodings γ(t) to 6-DOF motion parameters [Qj,Δj] (see Sec. 3.2 ). Timesteps {\color[rgb]{0.5117,0.2109,0.5508}t_{1}},{\color[rgb]{0.8906,0.0234,0.0742}t_{2}},{\color[rgb]{0,0.5898,0.25}t_{3}} are color-coded in the figure.
Figure 3: Surface-aligned Gaussians . We visualize canonical Gaussians of the upper body. Unlike standard 3DGS (left), where Gaussians are unordered and lack surface representation, our structured Gaussians align with the object’s surface, providing a more robust and generalizable representation for dynamic-time reconstruction.
Figure 4: Qualitative results . The first column depicts the training view from the monocular sequence; the second column depicts the ground-truth testing view at the same timestep. Novel views are selected at extreme angles to evaluate the completeness of dynamic 3D reconstructions. Our method drastically outperforms the baselines in maintaining a consistent object geometry and sharp appearance, while accurately modeling the scene dynamics. See our website for video results.
Figure 5: DRoPS achieves high-quality dynamic 3D reconstruction on in-the-wild monocular videos by generating the pre-scan with an image-to-3D model [ 68 ] . Our novel views are rendered from viewpoints that differ significantly from those in the training set. See our website for full video and additional results.
Figure 6: Text-to-4D application. Each column and row correspond to a fixed timestep and viewpoint, respectively. The colored lines visualize the 3D trajectories emerging from our representation. See our website for videos.
Figure 7: We additionally compare to DM4D [ 34 ] , with and without providing them the pre-scan. DM4D results in distorted geometry, inaccurate motion, and visual artifacts. In contrast, DRoPS achieves accurate, consistent and realistic reconstructions.
Figure 8: Ablation Results . Removing DMP ("w/o CNN") overfits to the training view, failing to capture the motion and geometry. Replacing the CNN with an MLP improves over it, but still suffers from geometry distortion and inaccurate motion. Lcrs_iso is crucial for global geometry preservation, while Ldense_iso has a relatively small impact since the inductive bias of DMP introduces strong local motion correlation.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Pre-scan Refinement optimization (Sec. 3.1 ) significantly improves high-frequency details in shape and texture, e.g. the geometry of the person’s face, the coyote’s fur and nose shape. Renders are obtained from test views.