Reconstructing complete 3D shapes from incomplete or noisy observations is a fundamentally ill-posed problem that requires balancing measurement consistency with shape plausibility. Existing methods for shape reconstruction can achieve strong geometric fidelity in ideal conditions but fail under realistic conditions with incomplete measurements or noise. At the same time, recent generative models for 3D shapes can synthesize highly realistic and detailed shapes but fail to be consistent with observed measurements. In this work, we introduce GG-Langevin: Geometry-Guided Langevin dynamics, a probabilistic approach that unifies these complementary perspectives. By traversing the trajectories of Langevin dynamics induced by a diffusion model, while preserving measurement consistency at every step, we generatively reconstruct shapes that fit both the measurements and the data-informed prior. We demonstrate through extensive experiments that GG-Langevin achieves higher geometric accuracy and greater robustness to missing data than existing methods for surface reconstruction.
Figures & tables
Figure 1: GG-Langevin. We combine the prior learned by a diffusion model with gradients from a geometric loss at inference time. By guiding the trajectories of Langevin dynamics, we obtain shapes that are both measurement-consistent and prior-consistent.
Figure 2: Blue trajectories: Non-guided Langevin dynamics on the prior distribution p(z) , initialized at an incomplete shape using the VAE encoder z0=E(P) . It generates plausible, complete shapes but quickly drifts from the measurements. Green trajectory: GG-Langevin generatively reconstructs the shape from the input point cloud. By incorporating gradients from a geometric loss L(z,P) , it keeps the sampling trajectory close to the manifold of measurement-consistent shapes where L(z,P)=0 (indicated by the dashed red line). On the right: A side-by-side comparison of sampling trajectories from Langevin dynamics and Geometry-Guided Langevin dynamics.
Figure 3: Toy example. Demonstration that our method generates samples from the geometry-guided distribution p~(z∣P) . Blue: Data distribution. Red: Two variants of guidance weight. Green: Geometry-guided product distribution. Solid lines indicate the predicted closed-form distributions. The histograms show samples generated using regular Langevin dynamics and GG-Langevin, respectively. We train an MLP diffusion model on samples from a bimodal Gaussian p(z) . The samples closely follow the predicted distributions.
Figure 4: Reconstruction results on sparse point clouds. Provided sparse point cloud scans as input, GG-Langevin recovers the complete surface and fine structures, significantly improving the reconstruction accuracy in comparison to previous work.
Sparse Scans
Incomplete Scans
Cars
Airplanes
Tables
Chairs
Cars
Airplanes
Tables
Chairs
CD
CA
CD
CA
CD
CA
CD
CA
CD
CA
CD
CA
CD
CA
CD
CA
Optimization
IGR [ 18 ]
1.07
27.7
2.80
33.4
2.36
25.1
2.52
29.1
4.47
32.8
3.82
31.7
6.04
29.2
5.38
30.4
DiffCD [ 20 ]
1.22
29.3
0.88
25.7
1.48
21.0
1.58
24.1
5.40
33.1
3.26
28.4
6.02
23.5
5.13
26.0
Learning-based
3DILG [ 51 ]
1.68
30.6
1.01
25.9
1.37
23.1
1.38
26.7
5.50
32.4
4.40
24.3
4.80
18.2
4.51
22.8
VecSet [ 52 ]
1.81
32.5
0.99
26.8
1.29
17.7
1.23
22.6
5.55
36.1
3.36
23.1
5.31
18.9
4.96
23.8
Table 1: Shape reconstruction. Comparison of different methods on ShapeNet categories for sparse and incomplete scans. Lower is better for both Chamfer Distance ×102 (CD) and Chamfer Angle (CA) in degrees. We highlight the best-performing method in each object category in bold and the second-best underlined.
Figure 5: Reconstruction results on incomplete point clouds. Despite incomplete point cloud scans as input, GG-Langevin recovers the missing structure with prior-consistent geometry. By comparison, previous work either struggles to complete the geometry or hallucinates implausible completions.
Figure 6: Sampler ablation. Performance comparison of methods for sampling from the geometry-guided distribution shape p~(z∣P) . We use the same latent space and loss function for all methods. The table on the right shows the Chamfer Distance (CD) and Chamfer Angle (CA) averaged across all object categories.
Figure 7: Impact of noise level σ and guidance strength β on reconstruction performance. If σ is too small relative to β the shape overfits the noise. Conversely, if σ is too large, the shape starts drifting away from the measurements.
Figure 8: Design choices for the autoencoder. We reduce the size of the decoder while maintaining the overall size of the autoencoder. Further, we predict a full SDF instead of a Truncated SDF (T-SDF) to enable gradients for the geometric prior.
Figure 9: Autoencoder ablation. Reconstruction quality with varying bottleneck position (26 layers total). Using 10 decoder layers balances quality and inference speed.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Failure cases. While rare, failure cases largely fall into two categories: incomplete reconstructions, where part of the shape is missing from the reconstruction, and spurious geometry, where redundant geometry is hallucinated.
Figure 11: Siren weight comparison. Increasing the Siren loss weight, μ , typically leads to less spurious geometry (top row), but can also lead to incomplete reconstructions (bottom row). The table shows metrics averaged across all shape categories.
Figure 12: Adding class conditioning. Comparing GG-Langevin with and without class conditioning. The class-conditioned model has a clear advantage in the incomplete setting, where predicting the object category from the point cloud is more difficult. In the sparse setting, both models perform similarly. For all our main experiments, we use the unconditioned model, which does not require the class label as input.
Figure 13: Initializing baselines. For a fair comparison, we adapt existing methods DPS [ 11 ] and DAPS [ 54 ] to make use of the encoder initialization z0=E(P) . In both cases, our suggested initialization improves performance.
Figure 14: Noise schedule for incomplete scans. Using cosine annealing for the noise level σ (rather than constant) and increasing the number of steps from 2000 (2k) to 5000 (5k) is beneficial for incomplete point cloud scans.
We consider the problem of regenerating 3D objects from 2D images and initial 3D shapes. Most 3D generators operate in a one-shot fashion, converting text or images to a 3D object with limited controllability. We introduce instead MeshReGen, a 3D regenerator that is conditioned on an initial 3D shape. This conceptually simple formulation allows us to support numerous useful tasks, including 3D enhancement, reconstruction, and editing. MeshReGen uses a new conditioning mechanism based on VecSet, which allows the regenerator to update or improve the input geometry with consistent fine-grained details. MeshReGen learns a widely applicable regeneration prior from off-the-shelf 3D datasets via self-supervised pretext tasks and augmentations, without additional annotations. We evaluate both the geometric consistency and fine-grained quality of MeshReGen, achieving state-of-the-art performance in controllable 3D generation across several tasks.
Geon Yeong Park, Roman Shapovalov, Rakesh Ranjan +3
KAIST · ⋆Work done during an internship at Meta Reality Labs. · Meta Reality Labs
Accurately reconstructing complex full multi-object scenes from sparse observations remains a core challenge in computer vision and a key step toward scalable and reliable simulation for robotics. In this work, we introduce RecGen, a generative framework for probabilistic joint estimation of object and part shapes, as well as their pose under occlusion and partial visibility from one or multiple RGB-D images. By leveraging compositional synthetic scene generation and strong 3D shape priors, RecGen generalizes across diverse object types and real-world environments. RecGen achieves state-of-the-art performance on complex, heavily occluded datasets, robustly handling severe occlusions, symmetric objects, object parts, and intricate geometry and texture. Despite using nearly 80% fewer training meshes than the previous state of the art SAM3D, RecGen outperforms it by 30.1% in geometric shape quality, 9.1% in texture reconstruction, and 33.9% in pose estimation.
Reconstructing complete 3D object assets from monocular or sparse multi-view observations remains challenging. Generative 3D foundation models can complete object geometry beyond the observed views, but their predictions may not faithfully reproduce the observed geometry, appearance, or pose. We introduce GenIA, a framework for test-time input-aligned generation that grounds SAM3D's generative prior in geometric and photometric observations without retraining the foundation model. We improve object pose by deriving translation and scale from geometry while retaining the learned rotation prior, and align appearance through visibility-biased attention, cross-observation fusion, and differentiable rendering guidance during denoising. An optional post-denoising refinement further adapts the appearance latent, lightweight decoder adapters, and object placement to the observations. Our framework also supports externally supplied geometry; when given temporal shapes of dynamic objects, it recovers a shared, input-aligned canonical appearance and stable world-space placement. Across synthetic and real benchmarks, GenIA improves pose prediction and object reconstruction from monocular, multi-view, and dynamic inputs, outperforming recent optimization-based, per-frame image-to-3D, and video-to-4D methods. Our project page is available at https://facebookresearch.github.io/GenIA.