Single-View 3D Reconstruction

Follow this topic Ask about this topic

Single-view 3D reconstruction estimates geometry and appearance from one image, where hidden surfaces, depth, and object relationships are inherently ambiguous. Methods represent outputs as meshes, point clouds, neural fields, or 3D Gaussians, using feed-forward prediction, parametric shape models, differentiable rendering, and diffusion or video-generation priors to infer unseen views. Work spans objects, indoor scenes, humans and animals, and specialized settings such as medical imaging and CAD; physical constraints and explicit image-to-geometry alignment increasingly support editable or simulation-ready results. The central tension is that generative priors can produce complete, plausible outputs but may hallucinate details or drift from the input, so geometry, identity, cross-view consistency, speed, and uncertainty remain important evaluation challenges.

Latest papers

121 papers17 papers in the last four weeks, up 6% on the four weeks before. 0.1% of all new papers.

PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing
Sep 16, 2024Peng Li, Wangguandong Zheng, Yuan Liu +9Multi-View Diffusion Models 3D Reconstruction
HKUST · Southeast University · Tsinghua University +1

Sep 16, 2024J/K move · Enter open · S save

PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing

Peng Li, Wangguandong Zheng, Yuan Liu, Tao Yu+8

HKUST · Southeast University · Tsinghua University+1

Multi-View Diffusion Models 3D Reconstruction 3D Mesh Reconstruction 3D Human Reconstruction Single-View 3D Reconstruction Diffusion Models

Open paper AskPDF arXiv

Detailed and photorealistic 3D human modeling is essential for various applications and has seen tremendous progress. However, full-body reconstruction from a monocular RGB image remains challenging due to the ill-posed nature of the problem and sophisticated clothing topology with self-occlusions. In this paper, we propose PSHuman, a novel framework that explicitly reconstructs human meshes utilizing priors from the multiview diffusion model. It is found that directly applying multiview diffusion on single-view human images leads to severe geometric distortions, especially on generated faces. To address it, we propose a cross-scale diffusion that models the joint probability distribution of global full-body shape and local facial characteristics, enabling detailed and identity-preserved novel-view generation without any geometric distortion. Moreover, to enhance cross-view body shape consistency of varied human poses, we condition the generative model on parametric models like SMPL-X, which provide body priors and prevent unnatural views inconsistent with human anatomy. Leveraging the generated multi-view normal and color images, we present SMPLX-initialized explicit human carving to recover realistic textured human meshes efficiently. Extensive experimental results and quantitative evaluations on CAPE and THuman2.1 datasets demonstrate PSHumans superiority in geometry details, texture fidelity, and generalization capability.