DecomVoxel: Harnessing 3D-Native Priors with Guided In-situ Denoising Optimization for Decompositional Scene Reconstruction

Junfeng Ni; Zirui Zhou; Yixin Chen; Yu Liu; Nan Jiang; Zhifei Yang; Song-Chun Zhu; Siyuan Huang

cs.CVOct 1, 2026

DecomVoxel: Harnessing 3D-Native Priors with Guided In-situ Denoising Optimization for Decompositional Scene Reconstruction

Authors: Junfeng Ni, Zirui Zhou, Yixin Chen, Yu Liu, Nan Jiang, Zhifei Yang, Song-Chun Zhu, Siyuan Huang

Organizations: Tsinghua University, State Key Laboratory of General Artificial Intelligence, BIGAI, China · Peking University, China

Abstract

Decompositional scene reconstruction aims to reconstruct high-quality objects and background, yet existing methods still struggle with the level of quality under heavy occlusions. While generative priors offer a potential solution, 2D image-based priors often suffer from multi-view inconsistency due to a lack of 3D awareness. Conversely, 3D-native priors provide stronger structural inductive biases but frequently lead to spatial drift and misalignment within complex scenes. To address these issues, we propose DecomVoxel, formulating object completion as a guided in-situ denoising optimization that bridges 3D-native priors with neural scene reconstruction. Our framework introduces a reformulated epsilon-based distillation loss to ensure stable latent refinement, alongside adaptive spatial guidance that utilizes occupied and vacant anchors with temporal annealing to suppress generative hallucinations and mitigate spatial drift. Experiments on Replica and ScanNet++ show that DecomVoxel significantly outperforms state-of-the-art methods while faithfully preserving the original spatial layout, structural fidelity, and style-consistent texture. Our method pushes the boundary of decompositional reconstruction by delivering high-quality textured meshes with clean topology, geometry, and appearance, providing a robust solution for the decompositional reconstruction of complex real-world scenes. Code is available at https://github.com/DecomVoxel/DecomVoxel.

Figures & tables

Figure 1 . We propose DecomVoxel , a framework that integrates guided 3D-native priors to enhance decompositional scene reconstruction. By formulating object completion as guided in-situ optimization, our method achieves high-quality topology, geometry, and appearance for both individual objects and backgrounds while faithfully preserving the original spatial layout.

Figure 2 . Overview of DecomVoxel . Our framework bridges 3D generative priors with neural reconstruction through a guided in-situ denoising optimization. We segment objects from the initial scene and then sequentially optimize their geometry and appearance under adaptive spatial guidance. This process recovers missing structures and consistent textures while preserving the original scene layout, producing high-fidelity topology, geometry, and appearance.

Figure 3 . Visualization of intermediate results. The left panel illustrates our spatial anchors: the occupied anchor $\Omega_{O}$ (purple) preserves verified structures, while the vacant anchor $\Omega_{V}$ (red) suppresses hallucinations in prohibited zones. The green regions denote the free space for structural synthesis. The right panel displays the progressive geometric completion over iterations. Guided by these deterministic anchors, the generative process restores missing regions while maintaining the original scene layout.

Figure 4 . Qualitative comparison. Our method reconstructs complete geometry and consistent appearance while faithfully preserving the original scene layout, whereas baselines exhibit substantial drift in position, rotation, and scale.

Dataset	Method	Reconstruction			Rendering
Dataset	Method	CD $\downarrow$	F-score $\uparrow$	NC $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Replica	SimRecon	17.55	40.61	63.25	18.04	0.858	0.152
	ReconViaGen	15.49	51.02	68.09	18.58	0.861	0.147
	CUPID	16.29	43.95	64.80	16.70	0.809	0.180
	SAM3D	10.59	45.46	66.33	18.89	0.847	0.143
	MV-SAM3D	12.87	40.46	64.19	18.83	0.848	0.142

Table 1. Quantitative comparison. Our method consistently outperforms state-of-the-art baselines across both reconstruction and rendering metrics. Top-3 results are highlighted as the first , second and third .

Method	Obj Geo. $\uparrow$	Obj App. $\uparrow$	Layout $\uparrow$
SimRecon	$2.12\pm 1.30$	$1.86\pm 1.25$	$2.01\pm 1.12$
SAM3D	$2.37\pm 1.40$	$2.21\pm 1.41$	$2.26\pm 1.22$
MV-SAM3D	$2.66\pm 1.59$	$2.52\pm 1.60$	$2.68\pm 1.01$
ShapeR	$2.19\pm 1.36$	-	$2.95\pm 1.20$
Ours	$\mathbf{3.81\pm 1.25}$	$\mathbf{3.78\pm 1.33}$	$\mathbf{4.08\pm 0.95}$

Table 2 . User Study. Based on 44 expert questionnaires, we assess the scores (rated 1–5) of object completeness and consistency in geometry and appearance, along with spatial-layout fidelity relative to the original scene. Our method significantly outperforms all baselines across all metrics, particularly in preserving global structural alignment.

GP	G-SG	AP	A-SG	Reconstruction			Rendering
GP	G-SG	AP	A-SG	CD $\downarrow$	F-score $\uparrow$	NC $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$\times$	$\times$	$\times$	$\times$	19.48	71.21	45.94	19.81	0.881	0.133
✓	$\times$	$\times$	$\times$	6.52	66.34	72.81	20.03	0.860	0.129
✓	✓	$\times$	$\times$	3.91	78.16	78.99	21.32	0.874	0.117
✓	✓	✓	$\times$	4.04	77.48	78.72	20.55	0.866	0.122
✓	✓	✓	✓	3.95	77.87	78.85	24.21	0.907	0.114

Table 3. Ablation study. Geometry (GP) and appearance (AP) priors improve the reconstruction and rendering quality, while adaptive spatial guidance (G-SG and A-SG) further enhances the consistency.

Panel of Figure 5 — Figure 5 . Comparison of different optimization strategies. Our epsilon-based denoising loss (a), linear noise schedule (b), re-distributed pruning strategy (c), and cosine-power preservation loss weight schedule (d) consistently achieve the most stable convergence.

Method	Runtime $\downarrow$	Peak Memory $\downarrow$	Avg. Memory $\downarrow$
SimRecon	0.18	17.659	13.446
SAM3D	0.13	32.797	22.997
MV-SAM3D	3.13	68.157	51.580
ShapeR	0.37	11.255	5.691
ReconViaGen	0.18	14.911	7.348
CUPID	0.14	13.778	7.123

Table 4. Runtime (hours) and GPU memory (GB) on Replica.

Figure 6 . Comprehensive Reconstruction Results. Our method demonstrates robust, high-quality scene decompositional reconstruction performance on complex real-world captures.

Figure 7 . More Qualitative Comparison. Our method consistently outperforms other methods across different datasets.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Metric	Definition
cd ( cd )	$\frac{\textit{Accuracy}+\text{Completeness}}{2}$
Accuracy	$\underset{\bm{p}\in P}{\mbox{mean}}\left(\underset{\bm{p}^{}\in P^{}}{\mbox{min}}\|\|\bm{p}-\bm{p}^{*}\|\|_{1}\right)$
Completeness	$\underset{\bm{p}^{}\in P^{}}{\mbox{mean}}\left(\underset{\bm{p}\in P}{\mbox{min}}\|\|\bm{p}-\bm{p}^{*}\|\|_{1}\right)$
F-score	$\frac{2\times\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}$
Precision	$\underset{\bm{p}\in P}{\mbox{mean}}\left(\underset{\bm{p}^{}\in P^{}}{\mbox{min}}\|\|\bm{p}-\bm{p}^{*}\|\|_{1}<0.05\right)$
Recall	$\underset{\bm{p}^{}\in P^{}}{\mbox{mean}}\left(\underset{\bm{p}\in P}{\mbox{min}}\|\|\bm{p}-\bm{p}^{*}\|\|_{1}<0.05\right)$

Appendix

Table 6 . Evaluation metrics. We list the metrics used to evaluate reconstruction quality and their definitions.

P

and

P^{*}

are the point clouds sampled from the predicted and ground-truth meshes.

\bm{n}_{\bm{p}}

is the normal vector at point

\bm{p}

Explore similar work

VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching
May 29, 2026Tuan Duc Ngo, Chuang Gan, Evangelos KalogerakisScene Reconstruction Volume
UMass Amherst · 2TU Crete
DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion
May 16, 2026Yuhan Ping, Yuan Liu, Xiaoxiao Long +63D Reconstruction 3D Scene Generation
Department of Computer Science, the University of Hong Kong · Department of Computer Science, City University of Hong Kong · Faculty of Humanities and Arts, Macau University of Science and Technology +2
Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction
Aug 12, 2026Yan Di, Chengxi Li, Yaoxing Wang +7Surface Reconstruction Voxel
HIT (Shenzhen) · Tsinghua University · NWPU +2

May 29, 2026, cs.CVJ/K move · Enter open · S save

VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching

Tuan Duc Ngo, Chuang Gan, Evangelos Kalogerakis

UMass Amherst · 2TU Crete

Scene Reconstruction Volume Spatial Priors Generative Models Conditional Flow Matching

Open paper AskPDF arXiv

Reconstructing the complete geometry of a scene from a single RGB image remains challenging - especially when inferring hidden structures where visual evidence is incomplete. We introduce VolFill, a generative framework that predicts the 3D structure of the complete scene rather than relying on traditional pixel-aligned regression. Our method utilizes a hybrid 3D VAE to compress sparse truncated unsigned distance function grids into a compact latent space, paired with a latent Diffusion Transformer that denoises this representation to recover the complete scene. We condition the generation on geometry foundation models, leveraging rich spatial priors for robust reasoning. Unlike existing methods limited by per-ray constraints or unstructured point-cloud queries, VolFill provides a structured representation that supports direct surface extraction and occupancy queries at scale. Extensive experiments on the SCRREAM and NRGB-D datasets demonstrate that our approach significantly outperforms current baselines, providing a robust foundation for holistic spatial understanding.

Abstract

Explore similar work

VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching

DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion

Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction