NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction

Weirong Chen; Chuanxia Zheng; Ganlin Zhang; Andrea Vedaldi; Daniel Cremers

cs.CVMar 4, 2026

NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction

Authors: Weirong Chen, Chuanxia Zheng, Ganlin Zhang, Andrea Vedaldi, Daniel Cremers

Organizations: Technical University of Munich · Munich Center for Machine Learning · University of Oxford · Nanyang Technological University

Abstract

We present NOVA3R, an effective approach for non-pixel-aligned 3D reconstruction from a set of unposed images in a feed-forward manner. Unlike pixel-aligned methods that tie geometry to per-ray predictions, our formulation learns a global, view-agnostic scene representation that decouples reconstruction from pixel alignment. This addresses two key limitations in pixel-aligned 3D: (1) it recovers both visible and invisible points with a complete scene representation, and (2) it produces physically plausible geometry with fewer duplicated structures in overlapping regions. To achieve this, we introduce a scene-token mechanism that aggregates information across unposed images and a diffusion-based 3D decoder that reconstructs complete, non-pixel-aligned point clouds. Extensive experiments on both scene-level and object-level datasets demonstrate that NOVA3R outperforms state-of-the-art methods in terms of reconstruction accuracy and completeness.

Figures & tables

Figure 1: NOVA3R enables non–pixel-aligned reconstruction by learning a global scene representation from unposed images. Compared to pixel-aligned methods, NOVA3R recovers both visible and occluded regions and produces more physically plausible geometry with fewer duplicated structures.

Figure 2: Comparison of different reconstruction paradigms. Our non-pixel-aligned approach combines feed-forward efficiency with a global, view-agnostic scene representation, removing the reliance on pixel-level supervision. NOVA3R provides a unified solution for various reconstruction tasks, achieving multi-view consistency and geometrically faithful results.

Figure 3: Overview of NOVA3R. Stage 1: a 3D point autoencoder encodes complete point clouds into latent scene tokens and decodes them with a flow-matching (FM) decoder. Stage 2: an image encoder with learnable scene tokens integrates multi-view information into a unified scene latent space, supervised by the FM loss with the Stage-1 decoder frozen. During inference , only the second stage pipeline is used to produce a complete, non–pixel-aligned point cloud.

Figure 4: Visible point clouds vs. complete point clouds. Our NOVA3R aims to recover the complete geometry within the input view’s frustum.

Figure 5: Different Decoder Architectures. The independent decoder uses cross-attention only, while the joint decoder implements an efficient self-attention, which yields more precise structures.

Type	Method	Visible ( $K$ =1)			Complete ( $K$ =1)			Complete ( $K$ =2)
Type	Method	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
Object	TripoSG*	(0.268)	(0.418)	(0.301)	0.242	0.467	0.333	-	-	-
Object	TRELLIS*	(0.301)	(0.420)	(0.313)	0.256	0.429	0.312	0.286	0.402	0.288
Single- view	Metric3D-v2	0.063	0.803	0.534	0.086	0.725	0.473	-	-	-
	DepthPro	0.055	0.852	0.603	0.079	0.764	0.535	-	-	-
	MoGe	0.035	0.945	0.786	0.063	0.836	0.668	-	-	-

Table 1: Quantitative results for scene completion on SCRREAM ( Jung et al., 2024 ) . The one-side Chamfer Distance (GT

\rightarrow

Prediction) results are shown in ( ).

K

is the number of input views.

*

denotes methods that are not trained on scene-level data. Our method shows better completion results compared to other competitive baselines. Note that, since NOVA3R is a non-pixel-aligned 3D reconstruction model, it does not explicitly distinguish the visible and occluded points.

Figure 6: Qualitative results for scene completion on SCRREAM ( Jung et al., 2024 ) . Our method produces more complete point clouds with clearer and less distorted geometry than other baselines.

Method	Complete (K=1)		Complete (K=2)		Complete (K=4)
Method	Hole Ratio $\downarrow$	Density Var. $\downarrow$	Hole Ratio $\downarrow$	Density Var. $\downarrow$	Hole Ratio $\downarrow$	Density Var. $\downarrow$
DUST3R	0.317	7.758	0.237	6.553	0.257	4.801
CUT3R	0.363	8.402	0.237	6.554	0.326	4.658
VGGT	0.307	7.105	0.238	6.546	0.261	5.217
Ours	0.088	5.127	0.121	2.188	0.134	1.881

Table 2: Quantitative results for hole area ratio and point cloud density variance on SCRREAM ( Jung et al., 2024 ) . Our method significantly outperforms pixel-aligned baselines, achieving both lower hole ratios and lower density variance.

Figure 7: Qualitative results for density evaluation on NRGBD ( $K=4$ ) ( Azinović et al., 2022 ) . Yellow regions denote higher density, and purple regions denote lower density. Despite being trained with only two views, NOVA3R generalizes well to multiple views ( $K=4$ ).

Method	# Tokens	Acc $\downarrow$		Comp $\downarrow$		NC $\uparrow$
Method	# Tokens	Mean	Med.	Mean	Med.	Mean	Med.
DUSt3R	2048	0.054	0.023	0.075	0.034	0.772	0.901
Spann3R	784	0.044	0.022	0.046	0.025	0.792	0.922
CUT3R	768	0.043	0.023	0.054	0.028	0.760	0.884
VGGT	2738	0.042	0.020	0.045	0.025	0.813	0.923
Ours	768	0.041	0.021	0.033	0.019	0.794	0.917

Table 3: Quantitative results on visible reconstruction on 7-Scenes (

K

=2) ( Shotton et al., 2013 ) . Our NOVA3R model can be trained on RGB-D data and achieves competitive results compared to multi-view reconstruction methods. Note that, we use fewer tokens to represent a 3D scene.

Type	Method	View-aligned ( $K$ =1)			View-aligned ( $K$ =2)
Type	Method	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
Single- view	SF3D	0.037	0.913	0.738	-	-	-
	SPAR3D	0.038	0.912	0.745	-	-	-
	LaRI	0.025	0.966	0.894	-	-	-
	TripoSG	0.025	0.961	0.899	-	-	-
Multi- view	TRELLIS	0.025	0.962	0.896	0.028	0.946	0.874

Table 4: Quantitative results for object completion on GSO ( Downs et al., 2022 ) . NOVA3R provides a unified solution for both scene and object completion from unposed images.

Figure 8: Qualitative results for object completion on GSO ( Downs et al., 2022 ) . Our method provides more precise geometry and better 3D consistency with multi-view inputs.

	Initial tokens (Stage 1)			# Scene tokens (Stage 1)			FM Decoder (Stage 1)		Img Resolution (Stage 2)
Settings	Point	Learnable	Hybrid	256	512	768	Indep.	Joint	224	518
CD $\downarrow$	0.011	0.013	0.011	0.014	0.013	0.011	0.012	0.011	0.054	0.048
[email protected] $\uparrow$	0.999	0.998	0.999	0.996	0.998	0.999	0.998	0.999	0.861	0.882
[email protected] $\uparrow$	0.991	0.981	0.993	0.975	0.986	0.993	0.990	0.993	0.648	0.687
[email protected] $\uparrow$	0.894	0.841	0.904	0.811	0.839	0.904	0.889	0.904	0.327	0.350

Table 5: Ablations. All models are evaluated on the SCRREAM complete (

K=1

) setting. We report CD

\downarrow

, [email protected]

\uparrow

, [email protected]

\uparrow

and [email protected]

\uparrow

across different ablation settings.

Training Loss	SCRREAM (Stage 1)
Training Loss	CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	Inference Time (s) $\downarrow$
Chamfer distance	0.024	0.981	0.907	0.575	0.557
Flow-matching	0.011	0.999	0.993	0.904	2.985

Table 6: Ablations on different training loss functions. All models are evaluated on the SCRREAM complete (

K=1

) setting. We report CD

\downarrow

, [email protected]

\uparrow

, [email protected]

\uparrow

and [email protected]

\uparrow

and inference time

\downarrow

for the decoder.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Visualization of point cloud generation at different resolutions. Our non–pixel-aligned formulation allows inference at arbitrary resolutions.

Figure 10: Qualitative results for scene completion on SCRREAM Jung et al. (2024) . Our method shows better scene completion results compared to other baselines.

Figure 11: Qualitative results for density evaluation on NRGBD ( $K{=}4$ ) ( Azinović et al., 2022 ) . Yellow regions denote higher density, and purple regions denote lower density. Our method provides more evenly-distributed point cloud (colored by density).

Figure 12: Extending 3D object generation model to real-world scene reconstruction. The pretrained TRELLIS model struggles to generate geometrically faithful reconstructions for real-world scenes. In contrast, NOVA3R can provide active voxel priors for the TRELLIS stage-1 generation process, enabling its extension to real-world scene synthesis.

Figure 13: Qualitative results for outdoor scenes reconstruction on Virtual KITTI 2. Our method is also applicable to outdoor scene reconstruction (colored by Y axis).

Explore similar work

UniQueR: Unified Query-based Feedforward 3D Reconstruction
Mar 24, 2026Chensheng Peng, Quentin Herau, Jiezhi Yang +63D Gaussian Splatting 3D Reconstruction
Applied Intuition · UC Berkeley · Stanford University
CAM3R: Camera-Agnostic Model for 3D Reconstruction
Mar 23, 2026Namitha Guruprasad, Abhay Yadav, Cheng Peng +13D Reconstruction Multi-View 3D Reconstruction
Johns Hopkins University, Baltimore, MD 21218, USA · University of Virginia, Charlottesville, VA 22904, USA
G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing
May 26, 2026Bharath Raj Nagoor Kani, Noah Snavely3D Reconstruction Camera Pose Estimation
Cornell University

Mar 24, 2026J/K move · Enter open · S save

UniQueR: Unified Query-based Feedforward 3D Reconstruction

Chensheng Peng, Quentin Herau, Jiezhi Yang, Yichen Xie+5

Applied Intuition · UC Berkeley · Stanford University

3D Gaussian Splatting 3D Reconstruction Multi-View 3D Reconstruction 3D Scene Representation Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

We present UniQueR, a unified query-based feedforward framework for efficient and accurate 3D reconstruction from unposed images. Existing feedforward models such as DUSt3R, VGGT, and AnySplat typically predict per-pixel point maps or pixel-aligned Gaussians, which remain fundamentally 2.5D and limited to visible surfaces. In contrast, UniQueR formulates reconstruction as a sparse 3D query inference problem. Our model learns a compact set of 3D anchor points that act as explicit geometric queries, enabling the network to infer scene structure, including geometry in occluded regions--in a single forward pass. Each query encodes spatial and appearance priors directly in global 3D space (instead of per-frame camera space) and spawns a set of 3D Gaussians for differentiable rendering. By leveraging unified query interactions across multi-view features and a decoupled cross-attention design, UniQueR achieves strong geometric expressiveness while substantially reducing memory and computational cost. Experiments on Mip-NeRF 360 and VR-NeRF demonstrate that UniQueR surpasses state-of-the-art feedforward methods in both rendering quality and geometric accuracy, using an order of magnitude fewer primitives than dense alternatives.

Abstract

Explore similar work

UniQueR: Unified Query-based Feedforward 3D Reconstruction

CAM3R: Camera-Agnostic Model for 3D Reconstruction

G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing