From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis

Ranran Huang; Weixun Luo; Ye Mao; Krystian Mikolajczyk

cs.CVMar 29, 2026

From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis

Authors: Ranran Huang, Weixun Luo, Ye Mao, Krystian Mikolajczyk

Organizations: Imperial College London

Abstract

In this paper, we introduce NAS3R, a self-supervised feed-forward framework that jointly learns explicit 3D geometry and camera parameters with no ground-truth annotations and no pretrained priors. During training, NAS3R reconstructs 3D Gaussians from uncalibrated and unposed context views and renders target views using its self-predicted camera parameters, enabling self-supervised training from 2D photometric supervision. To ensure stable convergence, NAS3R integrates reconstruction and camera prediction within a shared transformer backbone regulated by masked attention, and adopts a depth-based Gaussian formulation that facilitates well-conditioned optimization. The framework is compatible with state-of-the-art supervised 3D reconstruction architectures and can incorporate pretrained priors or intrinsic information when available. Extensive experiments show that NAS3R achieves superior results to other self-supervised methods, establishing a scalable and geometry-aware paradigm for 3D reconstruction from unconstrained data. Code and models are publicly available at https://ranrhuang.github.io/nas3r/.

Figures & tables

Supervised
Method	Priors	Training		Inference
Method	Priors	Pose	Intrinsics	Pose	Intrinsics
pixelSplat	DINO [ 6 ]	✓	✓	✓	✓
MVSplat	UniMatch [ 68 ]	✓	✓	✓	✓
NoPoSplat	MASt3R [ 34 ]	✓	✓	✗	✓
Self-Supervised

Table 1: Training and inference requirements of different methods. Training columns indicate the need for ground-truth camera poses and intrinsics, while inference columns indicate whether context-view camera parameters are required as model inputs.

Figure 2: Training pipeline of NAS3R. Subscripts “ $C$ ” and “ $T$ ” denote context and target views, respectively. Unconstrained images are patchified into visual tokens and concatenated with a learnable camera token for camera prediction. A masked decoder regulates cross-view interactions and prevents target-to-context leakage. Refined context tokens are then processed by the Gaussian head to predict Gaussian parameters, while a depth head estimates depth maps that are lifted into 3D Gaussian centers using the predicted context poses. The predicted target poses are finally used to render novel views, providing photometric supervision for end-to-end training.

Supervised
Method	Training Data	RE10K			ACID			DL3DV			DTU
Method	Training Data	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
pixelSplat	RE10K	23.859	0.808	0.184	25.477	0.770	0.207	21.370	0.713	0.250	15.067	0.539	0.341
MVSplat		24.012	0.812	0.175	25.525	0.773	0.199	20.344	0.673	0.258	14.542	0.537	0.324
NoPoSplat ∗		21.382	0.709	0.266	23.193	0.667	0.274	18.235	0.518	0.364	15.113	0.455	0.450
Self-Supervised

Table 2: Performance comparison of in-domain and out-of-domain two-view novel view synthesis for models trained with no supervised 3D priors . The best results among self-supervised methods are highlighted.

\ast

indicates the variants with random initialization.

Figure 3: Comparison of NVS results across different methods. The leftmost column shows the two-view context images. From top to bottom, the settings are RE10K, RE10K $\rightarrow$ ACID, RE10K $\rightarrow$ DTU, and RE10K $\rightarrow$ DL3DV.

Figure 4: Visual comparison of pose trajectories on RE10K. Camera frustums for ground-truth and predicted poses are shown in black and orange , respectively. The top two examples correspond to 5-view reconstruction, while the bottom example corresponds to 10-view reconstruction.

SfM-based
Method	RE10K						ACID						DL3DV
	Rotation		Translation		Overall		Rotation		Translation		Overall		Rotation		Translation		Overall
	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$
SP + SG	62.7	73.3	42.0	58.5	40.6	56.9	62.3	72.4	38.0	51.6	36.3	50.0	47.8	56.1	40.2	52.6	37.2	49.2
NVS-based
SelfSplat	48.7	62.7	19.2	32.8	18.4	31.8	64.5	75.2	26.4	40.5	25.8	39.6	21.0	34.4	8.9	22.4	6.1	16.8

Table 3: Performance comparison of two-view pose estimation (AUC, %) on RE10K, ACID and DL3DV datasets. All NVS models are trained on RE10K with no supervised 3D priors .

\ast

indicates the SPFSplat variant trained with random initialization.

Supervised
Method	RE10K			RE10K $\rightarrow$ ACID
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NoPoSplat	25.061	0.839	0.161	25.765	0.776	0.199
Self-Supervised
PF3plat	21.042	0.739	0.233	20.726	0.610	0.308
SPFSplat	25.484	0.847	0.153	25.965	0.781	0.190

Table 5: Performance comparison of two-view NVS when using supervised priors . Best and second-best results are highlighted.

SfM-based
Method	RE10K			RE10K $\rightarrow$ ACID
Method	5 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	5 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$
DUSt3R	33.6	54.1	70.2	11.8	27.9	47.0
MASt3R	28.1	49.4	67.1	13.8	31.2	50.7
VGGT	25.7	47.4	65.8	14.2	30.4	48.6
NVS-based

Table 6: Performance comparison of two-view pose estimation when using supervised priors . “-I” indicates using GT intrinsics.

Training Data	NVS			Pose		Depth
Training Data	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$	rel $\downarrow$	$\tau$ $\uparrow$
RE10K	15.146	0.520	0.442	19.2	34.1	0.206	71.4
DL3DV	16.171	0.542	0.431	19.1	33.2	0.175	71.9
RE10K+DL3DV	16.316	0.552	0.420	24.5	41.6	0.145	79.1

Table 7: Performance on NVS and pose estimation (ScanNet++) and depth estimation (BlendedMVS) with increasing training data.

Num of Views	NVS			Pose
Num of Views	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	5 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$
2 views	23.130	0.764	0.193	32.6	51.0	64.9
3 views	25.384	0.829	0.141	37.9	57.6	72.0
5 views	25.983	0.844	0.131	38.9	58.5	72.8
10 views	27.093	0.872	0.113	41.0	61.0	75.5

Table 8: Performance on novel view synthesis and pose estimation with varying number of input views.

Method	Depth		Pose
Method	rel $\downarrow$	$\tau$ $\uparrow$	5 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$
(1) Self-Supervised (NAS3R)	0.145	79.1	29.8	49.5	66.8
(2) Supervised from scratch	0.177	76.6	10.7	23.5	39.4
(3) Supervised from weights in (1)	0.119	90.0	33.9	54.3	71.3

Table 9: Self-supervised pretraining improves downstream finetuning. Supervised finetuning from self-supervised pretrained weights (3) outperforms both self-supervised learning (1) and supervised training from scratch (2) on BlendedMVS.

Figure 6: Examples of 3D Gaussians and rendered RGB and depth results on RE10K.

Figure 7: More novel view synthesis qualitative comparisons on DL3DV.

Method	NVS			Pose
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	5 ∘ $\uparrow$	10 ∘ $\uparrow$	20 ∘ $\uparrow$
No Curriculum	21.020	0.650	0.255	20.3	35.8	50.2
Progressive Interval Curriculum	23.130	0.764	0.193	32.6	51.0	64.9

Table 10: Ablation on progressive interval curriculum on RE10K.

Methods	w/o priors			w/ VGGT priors
Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NAS3R	23.130	0.764	0.193	25.888	0.861	0.136
NAS3R-I	23.144	0.758	0.196	25.872	0.861	0.135

Table 11: Ablation on ground-truth intrinsics on RE10K.

Method	Training Data	2 View			5 Views
Method	Training Data	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
(a) AnySplat	DL3DV+8 datasets	16.905	0.514	0.249	21.680	0.694	0.218
(b) NAS3R	DL3DV	20.069	0.588	0.281	21.307	0.646	0.240

Table 12: Comparison with AnySplat on DL3DV dataset.

Methods	Params (M)	FLOPs (GMac)	Time (s)
NAS3R (MASt3R)	613.47	404.52	0.042
NAS3R (VGGT)	1223.2	607.65	0.073

Table 13: Inference efficiency on an NVIDIA A6000 GPU.

Figure 8: More novel view synthesis qualitative comparisons on RE10K.

Figure 9: More novel view synthesis qualitative comparisons on ACID.

Explore similar work

Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself
Apr 15, 2026Yuhang Dai, Xingyi YangFeed-Forward 3D Reconstruction Multi-View 3D Reconstruction
The Hong Kong Polytechnic University, Hong Kong SAR
FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views
Apr 10, 2026Chaoyi Zhou, Run Wang, Feng Luo +43D Reconstruction Feed-Forward 3D Reconstruction
Microsoft · Clemson University · Texas A&M University
SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views
Sep 21, 2025Ranran Huang, Krystian Mikolajczyk3D Gaussian Splatting 3D Reconstruction
Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, United Kingdom

Apr 15, 2026J/K move · Enter open · S save

Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself

Yuhang Dai, Xingyi Yang

The Hong Kong Polytechnic University, Hong Kong SAR

Feed-Forward 3D Reconstruction Multi-View 3D Reconstruction Test-Time Adaptation Self-Supervised Learning

Open paper AskPDF arXiv

Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth. Our key insight is that, when the model receives more views, it produces more reliable and view-consistent reconstructions. Leveraging this property, given a testing sequence, we mask a subset of frames to construct a self-supervised task. Free Geometry enforces cross-view feature consistency between representations from full and partial observations, while maintaining the pairwise relations implied by the held-out frames. This self-supervision allows for fast recalibration via lightweight LoRA updates, taking less than 2 minutes per dataset on a single GPU. Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets, yielding an average improvement of 3.73% in camera pose accuracy and 2.88% in point map prediction. Code is available at https://github.com/hiteacherIamhumble/Free-Geometry .

Abstract

Explore similar work

Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself

FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views

SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views