Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself

Yuhang Dai; Xingyi Yang

cs.CVApr 15, 2026

Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself

Authors: Yuhang Dai, Xingyi Yang

Organizations: The Hong Kong Polytechnic University, Hong Kong SAR

Abstract

Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth. Our key insight is that, when the model receives more views, it produces more reliable and view-consistent reconstructions. Leveraging this property, given a testing sequence, we mask a subset of frames to construct a self-supervised task. Free Geometry enforces cross-view feature consistency between representations from full and partial observations, while maintaining the pairwise relations implied by the held-out frames. This self-supervision allows for fast recalibration via lightweight LoRA updates, taking less than 2 minutes per dataset on a single GPU. Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets, yielding an average improvement of 3.73% in camera pose accuracy and 2.88% in point map prediction. Code is available at https://github.com/hiteacherIamhumble/Free-Geometry .

Figures & tables

Figure 1: Free Geometry enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth and generalize on models and datasets.

# Views	ETH3D		7-Scenes		ScanNet++		HiRoom
# Views	AUC@3 $\uparrow$	F1 $\uparrow$	AUC@3 $\uparrow$	F1 $\uparrow$	AUC@3 $\uparrow$	F1 $\uparrow$	AUC@3 $\uparrow$	F1 $\uparrow$
8	0.445	0.536	0.315	0.439	0.733	0.436	0.827	0.787
$8\rightarrow 4$	0.424	0.188	0.302	0.285	0.747	0.306	0.857	0.682
4	0.318	0.142	0.254	0.239	0.586	0.231	0.700	0.561

Table 1: Long Sequence Provides Better Reconstruction Accuracy. Row 2 (

8\!\rightarrow\!4

) uses 8 input views to compute encoder features and forwards only the corresponding 4-view features to the decoder. We report pose accuracy (AUC@3

\uparrow

) and reconstruction quality (F1

\uparrow

). Rankings are highlighted with first , second , and third within each column.

Figure 2: Long Sequence Provides Better Reconstruction Geometry. The left example shows a HiRoom bedroom scene reconstructed by using 8 input views in VGGT while forwarding only the corresponding 4-view features to the decoder. The right example shows the same scene reconstructed using only 4 input views.

Figure 3: Architecture of Free Geometry. The test sequence is processed in two configurations. Top : the full observation (all views, e.g. 8 views) passes through the Image Patch Embedding (e.g. DINOv2 [ 9 ] ), the Multi-view Transformer, a randomized camera token, and encodes the views into feature representations. All encoders are frozen ( gray ). Bottom : the partial observation (half of views masked, e.g. 4 views) passes through the same frozen backbone ( gray ) with LoRA applied to multi-view transformer and the camera token is trainable ( orange ). All feature tokens of decoder input are extracted from both branches.

Figure 4: Self-Supervised Geometric Losses of Free Geometry: Left illustrates the intra-frame consistency loss for aligning teacher and student features at corresponding image token locations. Right illustrates the cross-frame relational loss for preserving geometric relations between unmasked tokens and masked-frame anchors. The two losses jointly recalibrate the student representation to stronger full-observation features.

#View	Method	ETH3D		ScanNet++		7-Scenes		HiRoom
#View	Method	AUC3 $\uparrow$	F1 $\uparrow$	AUC3 $\uparrow$	F1 $\uparrow$	AUC3 $\uparrow$	F1 $\uparrow$	AUC3 $\uparrow$	F1 $\uparrow$
4	VGGT	0.157	0.102	0.408	0.171	0.238	0.196	0.421	0.276
	VGGT+Free Geo	0.178	0.110	0.419	0.174	0.241	0.197	0.441	0.307
	DA3	0.286	0.207	0.620	0.236	0.280	0.244	0.708	0.557
	DA3+Free Geo	0.305	0.209	0.624	0.239	0.302	0.248	0.719	0.578
8	VGGT	0.207	0.301	0.496	0.326	0.252	0.329	0.516	0.502

Table 2: Free Geometry 3D Reconstruction Comparison: We report pose accuracy (AUC3

\uparrow

) and reconstruction F1-score (F1

\uparrow

). Each cell reports the mean over 3 seeds. Bold indicates the better result within each method pair.

Figure 5: Qualitative Results On Multi-view Depth. We extract the key frames from multi-view reconstruction depth outputs. In the error maps, red pixels mark regions where the model’s depth prediction deviates significantly from the ground truth, and gray pixels represent correctly reconstructed surfaces within threshold.

Figure 6: Qualitative Results on 3D Reconstruction. Free Geometry consistently improves geometry quality and reduce errors compared to baseline models. Red pixels mark regions where the model’s prediction deviates significantly from the ground truth, and gray pixels represent correctly reconstructed surfaces within threshold.

#View	VGGT w/. Free Geo				DA3 w/. Free Geo
#View	AUC3 $\uparrow$	AUC30 $\uparrow$	F1 $\uparrow$	CD $\downarrow$	AUC3 $\uparrow$	AUC30 $\uparrow$	F1 $\uparrow$	CD $\downarrow$
4	+5.33%	+4.73%	+4.51%	-8.15%	+2.74%	+1.03%	+2.85%	-4.03%
8	+2.19%	+1.35%	+4.32%	-3.53%	+1.82%	+0.57%	+0.21%	-0.35%
16	+3.93%	+1.33%	+3.03%	-5.35%	+2.61%	+0.89%	+0.70%	-7.67%
32	+3.73%	+1.03%	+2.88%	-4.34%	+2.89%	+0.94%	+0.78%	-8.73%

Table 3: Free Geo cross-view relative improvements (%): Each entry is the relative change of Free Geo over the corresponding baseline, averaged over seeds 43/44/45 across 4 datasets. We report pose accuracy (Auc3

\uparrow

and Auc30

\uparrow

) and reconstruction F1-score (F1

\uparrow

) and Chamfer Distance (CD

\downarrow

Configuration	AUC3 $\uparrow$	AUC30 $\uparrow$	F1 $\uparrow$	CD $\downarrow$
w/o Consistency Loss	35.87	72.12	0.2324	3.6976
w/o Relational Loss	36.37	72.22	0.2190	3.9567
Full Loss (Free Geo)	37.88	72.32	0.2475	3.5473

Table 4: Ablations for Loss Components. We run the identical test-time adaptation settings over same scenes on ETH3D dataset and selected images with loss difference only. Benchmark results illustrate both loss components are critical for optimizing feature representations.

	VGGT		DA3
Method	MSE $\downarrow$	Cosine $\uparrow$	MSE $\downarrow$	Cosine $\uparrow$
Base Model	0.9039	0.8581	45.9079	0.8605
Base Model + free Geo	0.8384	0.8684	42.8949	0.8679

Table 5: Feature Consistency Comparison: We evaluate encoder’s last-layer feature consistency over ETH3D datasets. Reference features are the unmasked 4 view features of full observation (8 views reconstruction); reported values are scene means for VGGT layer 23 and DA3 layer 39.

Dataset	Samples/scene	Training Epochs	Learning Rate Range
ETH3D	10	3	[ $1e$ -7, $1e$ -4]
7Scenes	20	3	[ $1e$ -8, $5e$ -5]
ScanNet++	10	3	[ $1e$ -8, $1e$ -5]
HiRoom	5	5	[ $1e$ -7, $1e$ -4]

Table 1: Free Geometry test-time adaptation settings for four datasets.

Selection Strategy	Pose Results		Recon Results
Selection Strategy	AUC@3 $\uparrow$	AUC@30 $\uparrow$	F-score $\uparrow$	Overall $\downarrow$
Top selection	36.87	72.22	0.2474	3.4653
Random selection	37.37	72.32	0.2418	3.5775
Mixed selection (Free Geo)	37.88	72.32	0.2475	3.5473

Table 2: Cross-frame Relational Loss Selection Strategy Comparison: We report the results on ETH3D with default training settings, where only the cross-frame patch selection strategy differs. Top selection uses top

K

most similar patches from masked frames. Mixed selection uses top-

K/2

most similar and top-

K/2

least similar patches from masked frames. he best results for each metrics are bold .

Variant	Trainable	Pose Results		Reconstruction Results
Variant	Trainable	Auc3 $\uparrow$	Auc30 $\uparrow$	F1 $\uparrow$	CD $\downarrow$
Baseline	0	0.514	0.906	0.730	0.634
LoRA r=8	5.311M	0.566	0.919	0.747	0.622
LoRA r=16	10.620M	0.563	0.920	0.732	0.644
LoRA r=32	21.237M	0.581	0.923	0.755	0.625
LoRA r=64	42.470M	0.556	0.922	0.698	0.637

Table 3: LoRA Rank Comparison. We compare the baseline model and LoRA variants with different ranks trainable parameter sizes on pose and reconstruction metrics on ETH3D dataset on Depth Anything 3. The best results for each metrics are bold .

View	Method	ETH3D		ScanNet++		7-Scenes		HiRoom
View	Method	Auc3 $\uparrow$	Auc30 $\uparrow$	Auc3 $\uparrow$	Auc30 $\uparrow$	Auc3 $\uparrow$	Auc30 $\uparrow$	Auc3 $\uparrow$	Auc30 $\uparrow$
4	VGGT	0.157	0.538	0.408	0.805	0.238	0.828	0.421	0.738
	VGGT+Free Geo	0.178	0.557	0.419	0.812	0.241	0.829	0.441	0.802
	DA3	0.286	0.675	0.620	0.798	0.280	0.817	0.708	0.925
	DA3+Free Geo	0.305	0.687	0.624	0.799	0.302	0.819	0.719	0.939
8	VGGT	0.207	0.666	0.496	0.886	0.252	0.822	0.516	0.864

Table 4: Free Geometry Pose Comparison: We report pose accuracy with AUC@3

\uparrow

and AUC@30

\uparrow

. Bold are the better result within baseline/Free Geo pair.

View	Method	ETH3D		ScanNet++		7-Scenes		HiRoom
View	Method	F1 $\uparrow$	CD $\downarrow$	F1 $\uparrow$	CD $\downarrow$	F1 $\uparrow$	CD $\downarrow$	F1 $\uparrow$	CD $\downarrow$
4	VGGT	0.102	3.624	0.171	0.536	0.196	0.351	0.276	0.420
	VGGT+Free Geo	0.110	4.097	0.174	0.535	0.197	0.351	0.307	0.331
	DA3	0.207	2.742	0.236	0.546	0.244	0.471	0.557	0.176
	DA3+Free Geo	0.209	2.754	0.239	0.543	0.248	0.481	0.578	0.157
8	VGGT	0.301	1.460	0.326	0.337	0.329	0.226	0.502	0.142

Table 5: Free Geometry Reconstruction Comparison: We report reconstruction F1-score

\uparrow

and Chamfer Distance (CD)

\downarrow

. Bold indicates the better result within each baseline/Free Geo pair.

Figure 1: Qualitative Results on 3D Reconstruction. In the error maps, red pixels denote regions whose reconstructed geometry deviates significantly from the ground truth, while gray pixels indicate regions that fall within the evaluation threshold.

Figure 2: Qualitative Results on Multi-view Depth. We visualize representative key frames from multi-view depth reconstruction. In the error maps, red pixels mark regions where the predicted depth deviates significantly from the ground truth, and gray pixels indicate pixels whose depth agrees with the ground truth within threshold.

Explore similar work

FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views
Apr 10, 2026Chaoyi Zhou, Run Wang, Feng Luo +43D Reconstruction Feed-Forward 3D Reconstruction
Microsoft · Clemson University · Texas A&M University
From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis
Mar 29, 2026Ranran Huang, Weixun Luo, Ye Mao +13D Reconstruction Multi-View 3D Reconstruction
Imperial College London
GGPT: Geometry Grounded Point Transformer
Mar 11, 2026Yutong Chen, Yiming Wang, Xucong Zhang +2Sparse-View 3D Reconstruction Feed-Forward 3D Reconstruction
ETH Zurich · Delft University of Technology

Apr 10, 2026J/K move · Enter open · S save

FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views

Chaoyi Zhou, Run Wang, Feng Luo, Mert D. Pesé+3

Microsoft · Clemson University · Texas A&M University

3D Reconstruction Feed-Forward 3D Reconstruction Multi-View 3D Reconstruction Open-Vocabulary 3D Segmentation

Open paper AskPDF arXiv

Recent advances in vision foundation models have revolutionized geometry reconstruction and semantic understanding. Yet, most of the existing approaches treat these capabilities in isolation, leading to redundant pipelines and compounded errors. This paper introduces FF3R, a fully annotation-free feed-forward framework that unifies geometric and semantic reasoning from unconstrained multi-view image sequences. Unlike previous methods, FF3R does not require camera poses, depth maps, or semantic labels, relying solely on rendering supervision for RGB and feature maps, establishing a scalable paradigm for unified 3D reasoning. In addition, we address two critical challenges in feedforward feature reconstruction pipelines, namely global semantic inconsistency and local structural inconsistency, through two key innovations: (i) a Token-wise Fusion Module that enriches geometry tokens with semantic context via cross-attention, and (ii) a Semantic-Geometry Mutual Boosting mechanism combining geometry-guided feature warping for global consistency with semantic-aware voxelization for local coherence. Extensive experiments on ScanNet and DL3DV-10K demonstrate FF3R's superior performance in novel-view synthesis, open-vocabulary semantic segmentation, and depth estimation, with strong generalization to in-the-wild scenarios, paving the way for embodied intelligence systems that demand both spatial and semantic understanding.

Abstract

Explore similar work

FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views

From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis

GGPT: Geometry Grounded Point Transformer