VGGT-Align: Bridging Local Reconstruction and Global Consistency for Long-Sequence 3D Reconstruction

Wei Zhang; Yihang Wu; Songhua Li; Qi Wang

cs.CVAug 15, 2026

VGGT-Align: Bridging Local Reconstruction and Global Consistency for Long-Sequence 3D Reconstruction

Authors: Wei Zhang, Yihang Wu, Songhua Li, Qi Wang

Organizations: School of Computer Science Northwestern Polytechnical University Xi’an, China

Abstract

Maintaining global geometric consistency is a central challenge in long-sequence 3D reconstruction, with scale drift being the most critical failure mode. In chunk-based inference pipelines, the scale degree of freedom in sequential Sim(3) alignment is left unconstrained, causing estimation errors to compound multiplicatively and distort global trajectories and point cloud geometry. We present a scale-consistency enhancement framework built on a key insight: in structured environments such as driving scenes, geometric quantities arising from environmental regularity remain inherently invariant across temporal segments, and discrepancies in their per-chunk measurements directly expose inter-chunk scale drift. We propose Scene Geometric Invariant Anchoring (SGIA), which extracts dominant geometric invariants from each chunk's predicted point cloud via coarse-to-fine robust estimation and exploits their cross-chunk consistency to establish scale constraints independent of point cloud registration, explicitly degenerating 7-DoF Sim(3) alignment into 6-DoF rigid-body transformation and severing chain-wise scale error propagation at its source. We further introduce a lightweight test-time adaptation strategy that fine-tunes only normalization-layer parameters via multi-objective self-supervision, progressively improving intra-chunk predictions along the sequence. Both modules are plug-and-play and require no offline retraining. Experiments on multiple long-sequence benchmarks demonstrate state-of-the-art performance, reducing absolute trajectory error by up to 32% with significant gains in trajectory stability and reconstruction quality. Code: https://github.com/WZ-CS/VGGT-Align

Figures & tables

Figure 1. Comparison of chunk-based long-sequence reconstruction on KITTI. Existing baselines (a, b) suffer from unconstrained scale drift in Sim(3) alignment, manifesting as scale drift (trajectory shrinks into a distorted cluster) and drift accumulation (trajectory diverges from ground truth). (c) VGGT-Align maintains scale consistency by anchoring each chunk via scene geometric invariants, reducing ATE by 30% on Seq 08 and 45% on Seq 05. Right: inter-chunk point cloud overlay shows our scale correction preserves geometric consistency between adjacent chunk (top, ✓), while the baseline exhibits severe drift at chunk boundaries (bottom, ✗). VGGT-Align achieves state-of-the-art among chunk-based methods. A comparison of long-sequence 3D reconstruction methods on KITTI. Baseline trajectories exhibit accumulated scale drift and geometric misalignment at chunk boundaries, whereas VGGT-Align remains close to the ground-truth trajectory and preserves overlap between adjacent reconstructed point-cloud chunks.

Figure 2. Qualitative analysis of camera pose estimation on KITTI Seq 08. (a) Bird’s-eye view trajectory comparison. The baseline exhibits significant scale drift in the later portion of the sequence, causing the estimated trajectory to deviate substantially from the ground truth, while our method maintains closer agreement throughout by introducing global scale prior constraints during cross-chunk alignment. (b) Per-chunk scale consistency after Sim(3) alignment. Each point represents the local path length ratio (predicted / ground truth) within a chunk of 75 frames. $\sigma$ denotes its standard deviation across all chunks. The baseline suffers progressive scale compression, dropping to approximately 0.5 by the end of the sequence ( $\sigma$ =0.157), while our method maintains the ratio near the ideal value of 1.0 ( $\sigma$ =0.065).

Figure 3. Per-chunk scale ratio analysis across three KITTI sequences of varying length. The ideal ratio is 1.0 (dashed line); the green band denotes $\pm$ 5% tolerance. The baseline (red) exhibits systematic scale drift with high variance, while VGGT-Align (blue) maintains ratios tightly around 1.0 with consistently lower variance (e.g., $\sigma$ : 0.184 $\to$ 0.079 on Seq 10, 0.064 $\to$ 0.030 on Seq 04). Inset trajectory plots show corresponding ATE improvements. On Seq 02, scale correction is visibly effective ( $\sigma$ : 0.178 $\to$ 0.097), though the trajectory exhibits a slight positional offset due to residual rotational drift in this 5 km loop; see supplementary material for extended visualizations.

Figure 4. Overview of the VGGT-Align framework. A long RGB sequence is partitioned into overlapping chunks and processed independently by a feed-forward 3D model. Our two plug-and-play contributions are highlighted: (1) Scene Geometric Invariant Anchoring (SGIA, green, top-right) extracts dominant geometric structures from each chunk’s predicted point cloud via region selection with confidence filtering, followed by coarse-to-fine plane estimation (RANSAC + SVD). Two complementary invariant types are exploited: vertical structures (e.g., camera-to-ground distance $g_{k}$ ) and horizontal structures (e.g., road width $w_{k}$ ), whose cross-chunk measurement ratios yield per-pair scale constraints $s_{\text{sgia}}$ independent of point cloud registration. This constraint is injected into the overlap-based IRLS alignment, explicitly replacing the estimated scale factor (lock symbol) and degenerating the 7-DoF Sim(3) transformation into a 6-DoF rigid-body transform SE(3), thereby severing chain-wise scale drift at its source. (2) Test-time adaptation (TTA, pink, bottom-left) fine-tunes only normalization-layer parameters using three self-supervised objectives: photometric consistency, geometric alignment, and temporal smoothness, with the adapted parameters improving predictions for subsequent chunks (dashed feedback arrow). The aligned chunks are further refined by loop closure optimization to produce the final globally consistent trajectory and dense point cloud.

Methods	LC	Calibration	Recon.	Avg.	00	01	02	03	04	05	06	07	08	09	10
seq. frames	-	-	-	2109	4542	1101	4661	801	271	2761	1101	1101	4071	1591	1201
seq. length (m)	-	-	-	2012	3724	2453	5067	561	394	2206	1233	650	3223	1705	920
contains loop	-	-	-	-	✓	✗	✓	✗	✗	✓	✓	✓	✗	✓	✗
DROID-VO ( Teed and Deng, 2021 )	✗	Required	Dense	54.19	98.43	84.20	108.80	2.58	0.93	59.27	64.40	24.20	64.55	71.80	16.91
DPVO ( Teed et al., 2024 )	✗	Required	Sparse	53.61	113.21	12.69	123.40	2.09	0.68	58.96	54.78	19.26	115.90	75.10	13.63
DROID-SLAM ( Teed and Deng, 2021 )	-	Required	Dense	100.28	92.10	344.60	107.61	2.38	1.00	118.50	62.47	21.78	161.60	72.32	118.70

Table 1. Camera tracking results (ATE RMSE [m]

\downarrow

) on the KITTI Odometry benchmark. LC : loop closure capability. VGGT-Align achieves the best overall accuracy among all calibration-free or calibration-required methods. OOM : CUDA Out-Of-Memory on a single RTX 4090. TL : Tracking Lost. Color: 1st , 2nd , 3rd .

Methods	Calib.	Avg.	163453191	183829460	315615587	346181117	371159869	405841035	460417311	520018670	610454533
Frames / Length (m)	-	198 / 173	198 / 160	199 / 42	199 / 165	199 / 351	196 / 273	199 / 86	198 / 266	199 / 135	198 / 63
DROID-SLAM ( Teed and Deng, 2021 )	Required	4.396	3.705	0.301	0.447	8.653	9.320	7.621	4.170	TL	0.264
MASt3R-SLAM ( Murai et al., 2025 )	No Need	5.560	4.500	0.556	1.833	12.544	8.601	1.412	5.428	7.910	1.195
CUT3R ( Wang et al., 2025b )	No Need	9.872	8.781	3.810	5.790	24.015	13.070	7.261	13.206	8.597	3.229
Fast3R ( Yang et al., 2025 )	No Need	/	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
VGGT ( Wang et al., 2025a )	No Need	/	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM

Table 2. Camera tracking results (ATE RMSE [m]

\downarrow

) on the Waymo Open Dataset. VGGT-Align achieves the best average accuracy among all calibration-free methods and surpasses calibration-required DROID-SLAM. Color: 1st , 2nd , 3rd .

Figure 5. Qualitative comparison of trajectories and dense reconstructions across four KITTI sequences. Columns 1–4: DROID-SLAM, VGGT-Long, SwiftVGGT, and VGGT-Align (Ours); ground truth in gray dashed, predictions in green, red boxes highlight deviations. Column 5: LiDAR pseudo ground-truth point cloud. VGGT-Align achieves the lowest ATE on all four sequences with notable improvements on Seq 06 (62.47 $\to$ 4.21), Seq 09 (72.32 $\to$ 23.11), and Seq 10 (118.70 $\to$ 11.66).

Figure 6. Dense point cloud comparison on KITTI Seq 05. Top: reference frames (#2580, #2588). (a) VGGT-Long: misaligned surfaces, duplicated structures, and scattered outliers at chunk boundaries. (b) VGGT-Align: coherent reconstruction with continuous surfaces and sharp edges from SGIA.

	Calib.	Scene 20 837 frames, 711 m
Condition	-	Clone	Fog	Morning	Overcast	Rain	Sunset
DROID-SLAM ( Teed and Deng, 2021 )	Required	3.59	5.08	3.73	3.85	3.78	4.91
MASt3R-SLAM ( Murai et al., 2025 )	No Need	TL	TL	TL	TL	TL	TL
CUT3R ( Wang et al., 2025b )	No Need	129.50	76.96	117.95	114.51	66.70	116.53
Fast3R ( Yang et al., 2025 )	No Need	OOM	OOM	OOM	OOM	OOM	OOM
VGGT ( Wang et al., 2025a )	No Need	OOM	OOM	OOM	OOM	OOM	OOM

Table 3. Camera tracking results (ATE RMSE [m]

\downarrow

) on Virtual KITTI Scene 20 under six appearance conditions. VGGT-Align significantly narrows the gap to calibration-required DROID-SLAM and surpasses it under Morning and Sunset.

Methods	Metric	Calib.	Avg.	163453191	183829460	315615587	346181117	371159869	405841035	460417311	520018670	610454533
Frames / Length (m)	-	-	198 / 173	198 / 160	199 / 42	199 / 165	199 / 351	196 / 273	199 / 86	198 / 266	199 / 135	198 / 63
DROID-SLAM ( Teed and Deng, 2021 )	Accuracy $\downarrow$	Req.	1.201	0.781	1.136	2.247	2.393	1.090	0.539	0.740	TL	0.677
	Completeness $\downarrow$		8.540	4.610	10.245	5.540	8.669	8.592	11.144	5.320	TL	14.201
	Chamfer $\downarrow$		4.870	2.696	5.691	3.893	5.531	4.841	5.842	3.030	TL	7.439
MASt3R-SLAM ( Murai et al., 2025 )	Accuracy $\downarrow$	No	3.772	3.189	2.988	3.787	4.689	4.436	1.166	4.637	6.417	2.637
	Completeness $\downarrow$		3.177	1.715	3.284	2.047	2.981	2.679	2.895	2.002	4.429	6.560

Table 4. Dense reconstruction results on the Waymo Open Dataset. Metrics are Accuracy, Completeness, and Chamfer Distance (all in meters

\downarrow

) against LiDAR ground truth. Note that LiDAR has a narrower vertical FoV than RGB cameras, so metrics should be interpreted alongside qualitative results. Color: 1st , 2nd , 3rd .

GP	RW	AB	TTA	Avg.	163..	183..	315..	346..	371..	405..	460..	520..	610..
✗	✗	✗	✗	2.154	1.780	2.570	0.760	3.730	3.225	1.390	1.690	3.477	0.355
✓	✗	✗	✗	2.442	3.930	2.670	0.588	2.790	3.560	1.120	2.460	2.060	2.800
✓	✗	✓	✗	2.173	2.540	2.600	0.616	3.360	3.320	1.190	1.830	2.060	2.040
✓	✓	✓	✗	1.856	1.290	2.550	0.495	3.280	2.960	0.780	1.530	2.190	1.629
✓	✓	✓	✓	1.849	1.160	2.540	0.537	3.140	2.850	1.020	1.520	2.170	1.708

Table 5. Ablation on Waymo (ATE RMSE [m]

\downarrow

). Row 1: VGGT-Long baseline. GP : Ground plane Prior. RW : Road Width prior. AB : Adaptive Blending. TTA : Test-Time Adaptation. Row 5: full VGGT-Align. Color: 1st , 2nd , 3rd .

Method	VGGT	Align	SGIA	TTA	Total
	fwd	(IRLS)
VGGT-Long	0.68 min	1.05 min	—	—	$\sim$ 2.56 min
Ours (SGIA)	0.63 min	0.89 min	0.14 min	—	$\sim$ 2.31 min
Ours (full)	0.62 min	0.88 min	0.13 min	0.07 min	$\sim$ 2.32 min

Table 6. Runtime breakdown on the Waymo Open Dataset. See Appendix for detailed analysis.

Explore similar work

GeoWeaver: Accurate Long-Sequence 3D Reconstruction via Hierarchical Geometric Assembly
Aug 18, 2026Tinghao Jiang, Sheng Tang, Shengzhe Wei +43D Reconstruction Feed-Forward 3D Reconstruction
Kosmo Research · Shanghai Jiao Tong University · Tsinghua University
Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction
Sep 18, 2026Jingke Zhou, Chenhang Ma, Zhizhou Zhong +63D Reconstruction Streaming 3D Reconstruction
Zhejiang University · Fudan University · Peking University +1
VGGT-Bridge: Beyond Sequential Pose Graphs via Coarse-Stride Skip Edges
Oct 5, 2026Sungjae Choi, Hanna Bae, Sunghyun Baek +13D ViTs Streaming 3D Reconstruction
Korea Advanced Institute of Science and Technology, South Korea

Aug 18, 2026J/K move · Enter open · S save

GeoWeaver: Accurate Long-Sequence 3D Reconstruction via Hierarchical Geometric Assembly

Tinghao Jiang, Sheng Tang, Shengzhe Wei, Juntong Fang+3

Kosmo Research · Shanghai Jiao Tong University · Tsinghua University

3D Reconstruction Feed-Forward 3D Reconstruction Camera Pose Estimation Test-Time Adaptation

Open paper AskPDF arXiv

Long-sequence 3D reconstruction from RGB videos requires both accurate local geometry and globally consistent camera motion. Feed-forward models provide strong depth and pose predictions, but their memory cost prevents joint inference over long sequences. Chunk-wise processing improves scalability, yet independently predicted chunks often exhibit scale drift, pose errors, and point-cloud misalignment. We present GeoWeaver, a unified framework comprising a Geometric Prior Model (GPM) and Test-Time Adaptation (TTA). The GPM predicts chunk-wise depth, confidence, and camera parameters as adjustable geometric priors. TTA then performs sequential initialization, global chunk-level Sim(3) alignment, and coarse-to-fine refinement of camera poses, affine depth corrections, and intrinsics. Dense correspondences provide adjacent, cross-chunk, and long-range constraints, while a robust CDF-style objective jointly optimizes weighted 2D reprojection and 3D consistency residuals. This design preserves local geometric accuracy while correcting accumulated pose, scale, depth, and calibration errors. Experiments across diverse long-sequence benchmarks demonstrate improved camera accuracy, global consistency, and point-cloud quality. Ablations verify the contribution of each adaptation stage, and applying the same TTA procedure to different geometric prior models consistently improves their trajectory estimates, demonstrating that GeoWeaver is not tied to a specific GPM.

Abstract

Explore similar work

GeoWeaver: Accurate Long-Sequence 3D Reconstruction via Hierarchical Geometric Assembly

Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction

VGGT-Bridge: Beyond Sequential Pose Graphs via Coarse-Stride Skip Edges