VGGT-Align: Bridging Local Reconstruction and Global Consistency for Long-Sequence 3D Reconstruction

Wei Zhang; Yihang Wu; Songhua Li; Qi Wang

cs.CVAug 15, 2026

VGGT-Align: Bridging Local Reconstruction and Global Consistency for Long-Sequence 3D Reconstruction

Authors: Wei Zhang, Yihang Wu, Songhua Li, Qi Wang

Organizations: School of Computer Science Northwestern Polytechnical University Xi’an, China

Abstract

Maintaining global geometric consistency is a central challenge in long-sequence 3D reconstruction, with scale drift being the most critical failure mode. In chunk-based inference pipelines, the scale degree of freedom in sequential Sim(3) alignment is left unconstrained, causing estimation errors to compound multiplicatively and distort global trajectories and point cloud geometry. We present a scale-consistency enhancement framework built on a key insight: in structured environments such as driving scenes, geometric quantities arising from environmental regularity remain inherently invariant across temporal segments, and discrepancies in their per-chunk measurements directly expose inter-chunk scale drift. We propose Scene Geometric Invariant Anchoring (SGIA), which extracts dominant geometric invariants from each chunk's predicted point cloud via coarse-to-fine robust estimation and exploits their cross-chunk consistency to establish scale constraints independent of point cloud registration, explicitly degenerating 7-DoF Sim(3) alignment into 6-DoF rigid-body transformation and severing chain-wise scale error propagation at its source. We further introduce a lightweight test-time adaptation strategy that fine-tunes only normalization-layer parameters via multi-objective self-supervision, progressively improving intra-chunk predictions along the sequence. Both modules are plug-and-play and require no offline retraining. Experiments on multiple long-sequence benchmarks demonstrate state-of-the-art performance, reducing absolute trajectory error by up to 32% with significant gains in trajectory stability and reconstruction quality. Code: https://github.com/WZ-CS/VGGT-Align

Figures & tables

Figure 1. Comparison of chunk-based long-sequence reconstruction on KITTI. Existing baselines (a, b) suffer from unconstrained scale drift in Sim(3) alignment, manifesting as scale drift (trajectory shrinks into a distorted cluster) and drift accumulation (trajectory diverges from ground truth). (c) VGGT-Align maintains scale consistency by anchoring each chunk via scene geometric invariants, reducing ATE by 30% on Seq 08 and 45% on Seq 05. Right: inter-chunk point cloud overlay shows our scale correction preserves geometric consistency between adjacent chunk (top, ✓), while the baseline exhibits severe drift at chunk boundaries (bottom, ✗). VGGT-Align achieves state-of-the-art among chunk-based methods. A comparison of long-sequence 3D reconstruction methods on KITTI. Baseline trajectories exhibit accumulated scale drift and geometric misalignment at chunk boundaries, whereas VGGT-Align remains close to the ground-truth trajectory and preserves overlap between adjacent reconstructed point-cloud chunks.

Figure 2. Qualitative analysis of camera pose estimation on KITTI Seq 08. (a) Bird’s-eye view trajectory comparison. The baseline exhibits significant scale drift in the later portion of the sequence, causing the estimated trajectory to deviate substantially from the ground truth, while our method maintains closer agreement throughout by introducing global scale prior constraints during cross-chunk alignment. (b) Per-chunk scale consistency after Sim(3) alignment. Each point represents the local path length ratio (predicted / ground truth) within a chunk of 75 frames. $\sigma$ denotes its standard deviation across all chunks. The baseline suffers progressive scale compression, dropping to approximately 0.5 by the end of the sequence ( $\sigma$ =0.157), while our method maintains the ratio near the ideal value of 1.0 ( $\sigma$ =0.065).

Figure 3. Per-chunk scale ratio analysis across three KITTI sequences of varying length. The ideal ratio is 1.0 (dashed line); the green band denotes $\pm$ 5% tolerance. The baseline (red) exhibits systematic scale drift with high variance, while VGGT-Align (blue) maintains ratios tightly around 1.0 with consistently lower variance (e.g., $\sigma$ : 0.184 $\to$ 0.079 on Seq 10, 0.064 $\to$ 0.030 on Seq 04). Inset trajectory plots show corresponding ATE improvements. On Seq 02, scale correction is visibly effective ( $\sigma$ : 0.178 $\to$ 0.097), though the trajectory exhibits a slight positional offset due to residual rotational drift in this 5 km loop; see supplementary material for extended visualizations.

Figure 4. Overview of the VGGT-Align framework. A long RGB sequence is partitioned into overlapping chunks and processed independently by a feed-forward 3D model. Our two plug-and-play contributions are highlighted: (1) Scene Geometric Invariant Anchoring (SGIA, green, top-right) extracts dominant geometric structures from each chunk’s predicted point cloud via region selection with confidence filtering, followed by coarse-to-fine plane estimation (RANSAC + SVD). Two complementary invariant types are exploited: vertical structures (e.g., camera-to-ground distance $g_{k}$ ) and horizontal structures (e.g., road width $w_{k}$ ), whose cross-chunk measurement ratios yield per-pair scale constraints $s_{\text{sgia}}$ independent of point cloud registration. This constraint is injected into the overlap-based IRLS alignment, explicitly replacing the estimated scale factor (lock symbol) and degenerating the 7-DoF Sim(3) transformation into a 6-DoF rigid-body transform SE(3), thereby severing chain-wise scale drift at its source. (2) Test-time adaptation (TTA, pink, bottom-left) fine-tunes only normalization-layer parameters using three self-supervised objectives: photometric consistency, geometric alignment, and temporal smoothness, with the adapted parameters improving predictions for subsequent chunks (dashed feedback arrow). The aligned chunks are further refined by loop closure optimization to produce the final globally consistent trajectory and dense point cloud.

Methods	LC	Calibration	Recon.	Avg.	00	01	02	03	04	05	06	07	08	09	10
seq. frames	-	-	-	2109	4542	1101	4661	801	271	2761	1101	1101	4071	1591	1201
seq. length (m)	-	-	-	2012	3724	2453	5067	561	394	2206	1233	650	3223	1705	920
contains loop	-	-	-	-	✓	✗	✓	✗	✗	✓	✓	✓	✗	✓	✗
DROID-VO ( Teed and Deng, 2021 )	✗	Required	Dense	54.19	98.43	84.20	108.80	2.58	0.93	59.27	64.40	24.20	64.55	71.80	16.91
DPVO ( Teed et al., 2024 )	✗	Required	Sparse	53.61	113.21	12.69	123.40	2.09	0.68	58.96	54.78	19.26	115.90	75.10	13.63
DROID-SLAM ( Teed and Deng, 2021 )	-	Required	Dense	100.28	92.10	344.60	107.61	2.38	1.00	118.50	62.47	21.78	161.60	72.32	118.70

Table 1. Camera tracking results (ATE RMSE [m]

\downarrow

) on the KITTI Odometry benchmark. LC : loop closure capability. VGGT-Align achieves the best overall accuracy among all calibration-free or calibration-required methods. OOM : CUDA Out-Of-Memory on a single RTX 4090. TL : Tracking Lost. Color: 1st , 2nd , 3rd .

Methods	Calib.	Avg.	163453191	183829460	315615587	346181117	371159869	405841035	460417311	520018670	610454533
Frames / Length (m)	-	198 / 173	198 / 160	199 / 42	199 / 165	199 / 351	196 / 273	199 / 86	198 / 266	199 / 135	198 / 63
DROID-SLAM ( Teed and Deng, 2021 )	Required	4.396	3.705	0.301	0.447	8.653	9.320	7.621	4.170	TL	0.264
MASt3R-SLAM ( Murai et al., 2025 )	No Need	5.560	4.500	0.556	1.833	12.544	8.601	1.412	5.428	7.910	1.195
CUT3R ( Wang et al., 2025b )	No Need	9.872	8.781	3.810	5.790	24.015	13.070	7.261	13.206	8.597	3.229
Fast3R ( Yang et al., 2025 )	No Need	/	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
VGGT ( Wang et al., 2025a )	No Need	/	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM

Table 2. Camera tracking results (ATE RMSE [m]

\downarrow

) on the Waymo Open Dataset. VGGT-Align achieves the best average accuracy among all calibration-free methods and surpasses calibration-required DROID-SLAM. Color: 1st , 2nd , 3rd .

Figure 5. Qualitative comparison of trajectories and dense reconstructions across four KITTI sequences. Columns 1–4: DROID-SLAM, VGGT-Long, SwiftVGGT, and VGGT-Align (Ours); ground truth in gray dashed, predictions in green, red boxes highlight deviations. Column 5: LiDAR pseudo ground-truth point cloud. VGGT-Align achieves the lowest ATE on all four sequences with notable improvements on Seq 06 (62.47 $\to$ 4.21), Seq 09 (72.32 $\to$ 23.11), and Seq 10 (118.70 $\to$ 11.66).

Figure 6. Dense point cloud comparison on KITTI Seq 05. Top: reference frames (#2580, #2588). (a) VGGT-Long: misaligned surfaces, duplicated structures, and scattered outliers at chunk boundaries. (b) VGGT-Align: coherent reconstruction with continuous surfaces and sharp edges from SGIA.

	Calib.	Scene 20 837 frames, 711 m
Condition	-	Clone	Fog	Morning	Overcast	Rain	Sunset
DROID-SLAM ( Teed and Deng, 2021 )	Required	3.59	5.08	3.73	3.85	3.78	4.91
MASt3R-SLAM ( Murai et al., 2025 )	No Need	TL	TL	TL	TL	TL	TL
CUT3R ( Wang et al., 2025b )	No Need	129.50	76.96	117.95	114.51	66.70	116.53
Fast3R ( Yang et al., 2025 )	No Need	OOM	OOM	OOM	OOM	OOM	OOM
VGGT ( Wang et al., 2025a )	No Need	OOM	OOM	OOM	OOM	OOM	OOM

Table 3. Camera tracking results (ATE RMSE [m]

\downarrow

) on Virtual KITTI Scene 20 under six appearance conditions. VGGT-Align significantly narrows the gap to calibration-required DROID-SLAM and surpasses it under Morning and Sunset.

Methods	Metric	Calib.	Avg.	163453191	183829460	315615587	346181117	371159869	405841035	460417311	520018670	610454533
Frames / Length (m)	-	-	198 / 173	198 / 160	199 / 42	199 / 165	199 / 351	196 / 273	199 / 86	198 / 266	199 / 135	198 / 63
DROID-SLAM ( Teed and Deng, 2021 )	Accuracy $\downarrow$	Req.	1.201	0.781	1.136	2.247	2.393	1.090	0.539	0.740	TL	0.677
	Completeness $\downarrow$		8.540	4.610	10.245	5.540	8.669	8.592	11.144	5.320	TL	14.201
	Chamfer $\downarrow$		4.870	2.696	5.691	3.893	5.531	4.841	5.842	3.030	TL	7.439
MASt3R-SLAM ( Murai et al., 2025 )	Accuracy $\downarrow$	No	3.772	3.189	2.988	3.787	4.689	4.436	1.166	4.637	6.417	2.637
	Completeness $\downarrow$		3.177	1.715	3.284	2.047	2.981	2.679	2.895	2.002	4.429	6.560

Table 4. Dense reconstruction results on the Waymo Open Dataset. Metrics are Accuracy, Completeness, and Chamfer Distance (all in meters

\downarrow

) against LiDAR ground truth. Note that LiDAR has a narrower vertical FoV than RGB cameras, so metrics should be interpreted alongside qualitative results. Color: 1st , 2nd , 3rd .

GP	RW	AB	TTA	Avg.	163..	183..	315..	346..	371..	405..	460..	520..	610..
✗	✗	✗	✗	2.154	1.780	2.570	0.760	3.730	3.225	1.390	1.690	3.477	0.355
✓	✗	✗	✗	2.442	3.930	2.670	0.588	2.790	3.560	1.120	2.460	2.060	2.800
✓	✗	✓	✗	2.173	2.540	2.600	0.616	3.360	3.320	1.190	1.830	2.060	2.040
✓	✓	✓	✗	1.856	1.290	2.550	0.495	3.280	2.960	0.780	1.530	2.190	1.629
✓	✓	✓	✓	1.849	1.160	2.540	0.537	3.140	2.850	1.020	1.520	2.170	1.708

Table 5. Ablation on Waymo (ATE RMSE [m]

\downarrow

). Row 1: VGGT-Long baseline. GP : Ground plane Prior. RW : Road Width prior. AB : Adaptive Blending. TTA : Test-Time Adaptation. Row 5: full VGGT-Align. Color: 1st , 2nd , 3rd .

Method	VGGT	Align	SGIA	TTA	Total
	fwd	(IRLS)
VGGT-Long	0.68 min	1.05 min	—	—	$\sim$ 2.56 min
Ours (SGIA)	0.63 min	0.89 min	0.14 min	—	$\sim$ 2.31 min
Ours (full)	0.62 min	0.88 min	0.13 min	0.07 min	$\sim$ 2.32 min

Table 6. Runtime breakdown on the Waymo Open Dataset. See Appendix for detailed analysis.

Explore similar work

Aug 18, 2026

GeoWeaver: Accurate Long-Sequence 3D Reconstruction via Hierarchical Geometric Assembly

Long-sequence 3D reconstruction from RGB videos requires both accurate local geometry and globally consistent camera motion. Feed-forward models provide strong depth and pose predictions, but their memory cost prevents joint inference over long sequences. Chunk-wise processing improves scalability, yet independently predicted chunks often exhibit scale drift, pose errors, and point-cloud misalignment. We present GeoWeaver, a unified framework comprising a Geometric Prior Model (GPM) and Test-Time Adaptation (TTA). The GPM predicts chunk-wise depth, confidence, and camera parameters as adjustable geometric priors. TTA then performs sequential initialization, global chunk-level Sim(3) alignment, and coarse-to-fine refinement of camera poses, affine depth corrections, and intrinsics. Dense correspondences provide adjacent, cross-chunk, and long-range constraints, while a robust CDF-style objective jointly optimizes weighted 2D reprojection and 3D consistency residuals. This design preserves local geometric accuracy while correcting accumulated pose, scale, depth, and calibration errors. Experiments across diverse long-sequence benchmarks demonstrate improved camera accuracy, global consistency, and point-cloud quality. Ablations verify the contribution of each adaptation stage, and applying the same TTA procedure to different geometric prior models consistently improves their trajectory estimates, demonstrating that GeoWeaver is not tied to a specific GPM.

Tinghao Jiang, Sheng Tang, Shengzhe Wei +4

Kosmo Research · Shanghai Jiao Tong University · Tsinghua University

3D Reconstruction Feed-Forward 3D Reconstruction Camera Pose Estimation Test-Time Adaptation

Ask PDF

Sep 18, 2026

Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction

We propose LoG-VGGT, a memory-efficient framework for long-sequence 3D reconstruction that balances local temporal modeling with global camera consistency. Instead of relying on full global attention, our method introduces cross-window attention at a small subset of transformer blocks, enabling effective information propagation across adjacent temporal windows while keeping memory usage bounded. To mitigate long-term pose drift, we further design a global camera consistency refinement module, where camera tokens interact with compact register tokens via cross-attention to enforce scene-level constraints across the entire sequence. This design enables joint optimization of camera representations and significantly improves long-horizon pose stability without incurring the high cost of sequence-wide attention. Extensive experiments demonstrate that LoG-VGGT achieves improved depth accuracy and robust camera pose estimation across multiple long-sequence benchmarks, while delivering competitive streaming reconstruction performance.

Jingke Zhou, Chenhang Ma, Zhizhou Zhong +6

Zhejiang University · Fudan University · Peking University +1

3D Reconstruction Streaming 3D Reconstruction 3D ViTs Camera Pose Estimation

Ask PDF

Oct 5, 2026

VGGT-Bridge: Beyond Sequential Pose Graphs via Coarse-Stride Skip Edges

Feed-forward visual geometry transformers such as VGGT reconstruct dense 3D structure from images in a single forward pass, simplifying multi-view 3D reconstruction. However, their quadratic attention complexity makes them difficult to scale to long sequences with thousands of frames. Chunk-and-align frameworks address this by splitting a long sequence into overlapping chunks and stitching their local reconstructions into a pose graph. Yet existing methods connect only sequentially adjacent chunks, so small per-frame errors accumulate along the chain into large-scale drift. To move beyond sequential edges, we propose VGGT-Bridge, which adds long-range skip edges that directly constrain non-adjacent chunks without retraining. By running VGGT on sparsely sampled coarse chunks, each coarse chunk bridges distant fine chunks into a single direct constraint. We further turn VGGT's first-frame scale bias into a drift correction by feeding selected coarse chunks in reverse, and a loop-aware policy keeps this reversal compatible with existing loop closures. VGGT-Bridge reduces ATE by 28.3% on KITTI Odometry, 18.8% on Virtual KITTI, and 10.0% on Waymo Open over the SwiftVGGT baseline, achieving the best performance among all chunk-and-align methods.

Sungjae Choi, Hanna Bae, Sunghyun Baek +1

Korea Advanced Institute of Science and Technology, South Korea

3D ViTs Streaming 3D Reconstruction Multi-View 3D Reconstruction Pose Graph Optimization Feed-Forward 3D Reconstruction

Ask PDF