StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision

Ziyang Chen; Yansong Qu; You Shen; Xuan Cheng; Liujuan Cao

cs.CVMar 31, 2026

StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision

Authors: Ziyang Chen, Yansong Qu, You Shen, Xuan Cheng, Liujuan Cao

Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University

Abstract

Driven by the advancement of 3D devices, stereo vision tasks including stereo matching and stereo conversion have emerged as a critical research frontier. Contemporary stereo vision backbones typically rely on either Monocular Depth Estimation models or general-purpose Pre-trained Vision Models. Crucially, these models are predominantly pretrained without explicit supervision of camera poses. Given that such geometric knowledge is indispensable for stereo vision, the absence of explicit spatial constraints constitutes a significant performance bottleneck for existing architectures. Recognizing that the Visual Geometry Grounded Transformer (VGGT) operates as a foundation model pre-trained on extensive 3D priors, including camera poses, we investigate its potential as a robust backbone for stereo vision tasks. Nevertheless, empirical results indicate that its direct application to stereo vision yields suboptimal performance. We observe that VGGT suffers from a more significant degradation of geometric details during feature extraction. Such characteristics conflict with the requirements of binocular stereo vision, thereby constraining its efficacy for relative tasks. To bridge this gap, we propose StereoVGGT, a feature backbone specifically tailored for stereo vision. By leveraging the frozen VGGT and introducing a training-free feature adjustment pipeline, we mitigate geometric degradation and harness the latent camera calibration knowledge embedded within the model. StereoVGGT-based stereo matching network achieved the $1^{st}$ rank among all published methods on the KITTI benchmark, validating that StereoVGGT serves as a highly effective backbone for stereo vision.

Figures & tables

Panel of Figure 4 — Figure 4: VGGT suffers from structural degradation during its feature processing. The orange bar charts represent the Boundary F-score. Specifically, this metric quantifies the alignment between the boundaries of the feature map and those of the ground-truth disparity map. Concurrently, the red bar charts illustrate the SSIM values calculated between the extracted feature maps and the original input images across the ETH3D, KITTI, and Middlebury datasets. To ensure a fair comparison, all evaluated methods adhere to a consistent evaluation protocol; comprehensive details are provided in the supplementary material.

Method	KITTI		Middlebury		ETH3D
Method	SSIM $\uparrow$	F-score $\uparrow$	SSIM $\uparrow$	F-score $\uparrow$	SSIM $\uparrow$	F-score $\uparrow$
VGGT	0.68	0.25	0.75	0.88	0.74	0.62
Ours	0.97	0.29	0.94	0.93	0.96	0.69

Table 1: Quantitative evaluation of structural degradation. F-score means Boundary F-score here,

\downarrow

indicates that lower values are better.

Figure 5: Overview of the StereoVGGT architecture. StereoVGGT comprises three main stages. First, EMWM synthesizes a new set of optimized DINO weights by merging the weights of VGGT and an MDE model, guided by an entropy-based criterion. Second, the patch tokens generated by the re-weighted DINO are concurrently fed into both frozen VGGT Frame Attention (FA) Blocks and an MDE neck. The VGGT FA features subsequently modulate the MDE neck features, thereby achieving a balance between camera knowledge and fine-grained image-feature representation. Finally, the resulting latent features $X_{stereovggt}$ are passed through a DPT head to generate the disparity prior $d_{stereovggt}$ . The latent features and the disparity prior are then leveraged in downstream stereo vision tasks, including stereo conversion and stereo matching.

Figure 6: The bar charts show the median and mean camera FOV errors across different methods. This analysis evaluates FOV estimated from extracted features on the ETH3D dataset. We implement a Levenberg-Marquardt solver to estimate the camera Field-of-View (FOV) Devernay and Faugeras (2001) directly from these features. Detailed specifications of this solver are provided in Sec. 2.6

Panel of Figure 7 — Figure 7: The orange bar charts represent the Boundary F-score. This metric quantifies the alignment between the boundaries of the feature map and those of the ground-truth disparity map. Concurrently, the red bar charts illustrate the SSIM values calculated between the extracted feature maps and the original input images across the ETH3D, KITTI, and Middlebury datasets. To ensure a fair comparison, all evaluated methods adhere to a consistent evaluation protocol. Comprehensive details are provided in Sec. 3.2.1 .

Feature Backbone	Disparity Decoder	Reference	Non-occ $\downarrow$			All $\downarrow$
Feature Backbone	Disparity Decoder	Reference	all	fg	bg	all	fg	bg
ViT-Large	ViT-Base+DPT	CroCo-Stereo	1.51	2.56	1.30	1.59	2.65	1.38
MobileNet V2	CGEV+SRU	Selective-IGEV	1.44	2.55	1.22	1.55	2.61	1.33
DAv2	SGA&MGR+ConvGRU	MonSter	1.33	2.76	1.05	1.41	2.81	1.13
DAv2	SGA&MGR+PRU	PromptStereo	1.32	2.76	1.04	1.41	2.85	1.12
MobileNet V2	CGEV+ConvGRU Paradigm (IGEV-Stereo)	IGEV-Stereo	1.49	2.62	1.27	1.59	2.67	1.38

Table 2: Evaluation results on the KITTI benchmark under 3-pixel error threshold. ”Non-occ” and ”All” denote evaluation on non-occluded pixels and all pixels, respectively. ”all” indicates the outlier percentage over all ground-truth pixels, ”fg” and ”bg” correspond to foreground and background regions. Red, orange, and green shading indicate 1st , 2nd , and 3rd place, respectively. The row highlighted with yellow denotes our proposed method .

Figure 8: Our stereo matching network built upon StereoVGGT ranked $1^{st}$ on the KITTI benchmark for non-occluded pixels at the time of submission.

Figure 9: Visualization comparison on the KITTI dataset. The text in parentheses indicates the feature backbone used by each method. The preceding examples show that both input scenes contain gaps between the signs and poles.

Baseline Network	Type	Backbone	D1 $\downarrow$	EPE $\downarrow$
IGEV-Stereo ( Xu et al., 2023 )	PVM	MobileNet V2	5.3	0.47
	PVM	EfficientNet V2 $\dagger$	5.2	0.46
	3R	VGGT $\dagger$	5.3	0.46
	MDE	MoGe-2 $\dagger$	5.1	0.45
	MDE	DAv2 $\dagger$	5.1	0.44
	Ours		4.9	0.43

Table 3: Quantitative evaluation on Scene Flow test set. The error threshold is 1 px.

\dagger

denotes our reproduced version, for which no publicly available source code exists. Light Gray shading indicates the baseline method . The row highlighted with yellow denotes our proposed method .

Disparity Stage	Inpainting Stage	Reference	RMSE $\downarrow$	SSIM $\uparrow$	SIoU $\uparrow$	PSNR $\uparrow$
DPT-based	SD2-depth	StereoDiffusion	9.345	0.2703	0.1941	28.75
MiDaS +Stereo Pixel Shift	Stable Diffusion	StereoDiffusion	8.451	0.4472	0.2672	29.66
DepthCrafter	SVD	StereoCrafter	7.392	0.5866	0.2656	29.71
DAv2	Marigold VAE (Fine-tuned by Mono2Stereo)	Mono2Stereo	6.857	0.7227	0.2859	31.53
VGGT		-	6.734	0.7082	0.2801	31.57
MoGe-2		-	6.891	0.7247	0.2753	30.88

Table 4: Stereo conversion evaluation on the Inria 3D Movie dataset. In the upper table, StereoDiffusion uses SD2-depth as its inpainting model, while StereoCrafter uses SVD. In the lower table, both methods employ the Marigold VAE fine-tuned by Mono2Stereo as their inpainting model.

Figure 10: Visual comparison in anaglyph 3D format on the Inria 3D Movie dataset. The red boxes highlight the main differences.

Scenes	Method	RMSE $\downarrow$	SSIM $\uparrow$	SIoU $\uparrow$	PSNR $\uparrow$
Indoor	Mono2Stereo	5.218	0.8161	0.2661	34.12
Indoor	Ours	5.192	0.8354	0.2689	34.20
Outdoor	Mono2Stereo	5.655	0.7653	0.2708	33.51
Outdoor	Ours	5.553	0.7970	0.2743	33.72
Animation	Mono2Stereo	5.763	0.7628	0.2755	33.13
Animation	Ours	5.716	0.7904	0.2798	33.19

Table 5: Quantitative comparison of our proposed method and Mono2Stereo on the Mono2Stereo dataset. Bold font indicates the better-performing result.

	Stereo Matching			Stereo Conversion
	Middlebury	KITTI	ETH3D	Inria 3D	Mono2Stereo
	(Bad-2 $\downarrow$ )	(Bad-3 $\downarrow$ )	(Bad-1 $\downarrow$ )	(RMSE $\downarrow$ )
Baseline	6.21	6.03	4.05	6.86	5.34
Ours	4.89	5.52	3.37	6.46	4.14
w/o MDE DINO	6.20	6.04	4.01	6.58	4.36
w/o VGGT DINO	5.85	5.89	3.97	6.62	4.51

Table 6: Ablation studies. The results for stereo matching are obtained via zero-shot inference using weights trained on Scene Flow. The 2px error rate is employed for Middlebury, 3px error rate for KITTI, and 1px error rate for ETH3D. ”w/o MDE DINO” denotes EMWM re-weighted only by VGGT, ”w/o VGGT DINO” denotes EMWM re-weighted only by MoGe-2.

Type	Method	KITTI $\downarrow$				ETH3D $\downarrow$				Middlebury $\downarrow$
Type	Method	EPE	7 px	9 px	15 px	EPE	7 px	9 px	15 px	EPE	7 px	9 px	15 px
Gen.	ZeroStereo	27.91	77.48	72.40	58.85	23.85	68.35	61.71	48.66	19.98	80.43	72.61	52.81
MDE	Unidepth V2	9.07	63.64	47.74	9.97	3.03	15.06	9.13	0.21	27.72	89.99	84.13	70.17
	MoGe-2	6.74	32.16	18.58	6.15	4.84	19.97	12.01	0.30	18.11	68.80	52.14	39.57
	DAv2	5.88	32.91	23.21	7.19	8.24	36.17	28.17	16.70	24.94	86.90	82.91	67.44
3R	VGGT	13.07	68.40	58.06	33.99	13.11	69.88	54.31	28.21	18.71	71.25	60.99	40.87

Table 7: Cross-dataset performance comparison of disparity estimation in a monocular setting on binocular stereo datasets. The input consists of the left view of a stereo pair, along with the camera focal and baseline length. All models were evaluated in their pre-trained state without any fine-tuning. ”Gen.”: Generation-based method, ”MDE”: Monocular Depth Estimation method, ”3R”: 3D Reconstruction method.

No.	DINO Weights	Camera FOV $\downarrow$				SSIM $\uparrow$	Disparity Error $\downarrow$
		FOV x		FOV y			EPE	7 px	9 px	15 px
		med.	mean	med.	mean		EPE	7 px	9 px	15 px
1	DINOv2	80.3	84.4	65.8	62.6	0.965	20.6	69.1	62.4	39.9
2	DAv2	73.7	77.2	60.0	61.8	0.965	7.38	30.5	26.4	11.0
3	MoGe-2	30.4	35.1	15.8	14.2	0.965	3.68	16.5	9.78	0.24
4	VGGT	14.7	18.6	7.08	8.13	0.776	12.1	66.5	49.8	25.7

Table 8: Impact of weight selection. The weights of the DINO were re-weighted in StereoVGGT. SSIM calculation followed the same configuration as in Figs. 3 and 4. The camera FOV error is calculated following the methods in ( Wang et al., 2025b ; Wang et al., 2025a ) , with specific computational details provided in the Preliminary section.

$\lambda_{VGGT}$	$\lambda_{MDE}$	Camera FOV $\downarrow$				SSIM $\uparrow$	Disparity Error $\downarrow$
		FOV x		FOV y			EPE	7 px	9 px	15 px
		med.	mean	med.	mean		EPE	7 px	9 px	15 px
0.0	1.0	30.4	35.1	15.8	14.2	0.965	3.68	16.5	9.78	0.24
0.2	0.8	29.3	35.8	17.2	20.0	0.842	12.6	75.3	56.6	25.3
0.5	0.5	27.7	28.5	16.5	19.7	0.799	12.8	72.4	50.9	23.5
0.8	0.2	26.1	21.3	13.9	13.6	0.727	12.3	79.10	48.7	24.03

Table 9: Ablation study of

\lambda

selection. For each experiment, a fixed interpolation coefficient is applied uniformly across all network layers.

No.	MDE Neck	AA Block			Camera FOV $\downarrow$				SSIM $\uparrow$	Disparity Error $\downarrow$
		GA	FA	$\alpha$	FOV x		FOV y			EPE	7 px	9 px	15 px
		GA	FA	$\alpha$	med.	mean	med.	mean		EPE	7 px	9 px	15 px
1		✓	✓	-1.0	5.78	6.70	4.27	4.98	0.894	2.76	11.1	5.24	0.33
2	✓			0.0	4.05	6.76	2.73	5.19	0.964	2.43	10.7	7.80	0.15
3	✓	✓	✓	1.0	4.59	6.40	3.28	4.79	0.939	2.65	11.5	4.78	0.22
4	✓	✓		0.2	3.93	6.67	2.72	5.09	0.964	2.39	10.7	7.26	0.16

Table 10: Ablation Study on the Neck Component. “AA” is Alternative Attention (AA) Block in VGGT, “GA” is Global Attention in VGGT, “FA” is Frame Attention in VGGT. Given that the integration of the AA blocks is implemented via element-wise subtraction within our pipeline, the hyperparameter

\alpha

is accordingly assigned a negative value in Experiment No.1.

$\alpha$	Camera FOV $\downarrow$				Disparity Error $\downarrow$
	FOV x		FOV y		EPE	7 px	9 px	15 px
	med.	mean	med.	mean	EPE	7 px	9 px	15 px
1.0	4.59	6.40	3.28	4.79	2.65	11.5	4.78	0.22
0.8	4.73	6.42	3.46	4.82	2.58	11.74	5.47	0.20
0.5	3.97	6.52	2.91	4.94	2.43	11.26	7.36	0.17
0.4	3.81	6.57	2.62	4.99	2.40	11.07	7.45	0.16

Table 11: An ablation study on hyperparameter

\alpha

performed using the ETH3D dataset. A larger value of

\alpha

assigns a greater weight to the VGGT neck branch.

Figure 11: Disparity maps generated by SOTA models on the ETH3D dataset. The models use monocular condition input of binocular cameras, consisting of a left view, focal length, and baseline length. The white boxes highlight the main differences. The disparity maps generated by ZeroStereo exhibit instability and fail to accurately reconstruct some foreground objects. The disparity maps estimated by VGGT exhibit significant limitations in edge-texture fidelity. While DAv2 achieves precise edge localization, it demonstrates inaccuracies in estimating the relative depth ordering and inter-object spatial relationships. In contrast, StereoVGGT succeeds in providing both fine-grained edge estimation and a highly accurate global geometric layout.

Table 20

Figure 12: Disparity map comparisons across state-of-the-art models on challenging ill-posed regions. The models use monocular condition input of binocular cameras, consisting of a left view, focal length, and baseline length. The red boxes and white boxes highlight the main differences.

Stereo Vision

Stereo vision uses images from two or more viewpoints to estimate depth, recover 3D structure, and support spatial perception.

See all 43 papers

MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement
Apr 22, 2026Haoyu Zhang, Jingyi Zhou, Peng Ye +4Vision Transformer Multi-Scale Feature Fusion
Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China · Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China, and also with Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China, and also with the Chinese University of Hong Kong, Hong Kong · Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China +2
LinStereo: Linear-Complexity Global Attention for Multi-Scale Iterative Stereo Matching
Jun 24, 2026Yiran Wang, Oliver Turner, Viorela IlaMulti-Scale Feature Fusion Stereo Matching
Australian Centre for Robotics (ACFR), School of Aerospace, Mechanical and Mechatronic Engineering (AMME), Faculty of Engineering, The University of Sydney, Sydney, NSW, Australia
StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matching
Aug 22, 2026Bohan LiStereo Matching
Shanghai Jiaotong University, No.800 Dongchuan Road, Shanghai, 200240, China · Ningbo Institute of Digital Twin, Eastern Institute of Technology, No. 568, Tongxin Road, Ningbo, 315000, China

Apr 22, 2026J/K move · Enter open · S save

MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement

Haoyu Zhang, Jingyi Zhou, Peng Ye, Jiakang Yuan+3

Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China · Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China, and also with Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China, and also with the Chinese University of Hong Kong, Hong Kong · Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China+2

Vision Transformer Multi-Scale Feature Fusion Stereo Matching

Open paper AskPDF arXiv

With the development of deep learning, ViT-based stereo matching methods have made significant progress due to their remarkable robustness and zero-shot ability. However, due to the limitations of ViTs in handling resolution sensitivity and their relative neglect of local information, the ability of ViT-based methods to predict details and handle arbitrary-resolution images is still weaker than that of CNN-based methods. To address these shortcomings, we propose MLG-Stereo, a systematic pipeline-level design that extends global modeling beyond the encoder stage. First, we propose a Multi-Granularity Feature Network to effectively balance global context and local geometric information, enabling comprehensive feature extraction from images of arbitrary resolution and bridging the gap between training and inference scales. Then, a Local-Global Cost Volume is constructed to capture both locally-correlated and global-aware matching information. Finally, a Local-Global Guided Recurrent Unit is introduced to iteratively optimize the disparity locally under the guidance of global information. Extensive experiments are conducted on multiple benchmark datasets, demonstrating that our MLG-Stereo exhibits highly competitive performance on the Middlebury and KITTI-2015 benchmarks compared to contemporaneous leading methods, and achieves outstanding results in the KITTI-2012 dataset.

Abstract

Stereo Vision

Explore similar work

MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement

LinStereo: Linear-Complexity Global Attention for Multi-Scale Iterative Stereo Matching

StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matching