PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts

Xianqi Wang; Hao Yang; Hangtian Wang; Junda Cheng; Gangwei Xu; Min Lin; Xin Yang

cs.CVMar 2, 2026

PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts

Authors: Xianqi Wang, Hao Yang, Hangtian Wang, Junda Cheng, Gangwei Xu, Min Lin, Xin Yang

Organizations: Huazhong University of Science and Technology · Optics Valley Laboratory

Abstract

Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. Some methods treat monocular depth priors as guidance for iteration, but conventional GRU-based architectures struggle to exploit them due to the limited representation capacity. In this paper, we propose Prompt Recurrent Unit (PRU), a novel iterative refinement module based on the decoder of monocular depth foundation models. By integrating monocular structure and stereo motion cues as prompts into the decoder, PRU enriches the latent representations of monocular depth foundation models with absolute stereo-scale information while preserving their inherent monocular depth priors. Experiments demonstrate that our PromptStereo achieves state-of-the-art zero-shot generalization performance across multiple datasets, while maintaining comparable or faster inference speed. Our findings highlight prompt-guided iterative refinement as a promising direction for zero-shot stereo matching.

Figures & tables

Figure 1: Comparison of zero-shot generalization. Row 1: Our PromptStereo outperforms previous methods. Row 2-3: A visualization example from Middlebury 2021 [ 36 ] .

Figure 2: Overview of PromptStereo. It builds upon MonSter [ 14 ] , sharing the same feature extraction and cost volume construction. The initial disparity and relative depth are first fused via Affine-Invariant Fusion (Sec. 3.3 ). The fused disparity is then refined iteratively by our Prompt Recurrent Unit (Sec. 3.4 ), which replaces the GRU used in previous methods. It enables effective, prompt-guided iterative refinement, yielding the final high-quality disparity.

Figure 3: Overview of proposed modules. Left: Affine-Invariant Fusion. Right: Prompt Recurrent Unit.

Method	KITTI 2012				KITTI 2015				Midd-T (H)				Midd-2021				ETH3D
	EPE		Bad 3.0		EPE		Bad 3.0		EPE		Bad 2.0		EPE		Bad 2.0		EPE		Bad 1.0
	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc	All	Noc
Training Set	Scene Flow
RAFT-Stereo [ 26 ]	0.90	0.83	4.34	3.86	1.13	1.11	5.68	5.45	1.40	1.03	11.07	8.41	1.40	0.93	11.11	6.38	0.27	0.24	2.61	2.29
IGEV-Stereo [ 47 ]	1.03	0.93	5.13	4.50	1.21	1.17	6.03	5.79	1.48	0.88	9.95	7.09	1.33	0.86	10.00	5.35	0.33	0.29	4.05	3.61
DEFOM-Stereo [ 21 ]	0.83	0.78	3.90	3.52	1.06	1.04	4.99	4.76	0.94	0.65	6.77	4.17	1.24	0.76	8.62	4.98	0.35	0.33	2.40	2.16

Table 1: Zero-shot generalization basic benchmark. MonSter ∗ is re-trained with the official code. FoundationStereo † is reported for reference only, as it requires significantly large batch sizes and model parameters to train, making a fair comparison infeasible.

Figure 4: Visualization of Middlebury (unlimited training sets). The Bad 2.0 metric map is in the upper right corner.

Method	DrivingStereo (H)								Booster (Q)
	Cloudy		Foggy		Rainy		Sunny		EPE	Bad 2.0	Bad 4.0	Bad 6.0	Bad 8.0
	EPE	Bad 3.0	EPE	Bad 3.0	EPE	Bad 3.0	EPE	Bad 3.0	EPE	Bad 2.0	Bad 4.0	Bad 6.0	Bad 8.0
Training Set	Scene Flow
RAFT-Stereo [ 26 ]	0.97	3.58	0.95	2.63	1.80	11.01	1.01	3.97	3.95	16.36	12.57	10.73	9.44
IGEV-Stereo [ 47 ]	1.11	4.84	1.12	4.58	2.34	14.87	1.18	5.05	4.01	16.15	12.66	11.06	10.02
DEFOM-Stereo [ 21 ]	0.98	3.59	1.01	3.51	1.35	11.04	0.98	3.77	2.92	12.77	9.53	8.22	7.42

Table 2: Zero-shot generalization advanced benchmark. MonSter ∗ is re-trained with the official code. FoundationStereo † is reported for reference only, as it requires significantly large batch sizes and model parameters to train, making a fair comparison infeasible. MGStereo ‡ is reported from the official GitHub repository, which is fine-tuned on TranScene [ 28 ] without available checkpoints.

Figure 5: Visualization of Booster (unlimited training sets). The Bad 2.0 metric map is in the upper right corner.

Model	MP	SP	AIF	Midd-T (H)		ETH3D		Time (s)
Model	MP	SP	AIF	EPE	Bad 2.0	EPE	Bad 1.0	Time (s)
Baseline (MonSter)				0.94	7.27	0.26	2.86	0.64
PRU + MP	✓			0.66	4.18	0.22	1.38	0.35
PRU + MP + SP	✓	✓		0.62	3.90	0.21	1.35	0.36
Full model (PromptStereo)	✓	✓	✓	0.60	3.76	0.20	1.30	0.36

Table 3: Ablation study of module effectiveness. Time is recorded when inferencing Scene Flow.

Model	KITTI 2015	Midd-T (H)	ETH3D	Time (s)
Model	Bad 3.0	Bad 2.0	Bad 1.0	Time (s)
RAFT-Stereo	5.68	8.41	2.29	0.36
Prompt-RAFT	4.78	6.39	1.49	0.38
IGEV-Stereo	6.03	7.04	3.61	0.37
Prompt-IGEV	4.84	6.50	2.21	0.38
MonSter	5.52	7.27	2.86	0.64

Table 4: Ablation study of module universality.

Model	KITTI 2015	Midd-T (H)	ETH3D
Model	Bad 3.0	Bad 2.0	Bad 1.0
w/o pre-trained weights	5.03	4.39	1.39
w/ hidden state conv	4.86	4.37	1.34
w/ zero-init conv	4.83	4.04	1.42
PromptStereo	4.59	3.76	1.30

Table 5: Ablation study of PRU design.

Model	Iteration Number
Model	4	8	12	16	32
Evaluation Set	Midd-2021
MonSter	10.75	10.64	10.16	9.61	8.46
PromptStereo	4.35	3.28	2.93	2.79	2.78
Evaluation Set	Booster (Q)
MonSter	14.09	13.62	12.82	12.62	11.55

Table 6: Ablation study of iteration number.

Figure 6: Visualization of ETH3D (unlimited training sets). The Bad 1.0 metric map is in the upper right corner.

Model	KITTI 2012	KITTI 2015	Midd-T (H)	Midd-2021	ETH3D
Model	Bad 3.0	Bad 3.0	Bad 2.0	Bad 2.0	Bad 1.0
MonSter	3.27	3.72	3.75	8.46	1.02
Ours w/o FSD	3.34	3.50	2.41	3.97	0.71
Ours w/ FSD	3.33	3.40	2.21	2.78	0.79

Table 7: Ablation study of whether to use FSD [ 45 ] .

Figure 7: Visualization of Middlebury 2021 (unlimited training sets). The Bad 2.0 metric map is in the upper right corner.

Figure 8: Visualization of Booster (unlimited training sets). The Bad 2.0 metric map is in the upper right corner.

Model	KITTI 2012	KITTI 2015	Midd-T (H)	Midd-2021	ETH3D
Model	Bad 3.0	Bad 3.0	Bad 2.0	Bad 2.0	Bad 1.0
Stereo Anywhere	3.90	3.93	4.49	5.18	1.43
Ours	3.77	4.59	3.76	4.84	1.30
Ours w/ STA	3.33	4.12	4.03	5.06	1.26

Table 8: Ablation study of special training strategy.

Stereo Matching

Stereo matching estimates depth by finding corresponding points in paired images and measuring their disparity.

See all 56 papers

Rethinking Monocular Depth Embedding for Generalized Stereo Matching
Jul 10, 2026Libo Lin, Shuangli Du, Minghua Zhao +3Stereo Matching
Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an University of Technology, No. 5, Jinhua South Road, Xi’an, 710048, Shaanxi, China · College of Computer Science, Sichuan University, , Chengdu, 610065, Sichuan, China
Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching
Jun 23, 2026Junpeng Jing, Ronglai Zuo, Zhelun Shen +5Stereo Matching Stereo Depth Estimation
Imperial College London, London SW7 2AZ, United Kingdom
LiteMatch: Lightweight Zero-Shot Stereo Matching via Cost Volume Stabilization
Jun 30, 2026Md Raqib Khan, Santosh Kumar Vipparthi, Subrahmanyam MuralaZero-Shot Learning Stereo Matching
CVPR Lab, Trinity College Dublin, The University of Dublin, Dublin, Ireland · CVPR Lab, Indian Institute of Technology Ropar, Rupnagar, Punjab, India

Jul 10, 2026J/K move · Enter open · S save

Rethinking Monocular Depth Embedding for Generalized Stereo Matching

Libo Lin, Shuangli Du, Minghua Zhao, Zhenzhen You+2

Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an University of Technology, No. 5, Jinhua South Road, Xi’an, 710048, Shaanxi, China · College of Computer Science, Sichuan University, , Chengdu, 610065, Sichuan, China

Stereo Matching

Open paper AskPDF arXiv

Generally, monocular methods capture rich contextual priors but lack geometric precision, whereas stereo methods are geometrically accurate yet struggle in textureless and occluded regions. Several approaches attempt to combine their strengths to enhance the generalization of stereo matching (SM) by aligning monocular depth with stereo information. However, establishing a stable and generalizable alignment is challenging, and unreliable monocular cues can substantially degrade performance. This paper rethinks monocular depth embedding. First, to prevent shortcut learning, we reduce branch coupling instead of expanding network width. Second, we construct soft constraints instead of hard ones from monocular depth to improve tolerance to monocular depth errors. Based on the principles, we integrate monocular information into both feature extraction and GRU iterations. Specifically, the monocular depth map is fused with the RGB image to sharpen depth boundary perception and suppress matching ambiguities. The fused image is then used for feature extraction, allowing the contextual features to encode global geometric information. Furthermore, the monocular depth gradient feature is employed to guide disparity updates, helping to escape local oscillations. Finally, to address the boundary blurring of supervised disparity caused by data augmentation, we propose an edge confidence estimation method and an edge-aware loss function. Our method achieves state-of-the-art (SOTA) performance on multiple standard benchmarks, demonstrating excellent generalization while improving accuracy. The code is available at https://github.com/linliboabc-maker/stereo-matching-digital.

Abstract

Stereo Matching

Explore similar work

Rethinking Monocular Depth Embedding for Generalized Stereo Matching

Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching

LiteMatch: Lightweight Zero-Shot Stereo Matching via Cost Volume Stabilization