While iterative stereo matching achieves high accuracy, its dependence on Recurrent Neural Networks (RNN) hinders edge deployment, a challenge underexplored in existing researches. We analyze iterative refinement and reveal that disparity updates are spatially sparse and temporally redundant. First, we introduce a progressive iteration pruning strategy that suppresses redundant update steps, effectively collapsing the recursive computation into a near-single-pass inference. Second, we propose a collaborative monocular prior transfer framework that implicitly embeds depth priors without requiring a dedicated monocular encoder, thereby eliminating its associated computational burden. Third, we develop FlashGRU, a hardware-aware RNN operator leveraging structured sparsity and I/O-conscious design, achieving a 7.28× speedup, 76.6% memory peak reduction and 80.9% global memory requests reduction over natvie ConvGRUs under 2K resolution. Our PipStereo enables real-time, high-fidelity stereo matching on edge hardware: it processes 320×640 frames in just 75ms on an NVIDIA Jetson Orin NX (FP16) and 19ms on RTX 4090, matching the accuracy of large iterative based models, and our generalization ability and accuracy far exceeds that of existing real-time methods. Our embedded AI projects will be updated at: https://github.com/XPENG-Aridge-AI.
Figures & tables
Figure 1: Iteration update positions and hit ratios on Middlebury [ 19 ] test set. Red pixels denote updates; left color bars show consistency with the previous iteration. Refinements are sparse, highly redundant, and affect only a small fraction of the image.
Figure 2: Overview of two-stage training: (1) depth prior transfer via multi-level and cost volume feature alignment, and (2) pruned finetuning with progressively fewer iterations, training only the ConvGRU modules.
Figure 3: Illustrion of the FlashGRU in 2 resolution levels and T loops, with 70% sparisity.
Method
Iters.
Sceneflow [ 16 ]
ETH3D [ 21 ]
KITTI 2012 [ 18 ]
KITTI 2015 [ 8 ]
Latency(s)
EPE
Bad-1
Bad-1
RMSE
Out-2
Out-2
Out-3
Out-3
D1-bg
D1-all
D1-bg
D1-all
All
Noc
All
Noc
Noc
All
Noc
All
Noc
Noc
All
All
RaftStereo [ 15 ]
32
0.72
2.44
2.60
0.36
1.92
2.42
1.30
1.66
1.44
1.69
1.58
1.82
2.95
IGEV [ 33 ]
12
0.49
1.12
1.51
0.34
1.71
2.17
1.12
1.44
1.27
1.49
1.38
1.59
1.29
SelectiveIGEV [ 28 ]
12
0.44
1.23
1.56
0.29
1.59
2.05
1.07
1.38
1.22
1.44
1.33
1.55
1.61
DEFOMStereo [ 12 ]
32
0.42
0.70
0.78
0.22
1.43
1.79
0.94
1.18
1.25
1.41
1.15
1.33
5.05
Table 1: In-domain comparison with PipStereo. Top panel lists high-accuracy iterative approaches, while bottom panel includes real-time methods. PipStereo achieves accuracy competitive with state-of-the-art iterative methods while significantly outperforming all high-accuracy approaches in inference latency, and substantially surpassing existing real-time methods in accuracy. The item and item are the best and second best in each panel. Latencies are tested in 384×1344 resolution on Orin NX (25W power mode, FP32 precision).
Method
Iter. based
Sunny
Cloudy
Rainy
Foggy
Avg.
Latency(s)
ZeroStereo [ 29 ]
✗
3.15
2.69
11.71
1.70
4.81
OOM
Nerf-Stereo [ 24 ]
✗
2.88
2.95
8.47
3.41
4.43
-
SMoEStereo [ 30 ]
✗
3.51
3.11
6.08
4.77
4.37
-
IGEV [ 33 ]
✓
5.30
5.48
15.41
5.01
7.80
1.29
Selective-IGEV [ 28 ]
✓
5.67
5.35
13.84
4.79
7.41
1.61
DefomStereo [ 12 ]
✓
3.39
3.13
12.84
2.57
5.48
5.05
Table 2: Zero-shot generalization performance. Evaluation on DrivingStereo [ 38 ] under diverse weather conditions, reported using the D1-All metric.
Figure 4: Visual comparison. Leveraging depth prior, PipStereo successfully handles ill-posed regions and produces sharper details and more coherent structures compared to IGEV [ 33 ] .
ID
Method
Alias
Infer iters.
SceneFlow [ 16 ]
Latency(s)
1
Selective-IGEV [ 28 ]
-
12
0.44
1.61
2
-
-
32
0.44
3.57
3
-
-
1
0.64
0.60
4
+MPT (w./o. search)
-
32
0.40
3.40
5
+MPT
PipStereo-i32
1
0.56
0.44
6
+MPT
PipStereo-i32
32
0.38
3.40
Table 3: Integration ablations. MPT substantially boosts performance; PIP effectively curbs accuracy degradation from fewer iterations, while FlashGRU provides acceleration for methods that more than one iteration under PIP.
Flash.
Size
RNN Latency
Mem. Peak (MiB)
Mem. Request
✗
320 × 736
28.8ms
743
2.260 ×e6
✓
320 × 736
14.8ms ( × 1.94 )
561 ( ↓ 24.4% )
0.947×e6 ( ↓ 58.1% )
✗
640 × 1472
45.2ms
1431
9.258 ×e6
✓
640 × 1472
15.2ms ( × 2.97 )
715 ( ↓ 50.0% )
2.189×e6 ( ↓ 76.4% )
✗
1280 × 2944
122.4ms
4105
37.756 ×e6
✓
1280 × 2944
16.8ms ( × 7.28 )
957 ( ↓ 76.6% )
7.224×e6 ( ↓ 80.9% )
Table 4: Efficiency gain of P3-Flash on NVIDIA RTX4090 24GB (Ada Lovelace). Baseline without ”Flash” is the ConvGRU implementation of Selective-IGEV [ 28 ] , and tested on downsampled 4 × feature map for update block. Red marks the speedup ratio. Green marks the reduction ratio. The ”Request” is the sum of ”ld/st” micro requests to GPU global memory.
Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. Some methods treat monocular depth priors as guidance for iteration, but conventional GRU-based architectures struggle to exploit them due to the limited representation capacity. In this paper, we propose Prompt Recurrent Unit (PRU), a novel iterative refinement module based on the decoder of monocular depth foundation models. By integrating monocular structure and stereo motion cues as prompts into the decoder, PRU enriches the latent representations of monocular depth foundation models with absolute stereo-scale information while preserving their inherent monocular depth priors. Experiments demonstrate that our PromptStereo achieves state-of-the-art zero-shot generalization performance across multiple datasets, while maintaining comparable or faster inference speed. Our findings highlight prompt-guided iterative refinement as a promising direction for zero-shot stereo matching.
Xianqi Wang, Hao Yang, Hangtian Wang +4
Huazhong University of Science and Technology · Optics Valley Laboratory
Existing iterative stereo matching methods primarily adopt two types of correspondence representation: explicit matching search via correlation volumes and local residual refinement via warped features, yet the two remain separately modeled. We propose WAVE-Stereo, built on a core insight: correlation volumes and feature warping provide complementary matching cues. GeoWarp Correspondence Encoder (GWCE) encodes matching search, residual alignment, and disparity prior in parallel at the ConvGRU input. To mitigate matching degradation in textureless regions, we propose Periodic Global Context Propagation (PGCP), which propagates global spatial information in a periodic manner. On five real-world benchmarks -- Middlebury, ETH3D, KITTI 2012, KITTI 2015, and Booster -- WAVE-Stereo achieves competitive zero-shot generalization accuracy without any external foundation model prior, achieving 3.18% D1-all on KITTI 2015, 4.42% Bad-2.0 on Booster, and 66ms real-time inference, striking a favorable balance between accuracy and efficiency. Our code is available at https://github.com/yamanoko-do/WAVE-Stereo.
Zehan Liu, Yage He, Xianwu Gong
School of Electronics and Control Engineering Chang’an University Xi’an, China
Standard attention mechanisms are not well suited to stereo matching. Global attention scales quadratically and provides no explicit matching constraint, while local attention is efficient but loses long-range correspondences. We propose MatchAttention, an attention mechanism that embeds an explicit matching constraint into attention by treating the relative position between a query and its matched key as a learnable component of attention sampling. Centering a small contiguous sampling window on this learnable relative position enforces the matching constraint and supports long-range correspondence at strictly linear attention complexity. A differentiable contiguous attention sampling (CAS) operator enables sub-pixel accuracy, and cascaded MatchAttention blocks iteratively refine the relative positions through residual connections. We instantiate MatchAttention as a hierarchical coarse-to-fine stereo network with two variants. MatchAttentionXL targets accuracy and MatchAttentionRT targets real-time edge inference. MatchAttentionXL achieves state-of-the-art accuracy on Middlebury V3 and top results across KITTI 2012/2015 and ETH3D. MatchAttentionRT runs at 9.3 ms on RTX 4060 Ti and 79.1 ms on Jetson Orin NX 16 GB at 1024 x 512, making it the first stereo model to deliver real-time edge inference without sacrificing zero-shot generalization. The code is available at https://github.com/TingmanYan/MatchAttention.
Tingman Yan, Tao Liu, Chenghao Li +4
School of Control Science and Engineering, Dalian University of Technology, Dalian, 116024, China · Pika Labs · Department of Automation, Shanghai Jiao Tong University, Shanghai, 200240, China +1