StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matching
Authors: Bohan Li
Organizations: Shanghai Jiaotong University, No.800 Dongchuan Road, Shanghai, 200240, China · Ningbo Institute of Digital Twin, Eastern Institute of Technology, No. 568, Tongxin Road, Ningbo, 315000, China
With the advance of deep neural networks, the quality of disparity maps obtained through stereo matching has steadily improved. However, existing stereo matching methods still struggle to preserve fine-grained geometric details, resulting in blurred edges and over-smoothed predictions in challenging regions. To address these limitations, we propose StereoDiffuer, an iterative diffusion-based stereo matching framework that explicitly models geometric details and progressively refines disparity estimates. The framework incorporates a Saliency Attention Perception (SAP) module to extract salient geometric cues, including object boundaries, thin structures, and sharp edges. Confidence-guided SAP features are combined with the initial disparity estimate to condition an iterative denoising diffusion process, which corrects residual disparity errors and restores geometric details suppressed during cost-volume regularization and upsampling. Experimental results on the Scene Flow and KITTI benchmarks demonstrate the effectiveness of the proposed framework and its competitive performance relative to the compared stereo matching methods.
Figures & tables
Figure 1: Overall pipeline of the proposed StereoDiffuer framework. Given a pair of stereo images, we first estimate an initial disparity map and extract a saliency attention map that highlights high-frequency geometric cues, such as object boundaries, thin structures, and sharp edges. The initial disparity map provides a coarse geometric prior, while the saliency attention map supplies detail-aware guidance. Both are used to construct the condition for the iterative denoising diffusion module, which progressively corrects disparity errors and refines over-smoothed boundary regions. The highlighted road sign illustrates how the proposed refinement improves geometric details in challenging areas.
Figure 2: Overall architecture of StereoDiffuer. The framework consists of four main components: feature-pyramid extraction, multi-scale cost-volume construction, initial disparity regression, and saliency-conditioned diffusion refinement. The cost-volume pyramid provides a multi-scale geometric prior for initial disparity estimation. In parallel, the Saliency Attention Perception (SAP) module extracts detail-aware saliency features from the input image features. A confidence map derived from the cost volume adaptively selects reliable saliency information, which is combined with the initial disparity map to form the collaborative condition. Finally, the conditional diffusion U-Net progressively refines the initial disparity map under the guidance of this condition.
Figure 3: Architecture of the Saliency Attention Perception (SAP) module. SAP contains a spatial-wise saliency block (SSB) and a channel-wise saliency block (CSB). The SSB estimates pixel-level importance to emphasize spatially distinctive structures, while the CSB reweights feature channels to enhance geometry-related responses. The two saliency maps are fused and combined with the input feature to produce the final saliency attention map, which highlights boundaries, thin structures, and other high-frequency geometric details for disparity refinement.
Figure 4: Visualization of outputs from different attention modules. From top to bottom: input images, saliency attention maps from SAP, and attention maps from GCNet and NL-Net, respectively.
Figure 5: Architectural details of the conditional denoising diffusion module. In the Gaussian forward process, the target disparity representation y0 is gradually corrupted over T time steps. In the conditional reverse process, the diffusion U-Net progressively predicts the clean target y^0 . The initial disparity map and confidence-guided SAP feature are concatenated to construct the collaborative condition x used throughout the reverse process.
KITTI 2012
KITTI 2015
Method
2-pixel error (%)
3-pixel error (%)
Mean Error
D1 (%)
Noc
All
Noc
All
Avg-Noc
Avg-All
D1-bg
D1-fg
D1-all
GCNet [ 14 ]
2.71
3.46
1.77
2.30
0.6
0.7
2.21
6.16
2.87
PSMNet [ 15 ]
2.44
3.01
1.49
1.89
0.5
0.6
1.86
4.62
2.32
GwcNet [ 16 ]
2.16
2.71
1.32
1.70
0.5
0.5
1.74
3.93
2.11
GANet-deep [ 85 ]
1.89
2.50
1.19
1.60
0.4
0.5
1.48
3.46
1.81
Table 1: Quantitative results on the KITTI 2012 and KITTI 2015 test sets. For KITTI 2012, the evaluation metrics report the percentages of erroneous pixels at thresholds of 2 and 3 pixels, as well as the average disparity error in non-occluded regions (Noc) and all regions (All). For KITTI 2015, the metrics report the percentage of disparity outliers in the first frame (D1) over background regions (bg), foreground regions (fg), and all ground-truth pixels (all).
Figure 6: Visualization of refinement results on the Scene Flow test set. From left to right: input images, initial disparity maps, and refined disparity maps. In these examples, the refinement produces sharper object boundaries.
Figure 7: Visualization of refinement results on the KITTI 2015 validation set. From top to bottom: input images, initial disparity maps, and refined disparity maps. In these examples, the refinement preserves thin structures and produces sharper object boundaries.
Figure 8: Qualitative comparison with EdgeStereo, GwcNet, BGNet+, and SGNet on the KITTI 2012 test set. In these examples, StereoDiffuer preserves finer thin structures and object edges than the displayed comparison results.
Figure 9: Qualitative comparison with EdgeStereo, GwcNet, BGNet+, and SGNet on the KITTI 2015 test set. In these examples, StereoDiffuer produces sharper object boundaries and more coherent disparity estimates in low-texture regions.
Method
EPE ↓
Method
EPE ↓
GANet [ 85 ]
0.83
CVCNet [ 92 ]
0.68
AANet [ 34 ]
0.87
ACVNet [ 45 ]
0.48
FADNet [ 95 ]
0.84
RAFT-Stereo [ 91 ]
0.56
LEAStereo [ 87 ]
0.78
IGEV-Stereo [ 23 ]
0.47
GwcNet [ 16 ]
0.76
Selective-RAFT [ 94 ]
0.47
BGNet+ [ 90 ]
0.63
StereoDiffuer (ours)
0.47
Table 2: Quantitative results on the Scene Flow test set . We use end-point error (EPE) as the evaluation metric. The StereoDiffuer result uses 12 reverse steps; controlled ablations use four steps unless otherwise stated.
Method
Edge EPE ↓
Edge 3-pixel error (%) ↓
Non-edge EPE ↓
Non-edge 3-pixel error (%) ↓
GwcNet [ 16 ]
1.52
5.12
0.68
1.75
BGNet+ [ 90 ]
1.34
4.65
0.58
1.62
ACVNet [ 45 ]
1.15
3.84
0.45
1.35
IGEV-Stereo [ 23 ]
0.98
3.32
0.39
1.20
Selective-RAFT [ 94 ]
0.94
3.21
0.37
1.15
StereoDiffuer (ours)
0.76
2.58
0.35
1.12
Table 3: Performance on the 40-pair KITTI 2015 validation split in edge and non-edge regions. Valid pixels satisfy 0<dgt<192 . The initial edge mask is obtained by thresholding horizontal and vertical forward differences of valid ground-truth disparities at 1 pixel and is dilated once with a 3×3 square structuring element. All methods use the same masks and full-resolution predictions without boundary-specific post-processing. EPE and 3-pixel error are pooled over all valid pixels in each region.
Cost Volume Pyramid
Saliency Attention Perception
Iterative Diffusion
Iterative GRU
KITTI 2015 3-pixel error (%)
Scene Flow EPE
Large
Middle
Small
Spatial-wise Saliency
Channel-wise Saliency
√
√
√
√
2.03
1.30
√
√
√
√
√
1.91
1.22
√
√
√
√
√
√
1.81
1.08
√
√
√
√
1.74
0.90
√
√
√
√
√
1.69
0.81
Table 4: Ablation results for the proposed framework. The 3-pixel error is computed on the KITTI 2015 validation split, and end-point error (EPE) is computed on the Scene Flow test set. These controlled experiments use four reverse diffusion steps unless otherwise stated.
Attention Module
KITTI 2015 3-pixel error (%)
Scene Flow EPE
Baseline (None)
1.74
0.90
SE [ 96 ]
1.70
0.85
CBAM [ 97 ]
1.67
0.80
GC block [ 81 ]
1.65
0.76
Non-local [ 80 ]
1.61
0.70
SAP (ours)
1.52
0.51
Table 5: Quantitative comparison of attention mechanisms on the KITTI 2015 validation split and Scene Flow test set. The baseline excludes attention but retains the multi-scale cost volume and four-step diffusion refinement.
Refinement Strategy
Condition
Updates
KITTI 2015 3-pixel error (%) ↓
Scene Flow EPE ↓
Initial disparity without refinement
–
0
1.95
1.25
CNN refinement
Dini+C′
1
1.82
1.02
ConvGRU refinement
Dini+C′
4
1.73
0.81
Diffusion refinement without SAP
Dini
4
1.74
0.90
SAP-conditioned diffusion refinement
Dini+C′
4
1.52
0.51
Table 6: Controlled comparison of refinement strategies on the KITTI 2015 validation split and Scene Flow test set. Dini+C′ denotes concatenation of the initial disparity map and confidence-guided SAP feature. All variants use the same stereo matching backbone and training protocol; the listed refinement operator and condition are the controlled differences.
Diffusion steps
KITTI 2015 3-pixel error (%)
Scene Flow EPE
Runtime (s)
1
1.82
1.03
0.15
2
1.64
0.74
0.19
4
1.52
0.51
0.28
8
1.50
0.48
0.45
12
1.49
0.47
0.61
Table 7: Runtime per stereo pair and estimation accuracy for different numbers of reverse diffusion steps. Runtime is measured on an NVIDIA RTX 4090 under the same inference setup. The 3-pixel error is evaluated on the KITTI 2015 validation split, and EPE on the Scene Flow test set. Four steps are used by default as an accuracy–efficiency compromise.
Stereo matching is a fundamental task in 3D reconstruction. Despite remarkable advances, the prevailing paradigms formulate stereo matching as a deterministic regression problem, collapsing the multimodal distribution modeling into a single-point estimation. This formulation suffers from a regression-to-mean bias, frequently struggling with ambiguous regions. In contrast, we introduce a prior-guided generative framework that integrates deterministic matching regression and generative distribution modeling within a complementary formulation. Built upon this formulation, we introduce StereoFlow through three key components: (i) a two-stage progressive cascade matching network that progressively produces multi-resolution stereo conditions with complementary matching cues; (ii) a pixel diffusion transformer (termed StereoDiT) with a frequency-decoupled architecture for modeling correspondence ambiguity; (iii) a few-step flow matching objective (termed Transition Flow Matching) for efficient optimization. In summary, StereoFlow achieves strong geometric consistency and rich fine-grained details in ill-posed, discontinuous regions and under zero-shot generalization. Extensive experiments demonstrate that the proposed StereoFlow establishes multiple state-of-the-art results across benchmarks, including Scene Flow, KITTI, ETH3D, and Middlebury.
Hao Wang, Haoran Geng, Xiaotong Yang +7
School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Google · State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China +3
Stereo matching and surface normal estimation are fundamental tasks in 3D vision. However, existing feed-forward stereo methods still struggle to produce reliable predictions in challenging regions, mainly due to the lack of strong geometric priors. In this paper, we propose GeoStereo, a unified stereo geometry estimation framework that leverages powerful diffusion priors to jointly predict disparity and surface normals. Specifically, GeoStereo couples a feed-forward stereo matching pipeline with a diffusion-based normal estimation branch. To enable effective interaction between the two tasks, we introduce a disparity to normal initialization strategy and construct a warp to left-view condition for the diffusion process. This coupled design allows the diffusion branch to provide strong structural priors that enhance disparity estimation in ill-posed regions, while the feed-forward branch offers reliable geometric guidance for accurate normal prediction. Extensive experiments show that GeoStereo performs reliably in challenging scenarios, including low-light environments, highly reflective surfaces, and transparent objects. Under zero-shot settings, it achieves Rank-1 disparity estimation on multiple benchmarks, including KITTI and NYUv2, and delivers the best normal estimation accuracy on many real indoor benchmarks, such as iBims-1 and ScanNet. Project page: https://qz-wei.github.io/GeoStereo.github.io/
Qizhe Wei, Xianda Guo, Shaocong Xu +3
Beijing Institute of Technology BAAI Beijing, China · School of Computer Science, Wuhan University Wuhan, China · BAAI Beijing, China +3
Depth boundaries remain a major source of error in stereo matching. For IGEV on Scene Flow, geometric edge pixels account for only 6% of evaluated pixels but about half of the total absolute disparity error. Photometric edges can respond to texture changes without corresponding depth discontinuities, making them unreliable cues for boundary refinement. We propose E3Stereo, which learns geometric edges from disparity supervision and combines these boundary cues with the backbone disparity and image details to predict full-resolution disparity corrections. The same edge map also emphasizes boundary errors during training, while joint optimization allows the edge predictor to adapt to disparity refinement. On the Scene Flow dataset, E3Stereo reduces end-point error (EPE) at depth boundaries by 9.5%-14.3% across RAFT-Stereo, IGEV, and MonSter, while also reducing overall EPE. After fine-tuning, our IGEV-based model improves upon IGEV on all eleven reported metrics across ETH3D, KITTI 2012, and KITTI 2015. The source code will be made publicly available upon acceptance.