DEQ-Stereo: Deep Euilibrium Video Stereo Matching via Adaptive Convergence
Authors: Kiljoon Han, Kyumin Hwang, Wonheok Choi, Jeonghoon Kim, Sunghoom Im
Organizations: Daegu Gyeongbuk Institute of Science and Technology, Daegu, South Korea · Korea Advanced Institute of Science and Technology, Daejeon, South Korea
Iterative refinement models have significantly improved stereo matching accuracy, but their rigid, fixed update schedules incur heavy computational costs and a strict trade-off between speed and accuracy. This challenge is magnified in video streams, where traditional online methods resort to reducing feature channels or relying on expensive temporal warping. To address this, we propose DEQ-Stereo, a novel video stereo matching framework based on the Deep Equilibrium (DEQ) paradigm. Instead of unrolling a fixed number of iterations, DEQ-Stereo reformulates the refinement as an implicit fixed-point problem, adaptively terminating once convergence is met. To ensure fast and stable intra-frame convergence, we introduce the Convolutional Simplified Gated Unit (ConvSGU), which eliminates unnecessary non-linearities by modeling the latent state as a gated, element-wise sum of inputs. Furthermore, to exploit cross-frame temporal coherence, we propose Temporal State Transfer (TST), a warm-start strategy that propagates the converged hidden state to the next frame without requiring explicit pose or warping priors. Extensive evaluations demonstrate that DEQ-Stereo delivers state-of-the-art performance with a superior accuracy-to-latency trade-off.
Figures & tables
Figure 1: Comparisons of Iterative Stereo Matching. Traditional methods (e.g., IGEV-Stereo) use a fixed number of iterations, resulting in a trade-off between accuracy and efficiency. In contrast, our method adaptively determines the number of iterations, achieving high accuracy and efficiency.
Figure 2: Overall pipeline. (a) Overview of DEQ-Stereo. From stereo pairs, we construct a correlation volume and regress an initial disparity, which is refined via an implicit, convergence-driven DEQ process. Unlike prior iterative methods, DEQ-Stereo supports adaptive termination and faster convergence, enabled by two key components: ConvSGU and Temporal State Transfer. We denote the frame index as t , the DEQ iteration step as k , and the converged state as ∗ . (b) Standard ConvGRU structure. (c) Our proposed ConvSGU, a simplified variant of ConvGRU, which defines the residual function gθ(⋅) for the DEQ solver.
Figure 3: Illustration of TST. The output state from the previous frame is reused to initialize the current frame, allowing faster and more stable disparity refinement.
Table 4
Figure 4: Visualizations of disparity results on SceneFlow dataset . Compared to CODD, TC-Stereo (video-based) and IGEV-Stereo (per-frame), our method shows more temporally consistent disparity predictions ( red box ) and better preserves object boundaries in ill-posed regions where temporal cues are unavailable ( black box ).
Type
Method
NOC
ALL
Time (s)
D1-bg
D1-fg
D1-all
D1-bg
D1-fg
D1-all
(%)
(%)
(%)
(%)
(%)
(%)
Per frame
RAFT-Stereo
1.45
2.94
1.69
1.58
3.05
1.82
0.30
IGEV-Stereo
1.27
2.62
1.53
1.38
2.67
1.59
0.34
Selective-IGEV
1.22
2.55
1.44
1.33
2.61
1.55
0.45
MoCha-Stereo
1.24
2.42
1.44
1.36
2.43
1.53
0.50
Table 3: Quantitative results on KITTI2015. A method † requires camera parameters as auxiliary input. Note that the TST cannot be applied to per-frame methods.
Figure 5: Visualization of disparity results on ETH3D SLAM dataset . DEQ-Stereo produces accurate and stable disparities across diverse real-world scenes, demonstrating robustness to photometric variations and challenging scene structures.
Table 8Table 9Figure 10
Iterations
1
2
4
8
16
Adaptive
w/o Temporal Cue
0.64
0.53
0.47
0.42
0.39
0.39
Fixed-point Reuse
13.39
9.18
6.79
5.77
4.94
6.70
TST
0.59
0.52
0.47
0.42
0.40
0.39
Table 8: Comparison of TST and Fixed-point Reuse on TartanAir.
Existing iterative stereo matching methods primarily adopt two types of correspondence representation: explicit matching search via correlation volumes and local residual refinement via warped features, yet the two remain separately modeled. We propose WAVE-Stereo, built on a core insight: correlation volumes and feature warping provide complementary matching cues. GeoWarp Correspondence Encoder (GWCE) encodes matching search, residual alignment, and disparity prior in parallel at the ConvGRU input. To mitigate matching degradation in textureless regions, we propose Periodic Global Context Propagation (PGCP), which propagates global spatial information in a periodic manner. On five real-world benchmarks -- Middlebury, ETH3D, KITTI 2012, KITTI 2015, and Booster -- WAVE-Stereo achieves competitive zero-shot generalization accuracy without any external foundation model prior, achieving 3.18% D1-all on KITTI 2015, 4.42% Bad-2.0 on Booster, and 66ms real-time inference, striking a favorable balance between accuracy and efficiency. Our code is available at https://github.com/yamanoko-do/WAVE-Stereo.
Zehan Liu, Yage He, Xianwu Gong
School of Electronics and Control Engineering Chang’an University Xi’an, China
While iterative stereo matching achieves high accuracy, its dependence on Recurrent Neural Networks (RNN) hinders edge deployment, a challenge underexplored in existing researches. We analyze iterative refinement and reveal that disparity updates are spatially sparse and temporally redundant. First, we introduce a progressive iteration pruning strategy that suppresses redundant update steps, effectively collapsing the recursive computation into a near-single-pass inference. Second, we propose a collaborative monocular prior transfer framework that implicitly embeds depth priors without requiring a dedicated monocular encoder, thereby eliminating its associated computational burden. Third, we develop FlashGRU, a hardware-aware RNN operator leveraging structured sparsity and I/O-conscious design, achieving a 7.28× speedup, 76.6% memory peak reduction and 80.9% global memory requests reduction over natvie ConvGRUs under 2K resolution. Our PipStereo enables real-time, high-fidelity stereo matching on edge hardware: it processes 320×640 frames in just 75ms on an NVIDIA Jetson Orin NX (FP16) and 19ms on RTX 4090, matching the accuracy of large iterative based models, and our generalization ability and accuracy far exceeds that of existing real-time methods. Our embedded AI projects will be updated at: https://github.com/XPENG-Aridge-AI.
Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. Some methods treat monocular depth priors as guidance for iteration, but conventional GRU-based architectures struggle to exploit them due to the limited representation capacity. In this paper, we propose Prompt Recurrent Unit (PRU), a novel iterative refinement module based on the decoder of monocular depth foundation models. By integrating monocular structure and stereo motion cues as prompts into the decoder, PRU enriches the latent representations of monocular depth foundation models with absolute stereo-scale information while preserving their inherent monocular depth priors. Experiments demonstrate that our PromptStereo achieves state-of-the-art zero-shot generalization performance across multiple datasets, while maintaining comparable or faster inference speed. Our findings highlight prompt-guided iterative refinement as a promising direction for zero-shot stereo matching.
Xianqi Wang, Hao Yang, Hangtian Wang +4
Huazhong University of Science and Technology · Optics Valley Laboratory