cs.CVSep 4, 2026

CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation

Authors: Samer Abualhanud, Max Mehltretter

Organizations: Leibniz University Hannover Hannover, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Reliable 3D understanding of the surrounding environment is a core requirement for autonomous driving. Multi-view surround camera rigs provide broad scene coverage, but the spatially adjacent images typically overlap only minimally. Consequently, the depth of most pixels must be inferred from monocular appearance cues. These cues can appear differently across images and may therefore be interpreted differently by the depth estimation model. We target two main sources of cross-image inconsistency: differences in camera intrinsics and the limited receptive field of each image. We address the former by conditioning the features on per-pixel camera-aware ray embeddings, enabling the network to account for camera-dependent variations in monocular cues. We address the latter by extending each pixel's context beyond its own image through cross-image attention constrained to geometrically plausible regions, derived from the calibrated rig setup. The model is trained in a fully self-supervised manner based on photometric consistency. Evaluations on DDAD and nuScenes show improved overall depth accuracy and cross-image depth consistency over state-of-the-art self-supervised methods under in-domain and cross-domain evaluation. Code is available at https://abualhanud.github.io/CrossDepthPage/.

Figures & tables

Figure 1: Comparison of our method (red) with recent related methods (blue) in overall depth accuracy, given as RMSE [m], and cross-image depth consistency error [m]. Showing clear improvement in both metrics on DDAD dataset.
Figure 2: Overview of the proposed method. The depth network independently encodes the target images It\mathbf{I}_{t} . An initial forward pass predicts depth without cross-image attention, which is then used for cylindrical projection to restrict cross-image attention in a second forward pass, resulting in the final depth prediction D^\hat{D} . The pose network predicts the relative transformation It′,1T^It,1{}^{\mathbf{I}_{t^{\prime},1}}\hat{\mathbf{T}}_{\mathbf{I}_{t,1}} between the target It,1\mathbf{I}_{t,1} and source front images It′,1\mathbf{I}_{t^{\prime},1} .
Figure 3: Visualization of the flattened cylindrical representation, where overlapping image regions are mapped to nearby locations. For a query token (red star), candidate tokens in the other image are retrieved from a local neighborhood window on the cylinder (blue box) that spans multiple images. Valid cross-image candidates within this window are shown as blue dots in the other overlapping image.
  Overall (ID ∣| CD)Overlap (ID ∣| CD)
DatasetMethodAbs RelSq Rel [m]RMSE [m]δ<1.25\delta<1.25 [%]Abs RelDepth Cons [m]
DDADFSM0.201 ∣| -- ∣| -- ∣| -- ∣| -- ∣| -- ∣| -
FSM*0.228 ∣| -4.409 ∣| -13.43 ∣| -68.7 ∣| -- ∣| -- ∣| -
VFDepth0.218 ∣| 0.2943.660 ∣| 3.81713.32 ∣| 10.7867.4 ∣| 45.10.222 ∣| 0.3324.82 ∣| 7.27
SurroundDepth0.208 ∣| 0.3443.371 ∣| 4.30512.97 ∣| 11.7769.3 ∣| 36.10.217 ∣| 0.3687.86 ∣| 7.18
CVCDepth0.203 ∣| 0.2663.363 ∣| 2.91212.80 ∣| 9.3370.6 ∣| 56.60.204 ∣| 0.2795.89 ∣| 9.06
Table 1: Comparison on DDAD and nuScenes under in-domain (ID) and cross-domain (CD) evaluation, reported for the full image and overlapping regions. Each entry is shown as ID ∣| CD. For DDAD, CD denotes training on nuScenes and testing on DDAD; for nuScenes, CD denotes training on DDAD and testing on nuScenes. FSM* denotes results reproduced by [ 22 ] . Best results are shown in bold and second-best are underlined.
Figure 4: Comparison of depth maps predicted by our method and by state-of-the-art methods on DDAD. Our results show better preserved details and well-defined object boundaries (green bounding boxes). Depth is shown from close in yellow to distant in blue.
Figure 5: Exemplary depth consistency error maps, computed using the metric described in Sec. 4.1 overlayed on the front-image, comparing our approach with state-of-the-art methods on DDAD. Our method maps overlapping regions from the two images to nearby 3D coordinates. In contrast, the other methods exhibit a higher inconsistency in these regions. (a) shows the combined relevant regions from the front-left and front-right images that overlap with the front image. Errors are visualized ranging from black (low error) to yellow (high error).
MethodTrain [GB]Inference [GB]
FSM*5.60.5
VFDepth11.03.3
SurroundDepth12.61.4
CVCDepth6.00.7
CylinderDepth8.00.7
CrossDepth (ours)8.11.0
Table 2: Efficiency comparison of our method against state-of-the-art in terms of peak allocated memory during training and inference. FSM* denotes the implementation from [ 22 ] . Best results are shown in bold and second-best are underlined.
 OverallOverlap
MethodAbs RelSq RelRMSEδ<1.25\delta<1.25Abs RelDepth Cons
(a)0.1862.83811.9474.10.1895.63
(b)0.1932.94412.5371.80.1985.39
(c)0.1832.86911.9874.50.1874.82
(d)0.1882.96011.9873.80.1904.76
(e)0.1852.89411.8774.70.1864.60
Table 3: Ablation study on our method. (a) without cross-image attention; (b) unconstrained attention; (c) frozen backbone weights; (d) focal length scaling; (e) full method: constrained attention, ray embedding and fine tuned backbone. RMSE, Sq Rel and Depth Cons are given in [m]. δ\delta is given in [%]. Abs Rel is unit-free. Results are reported for the entire images and for overlapping regions on the DDAD dataset. Best results are shown in bold and second-best are underlined.
KWK_{W}Overall (Abs Rel)Overlap (Depth Cons) [m]
550.1854.60
10100.1875.14
20200.1875.55
Table 4: Ablation study on the neighborhood size of the constrained cross-image attention. Metrics are reported for overall depth accuracy and cross-image consistency within overlapping regions on DDAD. Abs Rel is unit-free. Best results are shown in bold and second-best are underlined.
More in this topic

Cross-View Attention

See all 9 papers

Explore similar work

CardsList

  1. Cross-Vehicle 3D Geometric Consistency for Self-Supervised Surround Depth Estimation on Articulated Vehicles

    Apr 3, 2026Weimin Liu, Jiyuan Qiu, Wenjun Wang +1Self-Supervised Monocular Depth EstimationDepth Estimation

    School of vehicle and Mobility, Tsinghua University, Beijing, China · Remote Sensing and Earth Observation Laboratory, University of Copenhagen, Copenhagen K, Denmark · California PATH, University of California, Berkeley, CA, USA

  2. 3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation

    Jun 14, 2026Yuanye Liu, Ke Zhang, Junzhe Jiang +3Depth EstimationMonocular Depth Estimation

    Fudan University, Shanghai, China · Johns Hopkins University, Baltimore, USA

  3. CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention

    Sep 21, 2026Xuezhi Xiang, Jiayao Liu, Heqi Xiang +3Monocular Depth EstimationMulti-Scale Feature Fusion

    School of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China · Key Laboratory of Advanced Marine Communication and Information Technology, Harbin 150001, China · Department of Computer Science, University of Toronto, Toronto ON M5S 2E4, Canada +1