Cross-Vehicle 3D Geometric Consistency for Self-Supervised Surround Depth Estimation on Articulated Vehicles

Weimin Liu; Jiyuan Qiu; Wenjun Wang; Joshua H. Meng

cs.CVApr 3, 2026

Cross-Vehicle 3D Geometric Consistency for Self-Supervised Surround Depth Estimation on Articulated Vehicles

Authors: Weimin Liu, Jiyuan Qiu, Wenjun Wang, Joshua H. Meng

Organizations: School of vehicle and Mobility, Tsinghua University, Beijing, China · Remote Sensing and Earth Observation Laboratory, University of Copenhagen, Copenhagen K, Denmark · California PATH, University of California, Berkeley, CA, USA

Abstract

Surround depth estimation provides a cost-effective alternative to LiDAR for 3D perception in autonomous driving. While recent self-supervised methods explore multi-camera settings to improve scale awareness and scene coverage, they are primarily designed for passenger vehicles and rarely consider articulated vehicles or robotics platforms. The articulated structure introduces complex cross-segment geometry and motion coupling, making consistent depth reasoning across views more challenging. In this work, we propose ArticuSurDepth, a self-supervised framework for surround-view depth estimation on articulated vehicles that enhances depth learning through cross-view and cross-vehicle geometric consistency guided by structural priors from vision foundation model. Specifically, we introduce multi-view spatial context enrichment strategy and a cross-view surface normal constraint to improve structural coherence across spatial and temporal contexts. We further incorporate camera height regularization with ground plane-awareness to encourage metric depth estimation, together with cross-vehicle pose consistency that bridges motion estimation between articulated segments. To validate our proposed method, an articulated vehicle experiment platform was established with a dataset collected over it. Experiment results demonstrate state-of-the-art (SoTA) performance of depth estimation on our self-collected dataset as well as on DDAD, nuScenes, and KITTI benchmarks.

Figures & tables

Fig. 1 : Surround depth estimation for articulated vehicle.

Fig. 2 : Overview: (a) Network architecture of ArticuSurDepth; (b) Self-supervised training framework and its loss components: (Left) Within- and cross-vehicle spatial context enrichment. Example: for the target view $C_{5}$ , the within-vehicle right view is $C_{6}$ , while the type-2 cross-vehicle right view is $C_{0}$ . (Right) Cross-view pseudo surface normal consistency ( $\mathcal{L}_{\text{PNC}}$ ).

Fig. 3 : Example of cross-vehicle extrinsics calibration: (a) LiDAR pointclouds registration; (b) Pointclouds of $\mathcal{L}_{f}$ projected on camera $C_{6}$ (mounted on front vehicle); (c) Pointclouds of $\mathcal{L}_{r}$ projected on camera $C_{6}$ ; (d) Within- and cross-vehicle spatial contexts and transformations.

Fig. 4 : Example of spatial warps: (a) Color image of $C_{5}$ ; (b) Spatial warp from left camera $C_{9}$ ; (c) Spatial cross-vehicle warp (type-2) from left camera $C_{2}$ ; (d) Spatial cross-vehicle warp (type-2) from right camera $C_{3}$ .

Type	Camera pairs
Type-0	$(C_{8}\&C_{2})$ , $(C_{7}\&C_{3})$
Type-1	$(C_{9}\&C_{2})$ , $(C_{6}\&C_{3})$ , $(C_{8}\&C_{1})$ , $(C_{7}\&C_{4})$
Type-2	$(C_{5}\&C_{2})$ , $(C_{5}\&C_{3})$ , $(C_{9}\&C_{1})$ , $(C_{6}\&C_{4})$ , $(C_{8}\&C_{0})$ , $(C_{7}\&C_{0})$

TABLE I : List of camera pairs for cross-vehicle spatial contexts.

Fig. 5 : Comparison of depth and direct interpolation-based surface normal reprojection method. PSN implies pseudo surface normal. ST implies spatial-temporal context.

Fig. 6 : (a) Within- and (b) cross-vehicle pose consistency.

Fig. 7 : Our self-established experiment platform.

Fig. 8 : Self-occlusion masks overlaid on images: (a) $C_{2}$ , (b) $C_{3}$ , (c) $C_{7}$ , (d) $C_{8}$ are four cameras that have visible occlusion components on the other vehicle under certain motion cases.

Fig. 9 : Visualization on val split: (a) Examples of depth and surface normal estimation on self-collected dataset; (b) Example of surround-view depth estimation on test split of self-collected dataset; (c) Examples of direct inference on DDAD; (d) Examples of direct inference on nuScenes.

KITTI [ 20 ] (192 $\times$ 640)
Method	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE ${}_{\text{log}}$ $\downarrow$	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	$\delta_{3}$ $\uparrow$
MonoViT [ 23 ]	0.099	0.708	4.372	0.175	0.900	0.967	0.984
EDS-Depth [ 24 ]	0.095	0.619	4.184	0.170	0.905	0.969	0.985
GeoDepth [ 10 ]	0.094	0.641	4.175	0.167	0.906	0.970	0.986
Ours ${\dagger}$	0.094	0.628	4.134	0.165	0.908	0.971	0.986
DDAD [ 21 ] (384 $\times$ 640)

TABLE II : Depth estimation on KITTI [ 20 ] , DDAD [ 21 ] , nuScenes [ 22 ] and our own self-collected datasets (Results on KITTI are scale-ambiguous evaluation.

{\dagger}

indicates our modified GeoDepth or GeoSurDepth with proposed losses. # implies reproduced results by VFDepth [ 7 ] . * indicates modified framework for adaptation to articulated vehicle pattern. Results best in bold , second best underlined ).

$S_{0}$	$S_{1}$	$S_{2}$	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE ${}_{\text{log}}$ $\downarrow$	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	$\delta_{3}$ $\uparrow$
✗	✗	✗	0.192	1.235	6.237	0.259	0.656	0.921	0.973
✓	✗	✗	0.190	1.193	6.069	0.255	0.661	0.926	0.975
✗	✓	✗	0.187	1.165	5.983	0.251	0.667	0.929	0.976
✗	✗	✓	0.188	1.173	5.992	0.252	0.666	0.929	0.975
✓	✓	✓	0.185	1.153	5.973	0.250	0.670	0.931	0.976

TABLE III : Ablation study on cross-vehicle context enrichment (

S_{i}

indicate type-

i

cross-vehicle spatial contexts).

Method	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE ${}_{\text{log}}$ $\downarrow$	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	$\delta_{3}$ $\uparrow$
w/o $\mathcal{L}_{\text{PNC}}$	0.189	1.173	6.000	0.253	0.664	0.928	0.976
with $\tilde{\mathcal{L}}_{\text{NC}}$	0.186	1.162	5.981	0.250	0.668	0.930	0.976
Ours	0.185	1.153	5.973	0.250	0.670	0.931	0.976

TABLE IV : Ablation study on cross-view SN consistency (“D” indicates depth interpolation-based SN reprojection.

Fig. 10 : Example of ground plane detection or estimation, and camera height estimation (range of camera height estimation has been clipped for visualization).

Method	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMSE $\downarrow$	RMSE ${}_{\text{log}}$ $\downarrow$	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	$\delta_{3}$ $\uparrow$
w/o $\mathcal{L}_{\text{CH}}$	0.189	1.254	6.229	0.258	0.667	0.921	0.973
with $\hat{\boldsymbol{\mathcal{M}}}_{g}$	0.202	1.454	6.491	0.268	0.640	0.911	0.969
Ours	0.185	1.153	5.973	0.250	0.670	0.931	0.976

TABLE V : Ablation study on ground-plane aware camera height regularization (

\hat{\boldsymbol{\mathcal{M}}}_{g}

indicates ground-plane detected with estimated depth and surface normal).

Method	Abs Rel $\downarrow$		$\delta_{1}$ $\uparrow$
Method	$J_{f}-J_{r}$	$C_{6}-C_{4}$	$J_{f}-J_{r}$	$C_{6}-C_{4}$
w / w.o $\mathcal{L}_{\text{VPC}}$	0.185 / 0.188	0.187 / 0.189	0.670 / 0.665	0.665 / 0.664

TABLE VI : Ablation study on cross-vehicle pose consistency.

Self-Supervised Monocular Depth Estimation

See all 17 papers

Sep 4, 2026

CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation

Reliable 3D understanding of the surrounding environment is a core requirement for autonomous driving. Multi-view surround camera rigs provide broad scene coverage, but the spatially adjacent images typically overlap only minimally. Consequently, the depth of most pixels must be inferred from monocular appearance cues. These cues can appear differently across images and may therefore be interpreted differently by the depth estimation model. We target two main sources of cross-image inconsistency: differences in camera intrinsics and the limited receptive field of each image. We address the former by conditioning the features on per-pixel camera-aware ray embeddings, enabling the network to account for camera-dependent variations in monocular cues. We address the latter by extending each pixel's context beyond its own image through cross-image attention constrained to geometrically plausible regions, derived from the calibrated rig setup. The model is trained in a fully self-supervised manner based on photometric consistency. Evaluations on DDAD and nuScenes show improved overall depth accuracy and cross-image depth consistency over state-of-the-art self-supervised methods under in-domain and cross-domain evaluation. Code is available at https://abualhanud.github.io/CrossDepthPage/.

Samer Abualhanud, Max Mehltretter

Leibniz University Hannover Hannover, Germany

Cross-View Attention Multi-View Depth Estimation Autonomous Driving Perception Multi-View Consistency Cross-Attention

Ask PDF

Jun 15, 2026

SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving

Modern autonomous driving depends on accurate metric 3D understanding for perception, reconstruction, and planning, which in turn requires reliable multi-camera depth prediction. However, the outward-facing nature of vehicle-mounted surround-view camera rigs inherently limits visual overlap across views, challenging the correspondence-based assumptions that underpin conventional multi-view geometry. To bridge this gap, we present SurroundNEXO, named after the Spanish word nexo for a geometric link, a low-overlap multi-camera metric depth framework that grounds cross-view reasoning in ego-centric geometry rather than dense visual correspondences. Instead of directly enforcing early global fusion, SurroundNEXO first assigns image tokens globally comparable ego-frame viewing directions through Ego-Ray Positional Encoding, then uses sparse LiDAR measurements as metric anchors to propagate absolute scale cues, and finally expands feature interaction progressively from view-local modeling to decomposed spatio-temporal reasoning and global integration. This design enables metric-scale depth prediction with improved spatial consistency across weakly overlapping cameras. Across low-overlap autonomous driving benchmarks, including NuScenes, Waymo and DDAD, SurroundNEXO reduces single-view error by 33.2%, improves cross-view consistency by 10.5%, and enhances metric reconstruction quality by 25.6% compared with SOTA methods. It further remains robust under extremely sparse depth prompts and exhibits strong zero-shot generalization to unseen camera layouts.

Shuai Yuan, Runxi Tang, Yuzhou Ji +8

School of Artificial Intelligence, Shanghai Jiao Tong University · 2Hello Inc.

Multi-View Depth Estimation Multi-View Consistency Metric Depth Estimation Depth Estimation Camera-LiDAR Fusion Autonomous Driving Perception

Ask PDF

Jun 14, 2026

3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation

Reliable monocular video depth estimation is crucial for downstream 3D reasoning and embodied AI in endoscopic navigation. However, existing self-supervised approaches typically treat video frames independently or rely on weak temporal regularization. These methods, lacking a holistic perception of the underlying 3D scene, inevitably suffer from geometrically inconsistent predictions and severe cross-frame drift. To address these limitations, we introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem, enabling full exploitation of the powerful geometric priors embedded in recent 3D foundation models. The core of our approach is a 3D consistency optimization framework driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency. Such unified optimization elegantly anchors isolated frames to a globally coherent 3D structure. Our method has been validated in both the self-supervised training scenarios and challenging zero-shot clinical environments. Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.

Yuanye Liu, Ke Zhang, Junzhe Jiang +3

Fudan University, Shanghai, China · Johns Hopkins University, Baltimore, USA

Depth Estimation Monocular Depth Estimation Self-Supervised Monocular Depth Estimation Video Depth Estimation

Ask PDF

Abstract

Self-Supervised Monocular Depth Estimation

Explore similar work

CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation

SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving

3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation