Depth-Aware Image and Video Orientation Estimation
Authors: Muhammad Z. Alam, Larry Stetsiuk, M. Umair Mukati, Zeeshan Kaleem
Organizations: Faculty of Computer science, University of New-Brunswick · Department of Computer Science, Brandon University · Department of Electrical and Photonics Engineering, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark · Department of Electrical and Computer Engineering, COMSATS University
This paper introduces a novel approach for image and video orientation estimation by leveraging depth distribution in natural images. The proposed method estimates the orientation based on the depth distribution across different quadrants of the image, providing a robust framework for orientation estimation suited for applications such as virtual reality (VR), augmented reality (AR), autonomous navigation, and interactive surveillance systems. To further enhance fine-scale perceptual alignment, we incorporate depth gradient consistency (DGC) and horizontal symmetry analysis (HSA), enabling precise orientation correction. This hybrid strategy effectively exploits depth cues to support spatial coherence and perceptual stability in immersive visual content. Qualitative and quantitative evaluations demonstrate the robustness and accuracy of the proposed approach, outperforming existing techniques across diverse scenarios.
Figures & tables
Fig. 1 : Illustration of the orientation estimation pipeline.
Fig. 2 : Illustration of linear perspective in image acquisition.
Fig. 3 : Different viewing angles and their effect on depth displacement in corresponding disparity (inverse of depth) maps.
Method
Sun397 40 % test Accuracy
Sun397 20% test Accuracy
Vailaya et. al. [ 34 ]
80.1 %
–
Tolstaya [ 33 ]
85.6%
–
Ciocca et. al [ 12 ]
89.3%
–
Appia and Narasimah [ 8 ]
54.3%
–
Ciocca et. al [ 13 ]
92.4%
–
Joshi et. al [ 18 ]
–
98.5%
TABLE I : Performance comparison of the proposed method with existing image orientation estimation methods on two different test set splits on Sun397 [ 37 ] dataset adopted in various state-of-the-art methods.
Fig. 4 : Predictions of the proposed method on the subset of images used to produce the results in Table I . Orientation predictions by the proposed method using two different depth estimation techniques. (Depth-map) Image orientations predicted by the proposed method using complete depth-maps. (Dfd) Image orientations predicted by the proposed method using depth from defous estimates.
Fig. 5 : Comparison of the Proposed method with state-of-the-art OAD [ 22 ] on fine-scale (10°increment) image orientation estimation. Delta represents the acceptable error margin in calculating the accuracy.
Fig. 6 : A subset of images selected from four different categories to produce results in Fig. 5
Fig. 7 : Visualization of the image rotations tested for fine-scale image orientation to produce results in Fig. 5
Fig. 8 : Thumbnails of the selected subset of video sequences tested for the evaluation of the proposed orientation estimation technique.
Despite recent advances in Monocular Depth Estimation, state-of-the-art depth foundation models remain vulnerable to robustness issues. Particularly, even slight camera rolls can result in substantial degradation in depth estimations. We attribute this problem to a previously overlooked phenomenon, termed the Horizontal Prior, which is a manifestation of long-tailed distribution bias: most training images are captured in approximately horizontal orientations due to human visual preferences and photographic habits. While intuitive remedies such as re-balanced data augmentation and horizon leveling provide partial improvements, they fail to fully address the issue. In this paper, we introduce Invariant Depth Constraint (ID-Constraint), a training-time supervision strategy that improves roll robustness by fine-tuning and jointly regularizing the depth backbone with a series of geometric and spatial reasoning tasks. These auxiliary objectives encourage the backbone to learn rotation-stable, depth-relevant representations, while the auxiliary prediction heads are discarded after training, leaving the original inference architecture unchanged. Extensive experiments on five benchmark datasets across four roll settings demonstrate the effectiveness of the proposed method.
Existing video depth estimation faces a fundamental trade-off: generative models suffer from stochastic geometric hallucinations and scale drift, while discriminative models demand massive labeled datasets to resolve semantic ambiguities. To break this impasse, we present DVD, the first framework to deterministically adapt pre-trained video diffusion models into single-pass depth regressors. Specifically, DVD features three core designs: (i) repurposing the diffusion timestep as a structural anchor to balance global stability with high-frequency details; (ii) latent manifold rectification (LMR) to mitigate regression-induced over-smoothing, enforcing differential constraints to restore sharp boundaries and coherent motion; and (iii) global affine coherence, an inherent property bounding inter-window divergence, which enables seamless long-video inference without requiring complex temporal alignment. Extensive experiments demonstrate that DVD achieves state-of-the-art zero-shot performance across benchmarks. Furthermore, DVD successfully unlocks the profound geometric priors implicit in video foundation models using 163x less task-specific data than leading baselines. Notably, we fully release our pipeline, providing the whole training suite for SOTA video depth estimation to benefit the open-source community.
Hongfei Zhang, Harold Haodong Chen, Chenfei Liao +12
Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360∘ images due to the substantial geometric discrepancy between perspective and panoramic domains. Moreover, fully fine-tuning these models typically requires large amounts of panoramic data. To address this issue, we propose RePer-360, a distortion-aware self-modulation framework for monocular panoramic depth estimation that adapts depth foundation models while preserving powerful pretrained perspective priors. Specifically, we design a lightweight geometry-aligned guidance module to derive a modulation signal from two complementary projections (i.e., ERP and CP) and use it to guide the model toward the panoramic domain without overwriting its pretrained perspective knowledge. We further introduce a Self-Conditioned AdaLN-Zero mechanism that produces pixel-wise scaling factors to reduce the feature distribution gap between the perspective and panoramic domains. In addition, a cubemap-domain consistency loss further improves training stability and cross-projection alignment. By shifting the focus from complementary-projection fusion to panoramic domain adaptation under preserved pretrained perspective priors, RePer-360 surpasses standard fine-tuning methods while using only 1% of the training data. Under the same in-domain training setting, it further achieves an approximately 20% improvement in RMSE. The code is available at https://github.com/munimo/RePer360.
Cheng Guan, Chunyu Lin, Zhijie Shen +2
Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China. · Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing 100044, China.