PTC-Depth: Pose-Refined Monocular Depth Estimation with Temporal Consistency

Leezy Han; Seunggyu Kim; Dongseok Shim; Hyeonbeom Lee

cs.CVApr 2, 2026

PTC-Depth: Pose-Refined Monocular Depth Estimation with Temporal Consistency

Authors: Leezy Han, Seunggyu Kim, Dongseok Shim, Hyeonbeom Lee

Organizations: Ajou University · Sony Creative AI Lab

Abstract

Monocular depth estimation (MDE) has been widely adopted in the perception systems of autonomous vehicles and mobile robots. However, existing approaches often struggle to maintain temporal consistency in depth estimation across consecutive frames. This inconsistency not only causes jitter but can also lead to estimation failures when the depth range changes abruptly. To address these challenges, this paper proposes a consistency-aware monocular depth estimation framework that leverages wheel odometry from a mobile robot to achieve stable and coherent depth predictions over time. Specifically, we estimate camera pose and sparse depth from triangulation using optical flow between consecutive frames. The sparse depth estimates are used to update a recursive Bayesian estimate of the metric scale, which is then applied to rescale the relative depth predicted by a pre-trained depth estimation foundation model. The proposed method is evaluated on the KITTI, TartanAir, MS2, and our own dataset, demonstrating robust and accurate depth estimation performance.

Figures & tables

Figure 2: The overall framework of the proposed method. Camera pose and metric scale are estimated from optical flow and wheel odometry, respectively, and used to fuse triangulated metric depth with relative depth from a foundation model.

Figure 3: Triangulation and temporal depth propagation. The posterior depth $z^{\mathrm{post}}_{i-1}$ from frame $i{-}1$ is lifted to the 3D point $\mathbf{p}_{i}$ , transformed by the estimated pose $(\boldsymbol{\Omega},\boldsymbol{T})$ , and projected into frame $i$ to form the temporal prior $z^{\mathrm{prior}}_{i}$ . Independently, triangulation from optical flow provides a new metric observation. Bayesian fusion combines the temporal prior with this observation to produce the refined posterior $z^{\mathrm{post}}_{i}$ , which is recursively propagated to the next frame as $z^{\mathrm{prior}}_{i+1}$ .

Figure 4: Depth estimation on various datasets. We evaluated our method on both real-world RGB datasets, such as KITTI, and synthetic RGB datasets, such as TartanAir. We also perform the depth estimation on thermal and NIR imagery using the MS2 dataset as well as our newly collected dataset.

Figure 5: Comparison result of 3D reconstruction. Compared to conventional depth estimation, our approach produces reliable depth closely aligned to the ground-truth data. Furthermore, when multiple frames are accumulated, the reconstructed point clouds exhibit sufficient geometric consistency.

Figure 6: Comparison result of accumulated pointcloud. Through the accumulation of multiple point clouds, our algorithm consistently produces temporally coherent depth estimates.

Figure 7: AbsRel error over time. The line plot shows the depth error at every individual frame, end of each curve indicates the average error across the image sequence.

	RGB								Thermal
	KITTI		TartanAir		Roadside		Forest		MS2		Roadside
Method	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$
MADPose (UniDepth)	0.115	0.869	0.481	0.187	0.423	0.222	0.473	0.097	1.381	0.179	0.509	0.149
Ours † (DA v2 rel)	0.115	0.867	0.239	0.645	0.227	0.649	0.326	0.540	0.950	0.332	0.346	0.484
GT Pose	0.130	0.839	0.168	0.754	-	-	-	-	1.465	0.073	-	-

Table 1: Triangulated depth evaluation. Each method triangulates sparse depth using its own pose source. MADPose relies on a pretrained metric depth model (UniDepth); ours uses odometry only. GT Pose serves as a reference to illustrate the effect of temporal misalignment in real sequences. Errors are robust means over valid pixels (top-90%).

	RGB											Thermal
	KITTI			TartanAir			Roadside			Forest		MS2			Roadside
Method	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$
Full range (0–80 m)
UniDepth	0.047	0.977	4.34	0.503	0.176	11.11	0.465	0.201	11.92	0.444	0.088	0.205	0.698	5.84	0.394	0.245	11.92
DA v2 (metric)	0.171	0.773	5.21	0.513	0.372	5.53	0.494	0.177	3.28	0.418	0.336	0.405	0.187	4.87	0.527	0.193	2.89
VDA	0.356	0.321	5.06	0.599	0.342	4.44	2.198	0.010	1.98	1.339	0.041	0.590	0.078	3.67	2.275	0.011	1.98

Table 2: Depth estimation and temporal consistency. Bold : best, underline : second best per dataset. VDA [ 5 ] is a video-based depth model evaluated for temporal consistency (TAE); DA v2 metric uses the publicly available outdoor model fine-tuned on Virtual KITTI.

	KITTI		TartanAir		Roadside		Forest
Method	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$
Near range (0–20 m)
UniDepth	0.039	0.989	0.485	0.202	0.432	0.241	0.428	0.096
DA v2 (metric)	0.154	0.821	0.647	0.322	0.508	0.168	0.435	0.331
VDA	0.313	0.376	0.575	0.341	1.369	0.106	1.081	0.105
Ours	0.128	0.880	0.339	0.712	0.165	0.860	0.304	0.617

Table 3: Depth estimation accuracy across near and far ranges. We separately evaluate near (0–20 m) and far (20–80 m) regions on RGB sequences.

	KITTI			Roadside			TartanAir
Method	AbsRel	$\delta\!<\!1.25$	TAE	AbsRel	$\delta\!<\!1.25$	TAE	AbsRel	$\delta\!<\!1.25$	TAE
w/o fusion	0.088	0.945	6.28	0.428	0.662	12.99	-	0.517	7.84
w/o segment	0.088	0.952	4.38	0.270	0.746	6.16	0.274	0.685	5.40
Full	0.087	0.953	4.79	0.265	0.743	6.27	0.218	0.714	4.65

Table 4: Ablation study. Effect of removing Bayesian temporal fusion (w/o fusion) and segment-wise scale consolidation (w/o segment). Each dataset is evaluated on a single representative sequence.

Panel of Figure 11 — Figure S1 : Data collection platform. (Left) Sensor module with FLIR Boson 640, Intel RealSense D455, and Livox HAP LiDAR. (Right) Hunter SE platform in a forest environment.

Figure S2 : Overall Process of Our Proposed Algorithm

Figure S3 : Comparison of triangulated vs. refined point clouds. (Left) Raw triangulated depth produces scattered points and surface discontinuities due to imperfect flow and unstable geometry. (Right) After our Bayesian update and superpixel refinement, the reconstructed surfaces become smoother and more consistent.

Figure S4 : LAB-based vs. RGB superpixel segmentation. (Left) Our LAB+depth segmentation follows true object and geometric contours, making each label more consistent with the underlying 3D structure. (Right) RGB-based segmentation often misaligns with physical surface.

	TartanAir (Synthetic RGB)			MS2 (Thermal)
Pose setting	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$	AbsRel $\downarrow$	$\delta\!<\!1.25\uparrow$	TAE $\downarrow$
Estimated $\boldsymbol{\Omega}$ , $\boldsymbol{T}$	0.218	0.714	4.65	0.513	0.603	5.75
GT $\boldsymbol{\Omega}$ only	0.183	0.755	4.45	0.254	0.624	5.91
GT $\boldsymbol{\Omega}$ , $\boldsymbol{T}$	0.172	0.799	4.19	0.974	0.370	–

Table S1 : Ablation on odometry accuracy. TartanAir: neighborhood (4.2K frames), MS2: 2021-08-13-21-18-04 (10K frames). On TartanAir, GT pose consistently improves accuracy thanks to perfect synchronization. On MS2, full GT pose degrades performance due to temporal misalignment between thermal images and RTK-GPS.

	Seg+Flow	Motion	Scale	Tri+Fusion	C++ Total	+Depth
Time (ms)	43	25	15	6	90	156

Table S2 : Runtime breakdown at KITTI resolution (

376\times 1241

). The full pipeline, including monocular depth estimation, achieves 6.4 FPS .

Figure S5 : Failure cases. (Left) Forward-dominant motion in KITTI Seq 42 causes degenerate triangulation, resulting in unreliable depth observations. (Right) Scenes with dominant dynamic objects produce inconsistent optical flow, which degrades pose estimation and subsequent depth fusion.

Depth Estimation

Depth estimation infers scene distance from images and sensors, enabling 3D reconstruction, navigation, and spatial understanding.

See all 198 papers

STATIC: Surface-normal-guided Two-branch Alignment and Temporal Integration for Consistent Video Depth Estimation
Dec 2, 2024Sunghun Yang, Minhyeok Lee, Suhwan Cho +2Depth Estimation Video Depth Estimation
Yonsei University
3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation
Jun 14, 2026Yuanye Liu, Ke Zhang, Junzhe Jiang +3Depth Estimation Monocular Depth Estimation
Fudan University, Shanghai, China · Johns Hopkins University, Baltimore, USA
AsyncMDE: Real-Time Monocular Depth Estimation via Asynchronous Spatial Memory
Mar 11, 2026Lianjie Ma, Yuquan Li, Bingzheng Jiang +3Edge Inference Monocular Depth Estimation
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. · Yichang Testing Technology Research Institute, Yichang, China. · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China.

Dec 2, 2024J/K move · Enter open · S save

STATIC: Surface-normal-guided Two-branch Alignment and Temporal Integration for Consistent Video Depth Estimation

Sunghun Yang, Minhyeok Lee, Suhwan Cho, Jungho Lee+1

Yonsei University

Depth Estimation Video Depth Estimation Monocular Depth Estimation

Open paper AskPDF arXiv

Video depth estimation is essential for applications such as autonomous driving, AR/VR, and robotics. Recent transformer-based single-image monocular depth estimation models perform well on a single image but struggle with depth consistency across video frames. Traditional methods aim to improve temporal consistency using multi-frame temporal modules or explicit priors like optical flow and camera parameters. However, these approaches face issues such as high memory use, reduced performance with dynamic or irregular motion, and a heavy reliance on the accuracy of external motion estimators. We propose STATIC, a novel model that independently learns temporal consistency in geometrically stable and unstable regions using only two adjacent frames. By leveraging intrinsic geometric cues rather than external motion priors, STATIC generates a depth-derived geometry-aware difference mask from surface normals to identify regions with inter-frame geometric variation by measuring directional variance. For geometrically stable regions, the Masked Stable (MS) module enhances temporal consistency by focusing on structurally stable regions. For geometrically unstable regions, the Surface Normal Similarity (SNS) module models inter-frame correspondences and ensures temporal consistency by measuring feature similarities between frames. A final refinement integrates the independently learned stable and unstable features, enabling STATIC to improve consistent depth prediction across consecutive frames. Experiments on KITTI, NYUv2, and Sintel demonstrate strong depth accuracy and temporal consistency, as well as stable performance across the tested frame-rate and camera-motion conditions.

Abstract

Depth Estimation

Explore similar work

STATIC: Surface-normal-guided Two-branch Alignment and Temporal Integration for Consistent Video Depth Estimation

3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation

AsyncMDE: Real-Time Monocular Depth Estimation via Asynchronous Spatial Memory