Single-Eye View: Monocular Real-time Perception Package for Autonomous Driving

Haixi Zhang; Aiyinsi Zuo; Zirui Li; Chunshu Wu; Tong Geng; Zhiyao Duan

cs.CVMar 22, 2026

Single-Eye View: Monocular Real-time Perception Package for Autonomous Driving

Authors: Haixi Zhang, Aiyinsi Zuo, Zirui Li, Chunshu Wu, Tong Geng, Zhiyao Duan

Organizations: Department of Electrical and Computer Engineering, University of Rochester, Rochester, NY 14627, USA · Faculty of the Department of Electrical and Computer Engineering, University of Rochester, Rochester, NY 14627, USA

Abstract

Amidst the rapid advancement of camera-based autonomous driving technology, effectiveness is often prioritized with limited attention to computational efficiency. To address this issue, this paper introduces LRHPerception, a real-time monocular perception package for autonomous driving that uses single-view camera video to interpret the surrounding environment. The proposed system combines the computational efficiency of end-to-end learning with the rich representational detail of local mapping methodologies. With significant improvements in object tracking and prediction, road segmentation, and depth estimation integrated into a unified framework, LRHPerception processes monocular image data into a five-channel tensor consisting of RGB, road segmentation, and pixel-level depth estimation, augmented with object detection and trajectory prediction. Experimental results demonstrate strong performance, achieving real-time processing at 29 FPS on a single GPU, representing a 555% speedup over the fastest mapping-based approach.

Figures & tables

Fig. 1 : Innovation and architecture blueprint a) Paradigm of end-to-end solution b) Paradigm of camera-fusion for local map solution c) Paradigm of our LRHPerception package, extracts essences from monocular camera for cost-info trade-off.

Fig. 2 : Granular Model Structure.1 Design of convolution decoder, object tracking, trajectory prediction, and depth estimation; magnify for details. BTAE mechanism in Algorithm 1. Remaining components are shown in Fig. 3.

Fig. 3 : Convolution Kernel & Transformation Visualization On the right, blue and green points represent $\mathcal{P}_{t-1}$ and $\mathcal{P}_{t}$ . Purple and orange arrows denote displacement and rotational transform. Red dashed box is the predicted object position from KF.

Fig. 4 : Granular Model Structure.2 Design of road segmentation block, along with other components.

Fig. 5 : Results Visualization The quartet of images depicts the output of LRHPerception, with past trajectories delineated in blue and future trajectory predictions in red. The upper pair of images exemplifies two Success Cases , while the lower duo present one Failure Case .

Methods	Detect	MOTA $\uparrow$	IDF1 $\uparrow$	IDP $\uparrow$	Time(ms)
C-BYTE	YoloX	76.9%	81.2%	85.9%	31.0
OC(2023) [ 46 ]	YoloX	74.1%	77.8%	87.2 %	28.3
Byte(2022) [ 22 ]	YoloX	76.6%	79.3%	84.0%	27.1
BoT(2022) [ 24 ]	YoloX	76.8%	81.0%	85.7%	48.2

TABLE I : Object Tracking. Our model manifests a significant improvement over SOTA across all efficacy and efficiency metrics on MOT datasets, corresponding harmoniously with our structure- fundamentally predicated on achieving refined outputs for robustness. Hyperparameter

\theta_{th}

and

a

are empirically determined as

0.9

and

210

Methods	MSE	$C_{MSE}$	$CF_{MSE}$	FPS $\uparrow$
Dataset: JAAD	(0.5/ 1.0/ 1.5s)	(1.5s)	(1.5s)	(8 / 12 / 24)
LRHP (Ours)	43/ 113/ 283	239	662	111 / 104/ 92.6
SGNet (2021)	82/ 328/ 1049	995	4076	2.8/ 2.8/ 2.7
Bitrap (2020)	93/ 378/ 1206	1105	4565	99.3/ 105/ 97.1
PIE_traj (2019)	110/ 399/ 1280	1183	4780	-/ -/ -
B-LSTM (2017)	159/ 539/ 1535	1447	5615	-/ -/ -

TABLE II : Trajectory Prediction. Our model exhibits significant enhancements in all aspects of efficacy and efficiency relative to SOTA methods, particularly pronounced as prediction time expands.

Methods	mIOU $\uparrow$	FPS $\uparrow$	Remarks
LRHP (Ours) ∗	88.9	55.0	∗ speed on dual
Road Segmentation (Yolo Backbone) ∗	88.4	96.4	tasks of detecti-
InternImage (2023) [ 36 ]	86.1	79.5	on and segmen-
MSeg (2020) [ 34 ]	77.6	65.7	tation
UJS-base (2021) [ 35 ]	80.5	-
UJS-refined (2021) [ 35 ]	88.3	-

TABLE III : Road Segmentation. Our model exhibits an appreciable acceleration and precision enhancement when operating an additional detection task with the YOLO backbone- a testament to the robustness and effectiveness of our architectural design.

Methods	Backbone	RMS	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	FPS $\uparrow$
LRHP (Ours)	Swin-m	0.229	0.966	0.996	42.0
Depth Estimate	Swin-L	0.216	0.975	0.997	13.3
VA-Depth (2023)	Swin-L	0.209	0.977	0.997	6.2
AdaBins (2021)	EffNet&Vit	0.236	0.964	0.995	1.7
BTS (2019)	DenseNet	0.280	0.955	0.993	20.1
ASTrans (2021)	Vit-B	0.269	0.963	0.995	-

TABLE IV : Depth Estimation. Our model showcases a significant acceleration in processing speed relative to SOTA, all the while maintaining comparable accuracy levels, thus underscoring the remarkable efficiency of our design.

Number of DualConv	Road-Seg (mIOU) $\uparrow$	Depth Estimation (RMS)	Time Difference ( $T_{n}\!-\!T_{n=1}$ )(ms)
$n=1$	88.5	0.223	0
$n=3$	88.5	0.219	0.268
$n=2$	88.9	0.220	0.049

TABLE V : Ablation study. We studied the effect of different numbers of Dual Convolutions in the C2f module on depth estimation and road segmentation.

Methods	Category	FPS $\uparrow$
LRHP(Ours)	Monocular	28.8
LRHP in series	Monocular	16.3
SOTA in series	Monocular	1.8
Uni-AD (2023) [ 13 ]	Multi-Cam Map	2.1
BEVerse-Tiny (2022) [ 14 ]	Multi-Cam Map	4.4
DETR3D (2021) [ 5 ]	Multi-Cam Map	2.0

TABLE VI : Entire Model. Our model witnesses an improvement of more than an order of magnitude over existing local mapping methods, a testament to the exceptional efficiency of our model. Note that Uni-AD’s planning module was removed for a fair comparison.

Autonomous Driving Perception

Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.

See all 379 papers

Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving
May 11, 2026Donghyun Kim, Jaehyoung ParkAdaptive Inference Lidar-Based 3D Object Detection
Stony Brook University
Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation
Jul 1, 2026Zhaowen Zhu, Li Zhang, Yujie Chen +3Depth Estimation Monocular Depth Estimation
Hefei University of Technology, Hefei, China · Chengpin Home Tech, Nanjing, China · Differential Robotics, Hangzhou, China +1
Depth-Aware Rover: A Study of Edge AI and Monocular Vision for Real-World Implementation
Apr 24, 2026Lomash Relia, Jai G Singla, Amitabh +1Metric Depth Estimation Robot Navigation
Department of Computer Engineering Devang Patel Institute of Advanced Technology and Research Anand, India · Space Applications Centre Indian Space Research Organization Ahmedabad, India

May 11, 2026J/K move · Enter open · S save

Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving

Donghyun Kim, Jaehyoung Park

Stony Brook University

Adaptive Inference Lidar-Based 3D Object Detection Autonomous Driving Perception 3D Object Detection Multi-Object Tracking

Open paper AskPDF arXiv

Autonomous driving scenes range from empty highways to dense intersections with dozens of interacting road users, yet current 3D detection models apply a fixed computation budget to every frame, wasting resources on simple scenes while lacking capacity for complex ones. Existing approaches compound this problem: Transformer-based interaction models scale quadratically with the number of detected objects, and frame-by-frame processing causes the system to immediately forget objects the moment they become occluded. We propose Enhanced HOPE, an adaptive perception architecture that measures the geometric complexity of each incoming LiDAR frame using an unsupervised statistical estimator and routes it through a shallow or deep processing path accordingly, requiring no manual scene labels. To keep interaction modeling efficient, we replace quadratic pairwise attention with a linear-time subspace-based network that groups nearby objects into clusters and processes them jointly. The computational savings from these two mechanisms free up resources for a persistent temporal memory module that retains previously detected objects and traffic rules across frames, enabling the system to recall occluded objects seconds after they disappear from view. On the nuScenes and CARLA benchmarks, Enhanced HOPE reduces latency by 38% on simple scenes with no accuracy loss, improves mean Average Precision by 2.7 points on rare long-tail scenarios, and tracks objects through occlusions lasting over 5 seconds, where all tested baselines fail.

Abstract

Autonomous Driving Perception

Explore similar work

Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving

Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation

Depth-Aware Rover: A Study of Edge AI and Monocular Vision for Real-World Implementation