A Study on Real-time Object Detection using Deep Learning

Ankita Bose; Jayasravani Bhumireddy; Naveen N

cs.CVFeb 17, 2026

A Study on Real-time Object Detection using Deep Learning

Authors: Ankita Bose, Jayasravani Bhumireddy, Naveen N

Organizations: Department of Computer Science and Engineering, GITAM University, India, 561203

Abstract

Object detection has compelling applications over a range of domains, including human-computer interfaces, security and video surveillance, navigation and road traffic monitoring, transportation systems, industrial automation healthcare, the world of Augmented Reality (AR) and Virtual Reality (VR), environment monitoring and activity identification. Applications of real time object detection in all these areas provide dynamic analysis of the visual information that helps in immediate decision making. Furthermore, advanced deep learning algorithms leverage the progress in the field of object detection providing more accurate and efficient solutions. There are some outstanding deep learning algorithms for object detection which includes, Faster R CNN(Region-based Convolutional Neural Network),Mask R-CNN, Cascade R-CNN, YOLO (You Only Look Once), SSD (Single Shot Multibox Detector), RetinaNet etc. This article goes into great detail on how deep learning algorithms are used to enhance real time object recognition. It provides information on the different object detection models available, open benchmark datasets, and studies on the use of object detection models in a range of applications. Additionally, controlled studies are provided to compare various strategies and produce some illuminating findings. Last but not least, a number of encouraging challenges and approaches are offered as suggestions for further investigation in both relevant deep learning approaches and object recognition.

Figures & tables

Figure 1: a) Architecture of CNN, b)Illustrates Convolution Operation, c) Illustrates pooling operations

CNN Variation	Details	Application
LeNet (1998)	First CNN architecture for digit recognition.	Recognition of digits (for instance, MNIST dataset)[41].
AlexNet (2012)	Advanced the use of deep CNNs featuring ReLU activation and training on GPUs	Visual categorization (e.g., ImageNet contest)[42][43]
ZFNet (2013)	Enhanced AlexNet by modifying filter dimensions and integrating feature visualization techniques.	Image classification and interpretability [44]
VGGNet (2014)	Used deeper architectures with small 3x3 filters. Demonstrated the importance of network depth.	Image categorization, attribute extraction for transfer learning.[45]
GoogLeNet (Inception-v1) (2014)	Introduced inception modules with multi-scale convolutions and dimensionality reduction (1x1 convolutions).	Object detection and image categorization [46].
ResNet (2015)	Implemented residual learning and skip connections to support extremely deep networks (reaching 152 layers).	Image categorization, object recognition, healthcare imaging [47].

Table 1: Variations of CNN invented so far

YOLO Variations	Details
YOLOv1 (2016)	The first version of YOLO [24] introduced the concept of performing object detection in a single pass through the network. It divided the image into a grid, predicting bounding boxes and class probabilities for each cell. While it was significantly faster than previous methods, YOLOv1 had lower accuracy, especially when detecting smaller objects.
YOLOv2 (2017)	YOLOv2 [27], also known as "YOLO9000," introduced several improvements, including a more powerful backbone network (Darknet-19), higher grid resolution, and the ability to detect over 9,000 object categories. It also incorporated batch normalization and anchor boxes, significantly boosting both detection accuracy and speed.
YOLOv3 (2018)	YOLOv3 [28] improved its architecture by adopting Darknet-53 as the backbone, providing a deeper and more efficient network compared to Darknet-19. It introduced multi-scale predictions, enabling better detection of objects across different sizes. This enhancement significantly improved the model’s ability to identify small objects and increased accuracy, particularly in complex scenarios.
YOLOv4 (2020)	YOLOv3 enhanced its architecture by integrating Darknet-53 as the backbone, offering a deeper and more efficient network than Darknet-19. It introduced multi-scale predictions, allowing for improved detection of objects of varying sizes [29]. These advancements significantly boosted the model with enhanced accuracy, especially in challenging and complex environments.
Variations of YOLOv4 (2020)	YOLOv4-CSP [30]- Uses the CSPDarknet53 backbone for improved feature extraction, optimizing speed and accuracy in real-time applications. YOLOv4-tiny [31]- It is a fast, lightweight version for real-time detection on resource-limited devices, prioritizing speed over accuracy for low-latency applications. Scaled-YOLOv4 [32]-Scaled-YOLOv4 optimizes YOLOv4 for scalability, using CSPNet, PANet, and SPP to enhance accuracy and speed.
YOLOv5 (2020)	YOLOv5 [33], developed by Ultralytics, is a widely adopted variant known for its speed, versatility, and ease of use. It introduced multiple model sizes (small, medium, large) to suit different hardware configurations. With an optimized training process, YOLOv5 enhances performance across various datasets, making it highly efficient for real-time object detection applications.

Table 2: Variations of YOLO invented so far

Name of the Model(Year)	Base Architecture	Data Set	Model Trained on	Mean-Average Precision (mAp)	Frames Per Second (FPS)
YOLOv1 (2016)	Inspired by GoogleNet. It has replaced the inception module with a (1 x 1) convolution followed by (3 x 3) convolution filters	VOC Pascal Dataset.	Darknet framework	63.4%	45
YOLOv2 (2017)	Darknet-19 backbone of YOLOv2 was inspired by VGG & ResNet. A typical darknet-19 framework consists of 19 convolutional layers and 5 max-pooling layers	PASCAL VOC 2007 Test Set, COCO	ImageNet, COCO	67.8%	67
YOLOv3 (2018)	Darknet-53 is the backbone of YOLOv3. It has 53 convolution layers with (3 x 3) and (1 x1) convolutional filters and it uses residual connections like ResNet	COCO	COCO, PASCAL VOC 2007 and VOC 2012	57.9%	50
YOLOv4 (2020)	CSPDarknet-53 is the backbone of YOLOv4, it is the improved version of Darknet-53. Cross-Stage Partial (CSP) connections are used for better gradient flow and reduced computation.	COCO, PASCAL VOC 2007 and VOC 2012	COCO, ImageNet	65.7%	62
YOLOv5 (2020)	YOLOv5 is completely written in PyTorch, CSP module is used for feature extraction, FPN and PANet is there for better multi-scale detection	COCO	COCO	36.7(s),50.7(x)	156(s), 72(x)
YOLOv6 (2022)	It uses a RepVGG or CSPStackRep-style backbone and BiFPN (Bidirectional Feature Pyramid Network)	COCO	COCO	35.9(N),52.5(N)	1231(N), 420(N)

Table 3: A Comparative Analysis of the YOLO Models

Figure 10: Architecture of Light Head RCNN

Figure 11: Classification of deep learning algorithms into one-stage detectors, two-stage detectors, lightweight detectors, and anchor-free detectors.

Dataset	Description	Type of Images	Dataset Source
DUTS	DUTS is a large saliency detection dataset with 10,553 training and 5,019 testing images, sourced from ImageNet DET and SUN datasets. It includes challenging cases, with pixel-level annotations by 50 individuals.	Living and non-living species	Kaggle
Car Detection	The dataset contains media of cars in all views. Here we see 1001 images for training and 175 for testing	cars	Kaggle
Gun Detection	Images folder contains pictures in jpeg format. Labels folder holds txt files where each file’s first line shows the number of items in the matching image, and the following lines include the coordinates of the box outlining the item.	Guns	Kaggle
Human Face detection	A varied collection of images showing human faces from different races, ages, and angles, intended to build an impartial dataset that contains coordinates of facial areas suitable for training models that detect objects.	Human Faces	Kaggle
Brain tumor detection	The datasets JPGs exported at their native size and are separated by plane (Axial, Coronal and Sagittal). Here we see 310 images for training and 75 for testing	Brain Tumor	Kaggle
26 Class Object Detection Dataset	This Dataset contains labeled images of 26 common city and outdoor objects, including vehicles, traffic signs, pedestrians, and natural elements.	City elements	Kaggle

Table 15

Algorithm	mAP(%)	IoU	Inference Time(ms)	Model Size (MB)
YOLOv4	91.3	88.5	22	250
Faster R-CNN	93.5	90.8	120	500
SSD	87.4	83.2	18	95
RetinaNet	90.8	88.0	30	120
DETR	89.6	87.5	45	150
EfficientDet	92.1	89.2	25	75

Table 4: A comparative analysis of evaluation metrices for COCO dataset

Figure 12: A comparative analysis of evaluation metrices of object detection models for COCO dataset

Algorithm	Accuracy(%)	Precision(%)	Recall(%)	F1-Score(%)	Inference Time(ms)	Model Size(MB)
Custom CNN	80.5	79.8	80.1	80.0	5	10
Alex Net	85.3	84.5	84.8	84.6	20	230
VGG	92.4	92.0	92.2	92.1	35	528
RestNet	94.5	94.2	94.4	94.3	25	98
MobileNet	89.7	89.3	89.5	89.4	10	14
EfficientDet	95.1	94.8	94.9	94.9	20	36

Table 5: A comparative analysis of evaluation metrices for CIFAR-10 dataset

Figure 13: A comparative analysis of evaluation metrices of image classification models for CIFAR-10 dataset

Algorithm	mAP(%)	Recall(%)	Precision(%)	Inference Time(ms)
Faster RCNN	88.4	91.2	87.6	150
YOLOv4	85.6	88.0	85.0	25
SSD	82.4	84.2	81.5	18
RetinaNet	86.5	89.5	85.7	40
CenterNet	87.3	89.8	86.4	30
Cascade RCNN	90.1	92.7	89.5	200

Table 6: A comparative analysis of evaluation metrices for Caltech Pedestrian Dataset

Figure 14: A comparative analysis of evaluation metrices for Caltech Pedestrian Dataset

Algorithm	PCK(%)	mAP(%)	Inference Time(ms)	Model Size(MB)
OpenPose	85.3	80.1	35	200
HRNet	90.2	87.5	50	220
PoseNet	78.5	70.3	12	50
DeepPose	70.5	65.1	25	150

Table 7: A comparative analysis of evaluation metrices for Skeleton Detection models on COCO Dataset

Figure 15: A comparative analysis of evaluation metrices for Skeleton Detection models on COCO Dataset

Algorithm	F-measure	MAE	S-measure	Inference Time (ms)
U-Net	0.78	0.045	0.85	20
RFCN	0.85	0.032	0.88	25
PiCANet	0.90	0.020	0.91	40
DSS	0.87	0.028	0.90	28

Table 8: A comparative analysis of evaluation metrices for DUTS Dataset

Figure 16: A comparative analysis of evaluation metrices for DUTS Dataset

Algorithm	Task	mAP	IoU(%)	Inference Time (ms)	Model Size (MB)
YOLOv4	Object Detection	91.4	-	22	240
Faster R-CNN	Object Detection	94.2	-	120	512
Mask R-CNN	Object Detection+ Segmentation	92.8	-	150	600
UNet	Semantic Segmentation	-	83.2	20	30
LaneNet	Lane Detection	-	89.1	18	50

Table 9: A comparative analysis of evaluation metrices for KITTI Dataset

Algorithm	Precision	Recall	F1-Score	mAP	FPS
MTCNN	0.904	0.880	0.891	0.883	15
Faster R-CNN	0.936	0.885	0.910	0.891	8
YOLOv4	0.879	0.860	0.865	0.865	62
SSD	0.873	0.840	0.854	0.848	30
RetinaFace	0.940	0.910	0.925	0.905	18
DETR	0.913	0.900	0.907	0.890	12

Table 10: A comparative analysis of evaluation metrices for WIDER FACE Dataset

Figure 17: A comparative analysis of evaluation metrices for WIDER FACE Dataset

Algorithm	Accuracy(↑)	TAR@FAR=0.1%(↑)	EER(↓)
DeepFace	97.35%	91.50%	3.5%
FaceNet	99.63%	97.00%	0.9%
VGGFace	98.95%	95.20%	1.2%
DeepID	98.97%	95.40%	1.1%
SphereFace	99.42%	96.90%	0.8%
CosFace	99.73%	97.40%	0.7%

Table 11: A comparative analysis of evaluation metrices for Labeled Faces in the Wild (LFW) Dataset

Figure 18: A comparative analysis of evaluation metrices for DUTS Dataset

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

Feb 26, 2026

Don't let the information slip away

Real-time object detection has advanced rapidly in recent years. The YOLO series of detectors is among the most well-known CNN-based object detection models and cannot be overlooked. The latest version, YOLOv26, was recently released, while YOLOv12 achieved state-of-the-art (SOTA) performance with 55.2 mAP on the COCO val2017 dataset. Meanwhile, transformer-based object detection models, also known as DEtection TRansformer (DETR), have demonstrated impressive performance. RT-DETR is an outstanding model that outperformed the YOLO series in both speed and accuracy when it was released. Its successor, RT-DETRv2, achieved 53.4 mAP on the COCO val2017 dataset. However, despite their remarkable performance, all these models let information to slip away. They primarily focus on the features of foreground objects while neglecting the contextual information provided by the background. We believe that background information can significantly aid object detection tasks. For example, cars are more likely to appear on roads rather than in offices, while wild animals are more likely to be found in forests or remote areas rather than on busy streets. To address this gap, we propose an object detection model called Association DETR, which achieves state-of-the-art results compared to other object detection models on the COCO val2017 dataset.

Taozhe Li, Guansu Wang, Bo Yu +2

University of Oklahoma · University of Melbourne · University of Utah

Object Detection Detection Transformer

Ask PDF

Oct 29, 2025

RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models

Real-time object detection has achieved substantial progress through meticulously designed architectures and optimization strategies. However, the pursuit of high-speed inference via lightweight network designs often leads to degraded feature representation, which hinders further performance improvements and practical on-device deployment. In this paper, we propose a cost-effective and highly adaptable distillation framework that harnesses the rapidly evolving capabilities of Vision Foundation Models (VFMs) to enhance lightweight object detectors. Given the significant architectural and learning objective disparities between VFMs and resource-constrained detectors, achieving stable and task-aligned semantic transfer is challenging. To address this, on one hand, we introduce a Deep Semantic Injector (DSI) module that facilitates the integration of high-level representations from VFMs into the deep layers of the detector. On the other hand, we devise a Gradient-guided Adaptive Modulation (GAM) strategy, which dynamically adjusts the intensity of semantic transfer based on gradient norm ratios. Without increasing deployment and inference overhead, our approach painlessly delivers striking and consistent performance gains across diverse DETR-based models, underscoring its practical utility for real-time detection. Our new model family, RT-DETRv4, achieves state-of-the-art results on COCO, attaining AP scores of

49.8/53.7/55.4/57.0

at corresponding speeds of

273/169/124/78

FPS. Code is publicly available at https://github.com/RT-DETRs/RT-DETRv4.

Zijun Liao, Yian Zhao, Xin Shan +5

School of Electronic and Computer Engineering, Peking University, Shenzhen, China · Department of Automation and BNRist, Tsinghua University, Beijing, China

Real-Time Object Detection Efficient Object Detection Detection Transformer Knowledge Distillation

Ask PDF

Jul 26, 2026

Neuromorphic Object Detection: An In-Depth Study and Future Directions

Conventional frame-based cameras face significant challenges in detecting objects under high-speed motion blur or in low-light environments. Neuromorphic cameras provide asynchronous visual streams with high temporal resolution and a wide dynamic range, offering a promising solution for object detection under challenging conditions. Despite the development of numerous models and the emergence of various applications in neuromorphic object detection, there is still a lack of deep understanding and standardized benchmarks to assess progress and address key challenges. In this paper, we provide a comprehensive survey and benchmark of existing neuromorphic object detection algorithms. Specifically, we first present a problem description, review the available datasets, and revisit the evaluation metrics. We then explore existing neuromorphic object detection approaches from various perspectives, including event representation, temporal modeling, multimodal fusion, asynchronous processing, low-latency processing, and energy-efficient computing. Furthermore, we evaluate a wide range of representative neuromorphic object detection models and offer detailed analyses of the comparative results. Finally, we discuss unresolved issues in neuromorphic object detection and propose potential future research directions. We hope this survey and benchmark will be a valuable resource for researchers and provide guidance for future advancements in neuromorphic object detection.

Jianing Li, Dianze Li, Arren Glover +5

Beijing Key Laboratory of Brain-inspired Spiking Large Models, School of Computer Science, Peking University, Beijing 100871, China · Peng Cheng Laboratory, Shenzhen 518000, China · Event-Driven Perception for Robotics, Italian Institute of Technology, Genoa 16163, Italy +6

Neuromorphic Computing Event-Based Object Detection

Ask PDF

Abstract

Object Detection

Explore similar work

Don't let the information slip away

RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models

Neuromorphic Object Detection: An In-Depth Study and Future Directions