PTC-IoU: CPU-Friendly Detection Quality Estimation for Edge Video Analytics
Organizations: Technical University of Munich
Abstract
An edge video analytics system must estimate the quality of local detections since there are no ground-truth labels available during runtime. Detectors usually provide confidence scores for their results, but these scores can not reveal every missed object or misplaced bounding box. So a quality estimator is necessary to provide a decision signal indicating which frame has potential detection failures that need to be escalated to a more powerful model in the cloud. Neural network based estimators add workload to the GPU already running detection, while CPU capacity may remain available. We present Predictive Temporal Consistency IoU (PTC-IoU), which moves this workload to CPU by compacting detection boxes histories to support deterministic result scoring. Comparing to neural counterparts, experiments on representative edge hardware, Nvidia Jetson Orin Nano, show that our method reduced per-frame estimator latency from 10.3 ms for neural counterparts to 2.2 ms (4.7x faster), increased pipelined throughput by 13.6%, and cuts estimator energy consumption by up to 95%, while maintaining competitive final detection quality verified on multiple datasets.
Figures & tables
| Item | Setting |
|---|---|
| Platform | Orin Nano Super, six-core Cortex-A78AE CPU, Ampere GPU. L4T R36.5, MAXN_SUPER, stock DVFS, Docker/PyTorch 2.2.0 |
| Detector pair | COCO-pretrained Faster R-CNN: MobileNetV3-Large FPN (local), ResNet-50 FPNv2 (stronger) |
| Local resolution | System tests: FPN-320, min/max side 320/640 pixels. Quality replay: 800/1333 pixels |
| Placement | CPU : PTC and XGBoost. GPU : IoU and GFL heads |
| System input | MOT17-02-FRCNN, batch 1, person boxes, confidence 0.1 for every estimator |
| Timing | Bursty workload, eight rounds, each estimator condition paired with the adjacent detector-only runs. Mean frame time derived from throughput |
| Method | UA-DETRAC | UAVDT | MOT17 |
|---|---|---|---|
| Local ( ) | 0.636 | 0.119 | 0.254 |
| Random | 0.670 | 0.184 | 0.290 |
| IoU head | 0.682 | 0.194 | 0.319 |
| GFL head | 0.679 | 0.227 | 0.313 |
| XGBoost | 0.682 | 0.231 | 0.326 |
| PTC (ours) | 0.680 | 0.214 | 0.313 |