Interpretable Human Activity Recognition for Subtle Robbery Detection in Surveillance Videos
Authors: Bryan Jhoan Cazáres Leyva, Ulises Gachuz Davila, José Juan González Fonseca, Juan Irving Vasquez, Vanessa A. Camacho-Vázquez, Sergio Isahí Garrido-Castañeda
Organizations: Escuela Superior de Cómputo, Instituto Politécnico Nacional, México City, México. · Centro de Innovación y Desarollo Tecnológico en Cómputo, Instituto Politécnico Nacional, México City, México.
Non-violent street robberies (snatch-and-run) are difficult to detect automatically because they are brief, subtle, and often indistinguishable from benign human interactions in unconstrained surveillance footage. This paper presents a hybrid, pose-driven approach for detecting snatch-and-run events that combines real-time perception with an interpretable classification stage suitable for edge deployment. The system uses a YOLO-based pose estimator to extract body keypoints for each tracked person and computes kinematic and interaction features describing hand speed, arm extension, proximity, and relative motion between an aggressor-victim pair. A Random Forest classifier is trained on these descriptors, and a temporal hysteresis filter is applied to stabilize frame-level predictions and reduce spurious alarms. We evaluate the method on a staged dataset and on a disjoint test set collected from internet videos, demonstrating promising generalization across different scenes and camera viewpoints. Finally, we implement the complete pipeline on an NVIDIA Jetson Nano and report real-time performance, supporting the feasibility of proactive, on-device robbery detection.
Figures & tables
Figure 1: General flow of the proposed method for the recognition of robbery in surveillance videos.
Rank
Feature
Imp. ( 10−2 )
Rank
Feature
Imp. ( 10−2 )
1
dist_p95
5.236
6
distancet_max
4.188
2
handToHip_max
4.802
7
handToTorso_median
3.340
3
handToTorso_mean
4.781
8
handToHip_p95
2.763
4
handToTorso_p95
4.297
9
distance_mean
2.459
5
handToTorso_max
4.239
10
closeHandPct
2.352
Table 1: Top 10 features ranked by Random Forest importance (Two columns).
Figure 2: Class visualization using PCA. As we can observe, the projected samples are in a general way distinguishable. This graph is only a projection of the feature vectors of 10 elements.
Figure 3: Representative frames from the training/validation video dataset recorded for this work. Faces were deliberately blurred for this paper.
Figure 4: Representative frames from the test video dataset. These videos are publicly available on YouTube. Faces were deliberately blurred for this paper.
Figure 5: Example sequence of the functioning of the implemented method. The sequence shows the detection of a robbery.
Class
Acc.
Prec.
Recall
F1-Score
Support
Non-Robbery
0.83
0.91
0.83
0.87
12
Robbery
0.83
0.71
0.83
0.77
6
Table 2: Classification results for validation stage.
Class
Acc.
Precision
Recall
F1-Score
Support
Non-Robbery
73.3
0.78
0.83
0.81
30
Robbery
58.8
0.67
0.59
0.62
17
Table 3: Classification results for the test experiment.
Dept. of Electrical Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, Thailand · Media Technology Program, King Mongkut’s Univ. of Technology Thonburi, Bangkok 10150, Thailand