physics.app-phJul 1, 2026

How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification

Authors: David Shulman

Abstract

Radio-frequency (RF) sensing is a central modality for counter-unmanned-aerial-system (counter-UAS) defence because it exploits the control, telemetry, and video links between a drone and its operator. Reported accuracies for RF-based drone detection and identification are often very high, but many are obtained using cross-validation that splits a small number of continuous recordings into short segments. This can place near-duplicate slices of the same recording in both training and test partitions, creating data leakage. We study this leakage problem through theory and measurement. We formalise the optimism of segment-level cross-validation and show, using Cover's function-counting theorem, that a classifier can exactly memorise the recording-to-label map when the number of independent recordings, R, is small relative to the feature dimension, d. In particular, this can occur when 2R is less than or approximately equal to d. Under these conditions, naive accuracy approaches 1, and the inflation gap approaches 1 - ACC*, where ACC* is the Bayes accuracy. The inflation eases only once R grows beyond this separability threshold. A controlled synthetic experiment with 10 seeds confirms the predicted curves: naive balanced accuracy rises from the Bayes level toward 1.0 as recording-specific nuisance variation grows, while honest recording-grouped evaluation declines to chance, with a gap reaching about 0.5. On the public DroneRF dataset, pooled leave-one-recording-out cross-validation shows drone type identification, AR versus Bebop, collapsing from a naive macro-F1 of 0.74 to 0.46, the two-class chance level. A leakage-pathway ablation attributes essentially all of the inflation to segment-level leakage.

Explore similar work

CardsList
Jun 28, 2026cs.SD

EchoHawk: A Reproducible Acoustic Pipeline for Drone Detection, Classification, and Direction-Finding, with a Cautionary Study of Session-Level Data Leakage

Passive acoustic sensing is an attractive modality for counter-unmanned aerial system (counter-UAS) defence: it is covert, low-cost, and effective against drones with small radar cross-sections or minimal radio emissions. We present EchoHawk, an open and fully reproducible reference pipeline that detects a drone from its rotor harmonics, estimates its blade-passing frequency, and localises it with a microphone array via classical wideband beamforming (delay-and-sum, MVDR, MUSIC) and time-delay processing (GCC-PHAT, SRP-PHAT), followed by temporal tracking. We evaluate the system on a physically transparent synthetic benchmark that pits drones against hard low-frequency harmonic confusers, such as ground vehicles, and on real recorded audio. Our central methodological contribution is a documented case of session-level data leakage in a widely used public dataset: because its recordings are pre-segmented into short clips, naive clip-level splits place adjacent slices of the same continuous recording in both training and test sets, inflating reported performance. Enforcing recording-session-grouped cross-validation reduces, for example, a random-forest baseline's detection probability at a 1% false-alarm rate from 0.796 to 0.745, yielding honest numbers. All code, figures, and a synthetic data generator are released so that every result runs without any download.
David Shulman
Unmanned Aerial Vehicle DetectionPassive Acoustic MonitoringDroneUnmanned Aerial VehiclesFrequencyData Leakage
AskPDF
Jun 10, 2026cs.CV

DroneShield-AI: A Multi-Modal Sensor Fusion Framework for Real-Time Autonomous Drone Threat Detection, Behavioral Intent Classification, and Swarm Intelligence in Contested Airspace

Unmanned Aerial Vehicle (UAV) threats have emerged as a defining security challenge of the 21st century. This paper presents DroneShield-AI, a unified open framework integrating six processing layers: RF signal classification, acoustic motor-signature detection, YOLOv8-based visual detection, evidence-weighted sensor fusion, a Behavioral Intent Classification Engine (BICE), and a Graph Neural Network Swarm Intelligence Module (GNN-SIM). BICE introduces the first systematic six-class threat taxonomy for drone flight patterns, enabling predictive operator alerts with a 30-second advance-warning horizon. GNN-SIM is the first open framework for adversarial multi-drone formation analysis using Graph Attention Networks. Evaluated on three publicly available real-world datasets, the fused pipeline achieves 96.1% detection accuracy, 3.2% false alarm rate, AUC-ROC: 0.981, and 142ms end-to-end latency on commodity CPU-class hardware at approximately 500−500-780 USD total system cost. All code, model weights, and simulation datasets are publicly released at submission.
Marius Bayizere
Unmanned Aerial Vehicle DetectionDroneMulti-Drone CoordinationMulti-Sensor FusionThreat ModelsAir Traffic Control+1
AskPDF
Jul 17, 2026cs.LG

Compact convolutional neural networks for AI-based drone detection system

The increasing use of first-person-view drones in modern conflicts has created a demand for compact and reliable detection systems capable of operating in complex electromagnetic environments. These drones continuously transmit video signals through onboard video transmitters, generating radio-frequency emissions that can be exploited for early detection. This study investigates the use of lightweight convolutional neural networks for automated detection of drone signals captured by a software-defined radio-based electronic warfare framework. Samples are converted into rasterized time-domain images, providing a computationally efficient input representation suitable for embedded systems. Several custom model architectures were designed and benchmarked in terms of accuracy, model size, and inference performance using a dataset containing approximately 40,000 labeled images. In addition to offline testing, the models were integrated into a GNU Radio signal processing chain for real-time evaluation. The results show that compact models can achieve high detection accuracy while maintaining low computational requirements, making them suitable for embedded radio-frequency monitoring applications. Compared with existing spectrogram-based RF detection methods, the proposed approach eliminates frequency-domain preprocessing and achieves comparable accuracy with significantly reduced computational cost.
Gábor Farkas, Gábor Fazekas, Karakai Patrik +2
Unmanned Aerial Vehicle DetectionConvolutional Neural NetworksTime-Frequency
AskPDF