Hybrid acoustic and pretrained embedding feature fusion for unmanned aerial vehicle detection using stacking ensemble learning under a leakage-aware evaluation protocol

Zahid Iqbal

cs.SDNov 3, 2026

Hybrid acoustic and pretrained embedding feature fusion for unmanned aerial vehicle detection using stacking ensemble learning under a leakage-aware evaluation protocol

Authors: Zahid Iqbal

Organizations: Department of Computer Science, Faculty of Computing and Information Technology, University of Gujrat, 50700, Punjab, Pakistan

Abstract

Small multirotor drones produce a characteristic acoustic signature, and microphones offer a low-cost way to detect them. High accuracies have been reported on public drone audio benchmarks, often with random clip-level splits of datasets built from a few long recordings and pieces of existing sound collections, so pieces of one source can appear in both training and test data. We study the binary task of the public DroneAudioDataset (11,704 one-second clips). Its 1,332 drone clips come from 257 five-second chunks of 7 recordings, and 96.4% of its non-drone clips are pieces of the 2,000 clips of the ESC-50 sound dataset. A grouped, source-disjoint protocol (five folds, three repeats) is compared with a random split. Each clip is described by 57 hand-crafted acoustic descriptors, reduced by a hybrid minimum-redundancy maximum-relevance and ReliefF selection or by NSGA-II multi-objective search, and by a 1024-dimensional embedding from the pretrained YAMNet network. Random Forest and XGBoost learners on both views form a stacking ensemble. Under the grouped protocol, the two-view stack reached a mean accuracy of 95.29% (SD 1.62 points over 15 test folds), a specificity of 98.29% and an area under the ROC curve of 0.990, and it correctly classified 98.78% of unseen ESC-50 source clips as non-drone. The random split overstated accuracy by 1.89 to 3.22 points for the four models. Adding YAMNet embeddings to the hand-crafted view raised accuracy by 1.11 points (higher in 12 of 15 folds). Hybrid selection matched the accuracy of all 44 baseline descriptors with 16 to 18 features, and NSGA-II retained 4 to 6 features at lower accuracy. The stack took 51.0 ms per clip on one CPU thread. Source-disjoint evaluation is needed for a fair benchmark on this dataset, and fusing hand-crafted descriptors with pretrained embeddings gives a small but consistent gain at modest computational cost.

Figures & tables

Figure 1: Overview of the detection pipeline. Feature selection, base learners and the meta-learner are fitted on the training folds only.

Subset	Clips	Leakage groups
Drone (positive)	1,332	257 five-second chunks from 7 recordings
Non-drone: ESC-50 pieces	10,001	2,000 ESC-50 source clips (50 classes)
Non-drone: Speech Commands background noise	70	5 noise recordings
Non-drone: silence	301	1 group
Total	11,704

Table 1: Composition of the binary DroneAudioDataset task (1-s clips, 16 kHz) and the groups used to keep each source on one side of every split.

Figure 2: Random piece-level splitting places pieces of one source on both sides of the split (left); the grouped protocol keeps every recording chunk and every ESC-50 source clip on one side (right).

	Accuracy (%)			ESC-50 specificity (%)
Model	Random	Grouped	$\Delta$	Random	Grouped
RF, hybrid mRMR+ReliefF	94.92 $\pm$ 0.75	91.71 $\pm$ 3.30	3.22	96.83 $\pm$ 0.46	95.68 $\pm$ 0.06
XGBoost, hybrid-57	96.87 $\pm$ 0.39	94.50 $\pm$ 2.24	2.37	99.53 $\pm$ 0.03	98.27 $\pm$ 0.15
Stacking: hand-crafted only	96.55 $\pm$ 0.54	94.18 $\pm$ 2.10	2.37	99.52 $\pm$ 0.15	98.45 $\pm$ 0.18
Stacking: hand-crafted + YAMNet (proposed)	97.18 $\pm$ 0.50	95.29 $\pm$ 1.62	1.89	99.68 $\pm$ 0.10	98.78 $\pm$ 0.10

Table 2: Effect of the evaluation protocol.

Figure 3: Accuracy and ESC-50 specificity under the random and grouped protocols (mean $\pm$ SD).

Model	Acc.	Prec.	Rec.	F1	Spec.	AUC	ESC-50 spec.
Logistic regression, hybrid	91.12 $\pm$ 2.13	90.25 $\pm$ 2.95	92.35 $\pm$ 3.48	91.23 $\pm$ 2.11	89.90 $\pm$ 3.61	0.965 $\pm$ 0.016	91.12 $\pm$ 0.29
1-D CNN, hybrid	88.96 $\pm$ 3.19	88.50 $\pm$ 5.43	90.13 $\pm$ 5.14	89.10 $\pm$ 2.98	87.79 $\pm$ 6.69	0.957 $\pm$ 0.025	89.88 $\pm$ 1.83
RF, hybrid mRMR+ReliefF	91.71 $\pm$ 3.30	94.57 $\pm$ 1.96	88.57 $\pm$ 7.05	91.31 $\pm$ 3.86	94.85 $\pm$ 2.07	0.981 $\pm$ 0.010	95.68 $\pm$ 0.06
XGBoost, hybrid	94.10 $\pm$ 1.78	94.80 $\pm$ 1.58	93.36 $\pm$ 3.84	94.03 $\pm$ 1.92	94.84 $\pm$ 1.68	0.985 $\pm$ 0.009	95.50 $\pm$ 0.10

Table 3: Classifiers on the hybrid mRMR+ReliefF feature subset (configuration B, 1:1 training ratio).

Model	Acc.	Prec.	Rec.	F1	Spec.	AUC	ESC-50 spec.
RF, all 44 baseline features	91.61 $\pm$ 2.47	94.82 $\pm$ 2.57	88.12 $\pm$ 5.06	91.24 $\pm$ 2.81	95.09 $\pm$ 2.63	0.981 $\pm$ 0.010	96.17 $\pm$ 0.39
RF, mRMR top-10	91.33 $\pm$ 2.54	92.39 $\pm$ 2.69	90.15 $\pm$ 4.55	91.19 $\pm$ 2.75	92.50 $\pm$ 2.92	0.973 $\pm$ 0.014	94.42 $\pm$ 0.46
RF, ReliefF top-10	90.69 $\pm$ 3.65	93.42 $\pm$ 2.10	87.60 $\pm$ 7.51	90.25 $\pm$ 4.22	93.78 $\pm$ 2.20	0.973 $\pm$ 0.014	94.88 $\pm$ 0.23
RF, hybrid mRMR+ReliefF	91.71 $\pm$ 3.30	94.57 $\pm$ 1.96	88.57 $\pm$ 7.05	91.31 $\pm$ 3.86	94.85 $\pm$ 2.07	0.981 $\pm$ 0.010	95.68 $\pm$ 0.06

Table 4: Feature-selection ablation with a tuned Random Forest (configuration B).

Figure 4: Number of grouped test folds (of 15) in which each descriptor entered the hybrid mRMR+ReliefF subset of configuration E.

Fold	Front size	Chosen $k$	Inner-CV acc.	Test acc.	Test AUC	Hybrid $k$	Hybrid test acc.
1	10	6	92.07	88.46	0.953	17	92.69
2	12	5	91.19	91.54	0.972	16	94.62
3	10	6	92.44	86.97	0.959	16	92.15
4	10	5	91.04	87.20	0.951	18	90.00
5	10	4	92.68	86.54	0.968	17	87.38

Table 5: NSGA-II feature selection on the 44 baseline features (population 40, 40 generations; the five grouped folds of the first repeat).

Figure 5: NSGA-II Pareto fronts (inner cross-validation accuracy against subset size) for the five grouped folds of the first repeat; circles mark the pre-specified operating points.

Model	Acc.	Prec.	Rec.	F1	Spec.	AUC	ESC-50 spec.
RF, hybrid-57	91.87 $\pm$ 3.05	97.61 $\pm$ 1.02	85.86 $\pm$ 6.31	91.24 $\pm$ 3.61	97.88 $\pm$ 0.95	0.987 $\pm$ 0.009	98.80 $\pm$ 0.13
XGBoost, hybrid-57	94.50 $\pm$ 2.24	97.51 $\pm$ 0.89	91.33 $\pm$ 4.46	94.26 $\pm$ 2.50	97.66 $\pm$ 0.87	0.988 $\pm$ 0.009	98.27 $\pm$ 0.15
RF, YAMNet	91.98 $\pm$ 1.46	96.73 $\pm$ 1.15	86.90 $\pm$ 2.77	91.53 $\pm$ 1.64	97.05 $\pm$ 1.06	0.978 $\pm$ 0.010	98.25 $\pm$ 0.09
XGBoost, YAMNet	92.38 $\pm$ 1.44	96.90 $\pm$ 0.99	87.57 $\pm$ 2.82	91.97 $\pm$ 1.61	97.19 $\pm$ 0.92	0.980 $\pm$ 0.010	98.58 $\pm$ 0.10
Stacking: hand-crafted only	94.18 $\pm$ 2.10	97.68 $\pm$ 0.95	90.52 $\pm$ 4.35	93.91 $\pm$ 2.39	97.84 $\pm$ 0.94	0.988 $\pm$ 0.008	98.45 $\pm$ 0.18
Stacking: YAMNet only	92.38 $\pm$ 1.60	97.19 $\pm$ 1.03	87.29 $\pm$ 3.05	91.95 $\pm$ 1.79	97.47 $\pm$ 0.94	0.979 $\pm$ 0.010	98.50 $\pm$ 0.22

Table 6: Fusion ablation (configuration E, 1:3 training ratio with class weights): single learners, single-view stacks and the two-view stack.

Figure 6: Fusion ablation in configuration E: accuracy and AUC per grouped test fold (dots) with mean $\pm$ SD (markers).

Figure 7: Logistic-regression meta-learner weights of the proposed stack (mean $\pm$ SD over 15 folds).

Alternative	$\Delta$ acc. (pp), mean $\pm$ SD	Folds won / tied / lost
Stacking: hand-crafted only	1.11 $\pm$ 1.08	12 / 3 / 0
XGBoost, hybrid-57	0.80 $\pm$ 1.08	11 / 1 / 3
Stacking: YAMNet only	2.91 $\pm$ 1.74	14 / 0 / 1
RF, hybrid mRMR+ReliefF	3.59 $\pm$ 3.53	13 / 0 / 2
XGBoost, hybrid	1.19 $\pm$ 1.55	12 / 0 / 3

Table 7: Per-fold paired accuracy differences between the proposed stack and each alternative over the 15 grouped test folds (computed from unrounded per-fold accuracies).

	Repeat 1			Repeat 2			Repeat 3
Alternative	b/c	$p$	Holm	b/c	$p$	Holm	b/c	$p$	Holm
Stacking: hand-crafted only	55 / 33	0.025	0.074	55 / 27	0.003	0.008	57 / 19	$<$ 0.001	$<$ 0.001
XGBoost, hybrid-57	53 / 40	0.213	0.213	52 / 28	0.010	0.012	48 / 25	0.010	0.010
Stacking: YAMNet only	98 / 23	$<$ 0.001	$<$ 0.001	114 / 32	$<$ 0.001	$<$ 0.001	108 / 34	$<$ 0.001	$<$ 0.001
RF, hybrid mRMR+ReliefF	151 / 45	$<$ 0.001	$<$ 0.001	143 / 47	$<$ 0.001	$<$ 0.001	145 / 39	$<$ 0.001	$<$ 0.001
XGBoost, hybrid	76 / 55	0.080	0.160	95 / 60	0.006	0.012	95 / 48	$<$ 0.001	$<$ 0.001

Table 8: Exact McNemar tests of the proposed stack against each alternative on identical test clips, with the five folds of each repeat pooled (2,664 clips per repeat). b: clips classified correctly only by the proposed stack; c: clips classified correctly only by the alternative;

p

: exact two-sided

p

-value; Holm:

p

-value after Holm correction over the five comparisons within the repeat.

Model	Repeat 1	Repeat 2	Repeat 3
RF, hybrid mRMR+ReliefF	91.25 [90.17, 92.34]	91.55 [90.43, 92.61]	91.63 [90.61, 92.72]
XGBoost, hybrid-57	94.74 [93.88, 95.61]	94.26 [93.36, 95.16]	94.74 [93.88, 95.61]
Stacking: hand-crafted only	94.41 [93.54, 95.27]	94.11 [93.21, 95.01]	94.18 [93.28, 95.08]
Stacking: hand-crafted + YAMNet (proposed)	95.23 [94.41, 96.02]	95.16 [94.33, 95.98]	95.61 [94.78, 96.40]

Table 9: Pooled test accuracy (%) of each repeat with its 95% bootstrap confidence interval (2,000 resamples; 2,664 clips per repeat, five folds pooled).

Model	Acc.	Prec.	Rec.	F1	Spec.	AUC	ESC-50 spec.
Stacking: hybrid-44 + YAMNet (1:1 data)	95.52 $\pm$ 1.06	95.83 $\pm$ 1.77	95.23 $\pm$ 2.10	95.50 $\pm$ 1.08	95.81 $\pm$ 1.83	0.988 $\pm$ 0.008	96.73 $\pm$ 0.34
Stacking: hybrid-57 + YAMNet (1:1 data)	95.97 $\pm$ 1.70	96.45 $\pm$ 1.77	95.48 $\pm$ 2.79	95.94 $\pm$ 1.75	96.46 $\pm$ 1.81	0.990 $\pm$ 0.009	96.98 $\pm$ 0.35
Stacking: hand-crafted only	94.18 $\pm$ 2.10	97.68 $\pm$ 0.95	90.52 $\pm$ 4.35	93.91 $\pm$ 2.39	97.84 $\pm$ 0.94	0.988 $\pm$ 0.008	98.45 $\pm$ 0.18
Stacking: hand-crafted + YAMNet (proposed)	95.29 $\pm$ 1.62	98.19 $\pm$ 0.92	92.30 $\pm$ 3.14	95.12 $\pm$ 1.76	98.29 $\pm$ 0.89	0.990 $\pm$ 0.009	98.78 $\pm$ 0.10

Table 10: Effect of the training class ratio on the two-view stack: configuration B (1:1, no class weights) versus configuration E (1:3, class-weighted).

Figure 8: Confusion matrices of the Random Forest baseline and the proposed stack, grouped protocol, first repeat (five folds pooled, 2,664 test clips).

Figure 9: ROC curves of representative models, grouped protocol, first repeat (five folds pooled).

ESC-50 class	Major group	False-positive rate (%), mean $\pm$ SD
washing machine	Interior/domestic	20.00 $\pm$ 2.50
rain	Natural soundscapes	9.17 $\pm$ 3.82
vacuum cleaner	Interior/domestic	5.00 $\pm$ 0.00
chainsaw	Exterior/urban	4.17 $\pm$ 1.44
train	Exterior/urban	4.17 $\pm$ 1.44
airplane	Exterior/urban	3.33 $\pm$ 1.44

Table 11: ESC-50 classes most often scored as drone by the proposed stack.

Major group	False-positive rate (%)
Animals	0.00
Exterior/urban	2.00
Human non-speech	0.00
Interior/domestic	2.58
Natural soundscapes	1.50

Table 12: ESC-50 false-positive rate of the proposed stack by major sound group (400 source clips per group, mean over 3 repeats).

Figure 10: ESC-50 classes most often scored as drone by the proposed stack (each clip scored by a model that never saw it; mean over 3 repeats).

Pipeline	Hand-crafted features	YAMNet	Features (ms)	Classifier (ms)	Total (ms)	Size (MB)	Accuracy (%)
RF, hybrid mRMR+ReliefF	17	no	14.43	7.06	21.49	1.68	91.71 $\pm$ 3.30
RF, NSGA-II subset	6	no	14.43	7.02	21.45	2.15	88.14 $\pm$ 2.03 a
XGBoost, hybrid-57	18	no	14.43	0.24	14.67	0.69	94.50 $\pm$ 2.24
Stacking: hand-crafted only	18	no	14.43	14.32	28.75	5.44	94.18 $\pm$ 2.10
Stacking: hand-crafted + YAMNet (proposed)	18	yes	22.14	28.86	51.00	12.60	95.29 $\pm$ 1.62

Table 13: Single-thread CPU cost per 1-s clip on an AMD EPYC 7513 32-Core Processor (median of 100 clips

\times

3 repeats).

Figure 11: Accuracy against total single-thread CPU time per clip. The NSGA-II point averages the five folds of the first repeat; the others average 15 folds.

UAV Detection

See all 28 papers

EchoHawk: A Reproducible Acoustic Pipeline for Drone Detection, Classification, and Direction-Finding, with a Cautionary Study of Session-Level Data Leakage
Jun 28, 2026David ShulmanData Leakage Direction-of-Arrival Estimation
DRONEAUDIONET: Noise Suppression for Drone Audition-based Search and Rescue
Aug 1, 2026Chitralekha Gupta, Soundarya Ramesh, Yifei Luo +1
School of Computing, National University of Singapore
Unsupervised Maneuver-Aware Acoustic Fault Detection for Autonomous Drones
Oct 7, 2026Ali M Ali, Ziyi Tang, Nurdaulet Nazarbay +1Fault Detection Unsupervised Anomaly Detection
Department of Mechanical and Aerospace Engineering, Carleton University, Ottawa, ON, K1S-5B6, Canada · Department of Electrical and Computer Engineering at the University of Alberta, Edmonton, AB, Canada · The Hong Kong Polytechnic University, Hong Kong

Jun 28, 2026J/K move · Enter open · S save

EchoHawk: A Reproducible Acoustic Pipeline for Drone Detection, Classification, and Direction-Finding, with a Cautionary Study of Session-Level Data Leakage

David Shulman

Data Leakage Direction-of-Arrival Estimation Sound Event Detection Passive Acoustic Monitoring UAV Detection

Open paper AskPDF arXiv

Passive acoustic sensing is an attractive modality for counter-unmanned aerial system (counter-UAS) defence: it is covert, low-cost, and effective against drones with small radar cross-sections or minimal radio emissions. We present EchoHawk, an open and fully reproducible reference pipeline that detects a drone from its rotor harmonics, estimates its blade-passing frequency, and localises it with a microphone array via classical wideband beamforming (delay-and-sum, MVDR, MUSIC) and time-delay processing (GCC-PHAT, SRP-PHAT), followed by temporal tracking. We evaluate the system on a physically transparent synthetic benchmark that pits drones against hard low-frequency harmonic confusers, such as ground vehicles, and on real recorded audio. Our central methodological contribution is a documented case of session-level data leakage in a widely used public dataset: because its recordings are pre-segmented into short clips, naive clip-level splits place adjacent slices of the same continuous recording in both training and test sets, inflating reported performance. Enforcing recording-session-grouped cross-validation reduces, for example, a random-forest baseline's detection probability at a 1% false-alarm rate from 0.796 to 0.745, yielding honest numbers. All code, figures, and a synthetic data generator are released so that every result runs without any download.

Abstract

UAV Detection

Explore similar work

EchoHawk: A Reproducible Acoustic Pipeline for Drone Detection, Classification, and Direction-Finding, with a Cautionary Study of Session-Level Data Leakage

DRONEAUDIONET: Noise Suppression for Drone Audition-based Search and Rescue

Unsupervised Maneuver-Aware Acoustic Fault Detection for Autonomous Drones