CSCWD: Cross-Scale Channel-wise Knowledge Distillation for Lightweight Tiny Object Detection on Edge Devices

Amir Zamani; Zeinab Ghasemi-Naraghi

cs.CVSep 24, 2026

CSCWD: Cross-Scale Channel-wise Knowledge Distillation for Lightweight Tiny Object Detection on Edge Devices

Authors: Amir Zamani, Zeinab Ghasemi-Naraghi

Organizations: Department of Computer Engineering, Islamic Revolution Comprehensive University, Tehran, Iran · Department of Computer Engineering, K.N. Toosi University of Technology, Tehran, Iran

Abstract

Real-time tiny object detection in aerial imagery is constrained by the weak spatial evidence of very small objects and the loss of high-resolution detail in lightweight detectors. This study presents Cross-Scale Channel-wise Knowledge Distillation (CSCWD), a training-time framework that transfers high-resolution spatial representations from a YOLO11m-P2 teacher to a compact YOLO11n student without altering the student's inference architecture. Unlike conventional same-scale feature distillation, CSCWD transfers supervision from teacher P2 to student P3 after feature alignment while retaining same-scale distillation at deeper pyramid levels. Under the unified seven-sequence Drone-vs-Bird validation protocol, YOLO11n-CSCWD achieves 50.17% mean average precision at an intersection-over-union threshold of 0.5 ([email protected]) and 59.73% recall, improving the matched CA-YOLO11n baseline by 2.92 percentage points in [email protected] and 3.55 points in recall. Cross-scale alignment further increases [email protected] by 2.09 points over the corresponding same-scale channel-wise distillation configuration. In zero-shot evaluation on DUT-Anti-UAV, [email protected] increases from 48.29% to 50.06% without target-domain fine-tuning. This domain was included because its challenging small targets make low-latency, computationally efficient detection particularly relevant. On Raspberry Pi 5 using NCNN-FP16 at 640x640 resolution, the 2.58-million-parameter student achieves 50.32% [email protected] at 82.32 ms mean wall-clock latency, or 12.15 frames per second, while retaining essentially the same runtime and memory requirements as the matched baseline. The results support cross-scale distillation for improving tiny-target detection without increasing inference-time model complexity.

Figures & tables

Figure 1: Deployed accuracy and inference speed of lightweight detectors on Raspberry Pi 5 using NCNN-FP16 at

640\times 640

under the unified seven-sequence validation protocol

Figure 2: Overview of CSCWD training and inference. The frozen YOLO11m-P2 teacher provides cross-scale $P_{2}$ -to- $P_{3}$ supervision and matched-scale distillation at $P_{4}$ and $P_{5}$ during training. Inference retains only the distilled YOLO11n-CSCWD model. Raspberry Pi image: CC BY 2.0 SparkFun Electronics (2023)

Setting	Stage 1: VisDrone adaptation	Stage 2: Target fine-tuning
Initialization	COCO ( yolo11m.pt )	Stage-1 best
Epochs / batch	30 / 4	100 / 12
Optimizer	Default	SGD
Mosaic ( $p$ /close)	1.0 / last 5	0.75 / last 10
MixUp	0.0	0.15
Scale / translate	0.5 / 0.1	0.5 / 0.0

Table 1: Teacher training settings for VisDrone adaptation and target fine-tuning.

Method	Architecture	Split type	Params (M)	GFLOPs	mAP @0.5	mAP @0.5:0.95
Wong et al. Wong et al. (2025)	WideResNet-based	Frame-based	100.70	260.0	75.70 b	–
Sawada et al. Sawada et al. (2025)	Faster R-CNN + RFLA	Video-based	23.50	130.0	64.18 a	–
Laroca et al. Laroca et al. (2025)	YOLO11m (whole image)	Video-based	20.10	68.0	55.23 c	–
YOLOv8n Jocher et al. (2023)	YOLOv8n	Video-based	3.20	8.7	45.74	15.90
YOLOv10n Wang et al. (2024)	YOLOv10n	Video-based	2.30	6.7	44.49	15.42
YOLO11s Jocher and Qiu (2024)	YOLO11s	Video-based	9.40	21.6	46.17	16.37

Table 2: Comparison on Drone-vs-Bird. The first three rows report literature values obtained under different data splits or inference procedures; the remaining models are evaluated under the unified seven-sequence validation protocol.

Sequence	CA- YOLO11n	YOLO11n- CSCWD	YOLO11m- P2
Mavic	91.28	91.41	93.07
Inspire	31.86	35.99	36.56
Disco	79.33	79.48	84.38
GOPR5843	26.80	28.63	37.73
Swarm	48.98	59.32	56.65
Hillside	23.61	32.84	54.96

Table 3: Sequence-level [email protected] under the unified seven-sequence validation protocol.

Model	Precision	Recall	mAP @0.5	mAP @0.5:0.95
CA-YOLO11n	70.61	46.59	48.29	24.04
YOLO11n-CSCWD	74.59	48.20	50.06	24.00
YOLO11m-P2	83.33	56.34	62.57	30.43

Table 4: Zero-shot DUT-Anti-UAV evaluation without target-domain fine-tuning.

Panel of Figure 3 — Figure 3: Qualitative Drone-vs-Bird examples comparing ground truth, YOLO11m-P2, CA-YOLO11n, and YOLO11n-CSCWD. The selected cases illustrate successful tiny-target detection under challenging visual conditions

Model	Precision	Recall	mAP @0.5	mAP @0.5:0.95
CA-YOLO11n Zamani and Abedini (2026)	61.72	56.18	47.25	16.92
YOLO11n-MGD Yang et al. (2022b)	63.96	58.71	50.53	18.00
YOLO11n-Hybrid	63.68	57.16	49.70	17.36
YOLO11n-CSCWD	63.42	59.73	50.17	17.30

Table 5: Ablation of feature-distillation objectives under the unified seven-sequence validation protocol.

Configuration	Precision	Recall	[email protected]	[email protected]:0.95
YOLO11m	67.66	60.52	52.84	19.17
YOLO11m-P2	69.43	62.73	56.75	21.26
YOLO11n-CWD (YOLO11m teacher)	56.21	52.45	44.08	15.58
YOLO11n-CWD (YOLO11m-P2, same-scale)	64.84	55.36	48.08	16.64
YOLO11n-CSCWD	63.42	59.73	50.17	17.30

Table 6: Effect of teacher design and feature pairing under the unified seven-sequence validation protocol.

Model	Input size	mAP @0.5	mAP @0.5:0.95	Mean wall latency (ms)	P95 latency (ms)	FPS	Peak RSS (MB)
CA-YOLO11n	640	47.81	16.92	82.29	109.48	12.15	561.23
	960	47.99	17.54	183.64	213.21	5.45	628.02
	1280	39.17	13.61	–	–	–	–
YOLO11n-CSCWD	640	50.32	17.36	82.32	109.97	12.15	559.81
	960	52.85	19.38	183.44	212.05	5.45	624.31
	1280	48.53	17.76	–	–	–	–

Table 7: Raspberry Pi 5 deployment results for NCNN-FP16 artifacts. Accuracy follows the unified seven-sequence validation protocol. Runtime statistics at 640 and 960 are averaged over three complete validation runs; 1280 reports accuracy only.

Panel of Figure 4 — Figure 4: Target-centered feature responses on a representative Hillside frame. YOLO11n-CSCWD shows a more concentrated $P_{3}$ response than CA-YOLO11n; YOLO11m-P2 provides a higher-resolution comparison

Figure 5: Raspberry Pi 5 CPU temperature during 10 min of sustained NCNN-FP16 inference at

640\times 640

. YOLO11n-CSCWD remains cooler than YOLO11m-P2, with no observed throttling

Explore similar work

TinyFormer: Preserving Tiny Objects in YOLO-DETR Hybrid Real-time Detectors
May 24, 2026Jun-Wei Hsieh, Meng-Yu Kao, Ghufron Wahyu Kurniawan +1Hybrid CNN-Transformer Architectures Small Object Detection
College of Artificial Intelligence, National Yang Ming Chiao Tung University, Taiwan · 2Mitsubishi Electric Research Laboratories
Small Object Detection in Drone Aerial Imagery with LAF-YOLOv10
Sep 13, 2026Quratulain Nayeem, Fahmina Taranum, Mohammed Mudassir UddinObject Detection Small Object Detection
FD-CanKD: Frequency-Decoupled Cross-Attention Distillation as a Refinement Prior for Compact Object Detectors
Aug 19, 2026YoungJae Cheong, Jhonghyun AnKnowledge Distillation Efficient Object Detection
Department of AI and Software, Gachon University, Seongnam, Republic of Korea

May 24, 2026, cs.CVJ/K move · Enter open · S save

TinyFormer: Preserving Tiny Objects in YOLO-DETR Hybrid Real-time Detectors

Jun-Wei Hsieh, Meng-Yu Kao, Ghufron Wahyu Kurniawan, Kuan-Chuan Peng

College of Artificial Intelligence, National Yang Ming Chiao Tung University, Taiwan · 2Mitsubishi Electric Research Laboratories

Hybrid CNN-Transformer Architectures Small Object Detection Efficient Object Detection Detection Transformer Real-Time Object Detection

Open paper AskPDF arXiv

YOLO-series and DETR-based detectors struggle with tiny-object detection. YOLO-style models benefit from efficient dense prediction, but their large-stride backbones may suppress tiny instances in deep feature maps and make grid assignment ambiguous. DETR-based models remove hand-crafted post-processing through set prediction, yet they reason over coarse token grids, where tiny objects occupy only a few weak tokens and are easily overlooked during matching. To address these limitations, we propose TinyFormer, a unified YOLO--DETR hybrid real-time detector that combines ViT representations, NMS-free set prediction, and a YOLO-style pyramid neck for accurate small-object detection. TinyFormer introduces a Parallel Bi-fusion Module (PBM), which builds high-resolution shortcuts from shallow stages to the feature pyramid, preserving fine spatial details during multi-scale fusion. We further design a Spatial Semantic Adapter (SSA) to compensate for the spatial loss caused by coarse tokenization. SSA extracts high-resolution cues from early stages and injects them into transformer token embeddings, improving tiny-object localization without sacrificing the global modeling ability of DETR. Experiments on MS COCO show that TinyFormer consistently outperforms recent YOLO-series detectors and the strong DEIMv2 baseline. TinyFormer-X achieves 58.4% AP even without PBM, while adding PBM improves the overall AP to 58.5% and brings a 1.6% AP gain on small objects. With Objects365 pre-training, TinyFormer-X-PBM reaches 60.2% AP, surpassing RF-DETR and other Objects365-pretrained detectors with fewer parameters and lower computation. These results demonstrate that TinyFormer bridges dense YOLO-style feature fusion and DETR-style set prediction, providing a strong accuracy-efficiency trade-off for real-time tiny-object detection. Code is available at https://github.com/mmpmmpmmpjosh/TinyFormer.

Abstract

Explore similar work

TinyFormer: Preserving Tiny Objects in YOLO-DETR Hybrid Real-time Detectors

Small Object Detection in Drone Aerial Imagery with LAF-YOLOv10

FD-CanKD: Frequency-Decoupled Cross-Attention Distillation as a Refinement Prior for Compact Object Detectors