SGLDet: Spectral-aware Global-Local Dynamic Modeling for Multispectral Aerial Object Detection

Peng Zhang; Shuaihao Han; Tingfa Xu; Jianan Li

cs.CVOct 20, 2026

SGLDet: Spectral-aware Global-Local Dynamic Modeling for Multispectral Aerial Object Detection

Authors: Peng Zhang, Shuaihao Han, Tingfa Xu, Jianan Li

Organizations: Beijing Institute of Technology, Beijing 100081, China · Beijing Institute of Technology, Beijing 100081, China, and also with Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401135, China

Abstract

Multispectral aerial object detection aims to exploit spectral-spatial cues to address object detection challenges in complex aerial scenes. Existing methods mainly focus on spectral-spatial decoupling and object-aware spectral-spatial fusion to improve spectral utilization. However, two critical issues remain underexplored: 1) severe foreground-background imbalance caused by large scene coverage hinders reliable object recognition; 2) aerial objects exhibit substantial local variations in scale, orientation, and structure, which hinder accurate localization. To address these issues, we propose SGLDet, a spectral-aware global-local dynamic modeling framework for multispectral aerial object detection, which is motivated by the observation that object semantics are jointly determined by global spectral consistency and scene context, while precise localization requires dynamic feature modulation conditioned on the input spectral and spatial properties. To realize these properties, SGLDet consists of three coordinated components: a Spectral-driven Global Dynamic Modeling module that aggregates spectrally related features across the scene via similarity-guided sparse matching; a Multi-scale Global Context Integration module that constructs a compact scene memory for scene-guided contextual reasoning; and a Spectral-aware Input-dependent Calibration Aggregation module that performs expert-kernel aggregation with scene-guided spectral-spatial calibration to enable adaptive local feature modulation. Extensive experiments on MODA, HOD3K, and DroneVehicle demonstrate that SGLDet achieves superior performance with competitive computational cost. Code is available at https://github.com/Riiluo/SGLDet.

Figures & tables

Fig. 1: (a) Global spectral relations associate objects or regions with similar spectral responses across spatial locations, while global scene context further helps disambiguate visually confusing regions. SGDM and MGCI perform these two operations, respectively. (b) Local dynamic modulation produces more object-aligned responses and facilitates more accurate localization for objects in complex backgrounds. SICA implements this input-adaptive refinement.

Fig. 2: Accuracy–complexity comparisons on MODA and HOD3K using mAP 50 (left) and mAP (right).

Fig. 3: Overall architecture of SGLDet. The spectral-guided encoder first applies SGDM to establish long-range spectral dependencies through sparse retrieval of spectrally similar responses. The scene-conditioned detection decoder then couples MGCI and SICA for hierarchical refinement. (a) SGDM performs sparse spectral retrieval followed by spectral consensus formation and contextual integration. (b) MGCI consolidates distributed cross-scale scene evidence into a compact scene memory and reads out adaptive scene context, while SICA performs scene-dependent adaptive local filtering before detection.

Fig. 4: Illustration of the proposed SICA module. SICA performs input-dependent local expert aggregation on $\boldsymbol{X}_{l}$ and scene-guided calibration from $\boldsymbol{C}_{l}$ to adaptively refine multilevel local features.

Method	car	bus	van	aw-bike	truck	tricycle	bike	pedestrian	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$	FLOPs $\downarrow$	Params $\downarrow$
two-stage methods
Roi Transformer [ 8 ]	90.5	89.3	75.2	73.3	68.7	51.8	44.5	30.0	65.4	43.4	40.7	244.7G	55.3M
O-RCNN [ 21 ]	90.5	89.7	74.6	72.5	66.6	54.7	45.1	30.4	65.5	44.0	40.9	244.7G	55.3M
LSKNet [ 13 ]	90.4	88.5	74.5	72.5	66.0	52.5	41.0	28.2	64.2	42.8	39.4	221.9G	32.6M
StripRCNN [ 14 ]	90.5	89.0	76.1	73.6	67.1	56.5	45.2	30.7	66.1	44.0	41.0	231.8G	45.2M
ReDet [ 22 ]	90.6	89.6	76.8	72.9	69.6	52.4	41.5	41.0	66.8	44.1	41.3	245.2G	31.6M

TABLE I: Comparison with other methods on MODA. † indicates methods originally designed for multispectral object detection.

Fig. 5: Visualization comparison of detection results and feature maps obtained by different methods on MODA.

Fig. 6: Two-dimensional t-SNE visualization of feature distributions learned by OSSDet and SGLDet on MODA.

Fig. 7: Effectiveness visualization of key components.

Subset	OSSDet			SGLDet (Ours)
Subset	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$
Full test	69.0	45.9	42.7	71.8	47.0	44.3
Low illum.	61.4	38.2	35.5	65.1	40.8	38.0
Small objects	56.2	32.5	30.8	59.0	34.8	33.0
High density	63.8	41.0	38.5	67.0	43.5	40.8

TABLE II: Challenging-subset comparison on MODA.

Method	mAP ${}_{50}\uparrow$	mAP $\uparrow$	FLOPs $\downarrow$	ms/img $\downarrow$	FPS $\uparrow$
O-RCNN [ 21 ]	65.5	40.9	244.7G	$51\!\pm\!0.7$	19.6
Rot. Retina [ 23 ]	53.4	34.1	233.5G	$42\!\pm\!0.6$	23.8
Ori. RepPoints [ 11 ]	66.5	40.9	213.6G	$56\!\pm\!0.8$	17.9
Ori-DETR [ 32 ]	67.8	41.9	492.0G	$76\!\pm\!1.5$	13.2
S2ADet [ 1 ]	63.5	38.9	406.0G	$112\!\pm\!1.8$	8.9
OSSDet [ 2 ]	69.0	42.7	263.1G	$53\!\pm\!0.8$	18.9

TABLE III: Measured inference efficiency and floating-point operation count on MODA.

SGDM	MGCI	SICA	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$
✗	✗	✗	66.5	44.1	40.9
✓	✗	✗	68.4	44.5	41.4
✗	✓	✗	68.0	44.6	41.5
✗	✗	✓	68.3	44.8	41.7
✓	✓	✗	70.2	45.8	42.6
✓	✓	✓	71.8	47.0	44.3

TABLE IV: Ablation studies on the key components of SGLDet.

$r$	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$
0.2	70.6	45.7	42.5
0.4	71.4	46.4	43.2
0.6	71.8	47.0	44.3
0.8	71.0	46.1	42.9

TABLE V: Ablation study of the Top-

K

ratio in SGDM.

Table 13

Table 14

Table 15

Detector	Backbone	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$
O-RCNN	R50	65.5	44.0	40.9
O-RCNN	R50 ${}_{\text{SGDM}}$	67.1 (+1.6)	45.2 (+1.2)	41.9 (+1.0)
Rot. FCOS	R50	63.5	41.8	39.1
Rot. FCOS	R50 ${}_{\text{SGDM}}$	64.9 (+1.4)	42.9 (+1.1)	40.0 (+0.9)
CFA	R50	66.2	43.2	40.6
CFA	R50 ${}_{\text{SGDM}}$	67.7 (+1.5)	44.3 (+1.1)	41.6 (+1.0)

TABLE XII: Compatibility analysis on representative R50-based detectors on MODA. Baseline results are taken from Table I .

Method	mAP ${}_{50}\uparrow$	mAP ${}_{75}\uparrow$	mAP $\uparrow$	FLOPs $\downarrow$	Params $\downarrow$
D-DETR [ 7 ]	85.0	52.0	48.8	109.6G	40.1M
CO-DETR [ 37 ]	77.9	57.3	51.7	200.7G	64.5M
RetinaNet [ 23 ]	91.2	59.4	53.3	205.7G	36.2M
YOLOv5 [ 38 ]	88.1	59.8	54.4	48.3G	20.9M
DINO [ 39 ]	89.1	61.6	56.3	114.7G	47.6M
Fovea [ 40 ]	92.2	62.2	56.8	123.9G	38.0M

TABLE XIII: Comparison with state-of-the-art detectors on HOD3K.

Fig. 8: Detection results on HOD3K. Red boxes indicate the ground truth, while green boxes denote the predictions.

Fig. 9: Detection results on DroneVehicle. Red boxes indicate the ground truth, while green boxes denote the predictions.

Method	Car	Bus	Truck	Fre.	Van	mAP ${}_{50}\uparrow$	mAP $\uparrow$
C 2 Former [ 52 ]	90.2	89.8	68.3	64.4	58.5	74.2	47.3
YOLOv8l-RGB [ 53 ]	92.5	91.7	68.8	47.8	50.2	70.2	48.6
YOLOv8l-IR [ 53 ]	93.4	91.9	69.3	53.7	51.1	71.9	49.1
SLBAF-Net [ 54 ]	90.2	89.9	76.0	68.2	59.9	76.8	49.5
CSOM-ODAF [ 55 ]	90.1	89.8	75.6	68.2	61.8	77.1	50.1
Multi. DINO [ 56 ]	89.5	88.8	75.4	54.3	54.3	72.5	50.3

TABLE XIV: Performance comparisons on DroneVehicle.

Multispectral Object Detection

Multispectral object detection combines information from visible, infrared, and other spectral bands to locate objects in challenging scenes.

See all 43 papers

S $^3$ E-Det: A Strict Spatial-Spectral Equivariant Detector for Multispectral Object Detection
Oct 20, 2026Peng Zhang, Tingfa Xu, Jinrui Sui +2Multispectral Object Detection Object Detection
Beijing Institute of Technology, Beijing 100081, China. · Big Data and Artificial Intelligence Laboratory, Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401135, China.
LiG-DETR: Local-in-Global Reassembly in Latent Space for Aerial Object Detection
Oct 7, 2026Yupeng Zhang, Fangzhuo Gao, Juntao Cheng +3Aerial Object Detection Remote Sensing Small Object Detection
Tianjin University. · Shanghai Jiaotong University. · Shanghai AI Laboratory. +1
FMC-DETR: Frequency-Decoupled Multi-Domain Coordination for Aerial-View Object Detection
Sep 27, 2025Ben Liang, Hongguang Wei, Yuan Liu +4Aerial Object Detection Remote Sensing Small Object Detection
School of Electronic Engineering and Optoelectronic Technology, Nanjing University of Science and Technology, Nanjing 210094, China · State Key Laboratory of Extreme Environment Optoelectronic Dynamic Measurement Technology and Instrument, Nanjing 210094, China

Oct 20, 2026J/K move · Enter open · S save

S $^3$ E-Det: A Strict Spatial-Spectral Equivariant Detector for Multispectral Object Detection

Peng Zhang, Tingfa Xu, Jinrui Sui, Yuyang Tian+1

Beijing Institute of Technology, Beijing 100081, China. · Big Data and Artificial Intelligence Laboratory, Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401135, China.

Multispectral Object Detection Object Detection Feature Fusion

Open paper AskPDF arXiv

Multispectral object detection requires detectors to exploit material-dependent spectral responses while maintaining consistent localization under geometric variations. Existing MOD detectors learn discriminative spectral representations, yet lack a unified mechanism that preserves spectral order, adapts spectral-context fusion across pyramid levels, and regulates geometric responses under rotation. To address these limitations, we propose S

^3

E-Det, a strict spatial-spectral equivariant detector that couples ordered spectral modeling with transformation-consistent prediction. Spectral Order-aware Sampling Calibration (SOSC) imposes local order constraints on input-derived spectral features through first-order variation, while Common-Differential Equivariant Fusion (CDEF) adapts spectral-context aggregation across pyramid levels. A compact Geometric Alignment Head (GAH) reconciles invariant and equivariant output properties within a unified prediction stage. Across MODA, HOD3K, and DroneVehicle, S

^3

E-Det combines high detection accuracy with a compact parameter footprint and reduced rotation-induced variation. Code is available at https://github.com/Riiluo/S3E-Det.

Abstract

Multispectral Object Detection

Explore similar work

S3^33E-Det: A Strict Spatial-Spectral Equivariant Detector for Multispectral Object Detection

LiG-DETR: Local-in-Global Reassembly in Latent Space for Aerial Object Detection

FMC-DETR: Frequency-Decoupled Multi-Domain Coordination for Aerial-View Object Detection

S $^3$ E-Det: A Strict Spatial-Spectral Equivariant Detector for Multispectral Object Detection