RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection
Authors: Bin Wan, Runmin Cong, Xiaofei Zhou, Hao Fang, Chengtao Lv, Sam Kwong
Organizations: School of Control Science and Engineering, Shandong University, Jinan 250061, China, and also with the Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan 250061, China · School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China · School of information engineering, Huzhou University, Huzhou 313000, China · School of Data Science, Lingnan University, Tuen Mun, Hong Kong
This paper focuses on the inconsistency in salient regions between RGB and thermal images. To address this issue, we propose the Region-guided Selective Optimization Network for RGB-T Salient Object Detection, which consists of the region guidance stage and saliency generation stage. In the region guidance stage, three parallel branches with same encoder-decoder structure equipped with the context interaction (CI) module and spatial-aware fusion (SF) module are designed to generate the guidance maps which are leveraged to calculate similarity scores. Then, in the saliency generation stage, the selective optimization (SO) module fuses RGB and thermal features based on the previously obtained similarity values to mitigate the impact of inconsistent distribution of salient targets between the two modalities. After that, to generate high-quality detection result, the dense detail enhancement (DDE) module which adopts the multiple dense connections and visual state space blocks is applied to low-level features for optimizing the detail information. In addition, the mutual interaction semantic (MIS) module is placed in the high-level features to dig the location cues by the mutual fusion strategy. We conduct extensive experiments on the RGB-T dataset, and the results demonstrate that the proposed RSONet achieves competitive performance against 27 state-of-the-art SOD methods.
Figures & tables
Fig. 1: Examples of RGB-T challenge scenarios.
Fig. 2: Illustration of the RSONet.
Fig. 3: Illustration of the guidance map.
Fig. 4: Architecture of the context interaction module.
Fig. 5: Architecture of the spatial-aware fusion module.
Fig. 6: Architecture of the selective optimization module.
Fig. 7: Architecture of the dense detail enhancement module.
Fig. 8: Architecture of the mutual interaction semantic module.
Methods
VT5000
VT1000
VT821
M↓
Fβ↑
Sα↑
Eξ↑
M↓
Fβ↑
Sα↑
Eξ↑
M↓
Fβ↑
Sα↑
Eξ↑
RGB-T
MIDD [ 59 ]
TIP 21
.046
.788
.856
.893
.029
.870
.907
.935
.045
.803
.871
.897
MMNet [ 60 ]
TCSVT 21
.044
.777
.863
.888
.027
.859
.914
.931
.040
.792
.874
.893
CSRNet [ 61 ]
TCSVT 21
.042
.809
.868
.907
.024
.875
.918
.939
.038
.829
.885
.912
ECFFNet [ 62 ]
TCSVT 21
.038
.803
.875
.911
.022
.873
.924
.947
.035
.807
.877
.907
APNet [ 63 ]
TETCI 21
.035
.816
.876
.917
.022
.880
.922
.950
.034
.814
.868
.911
TABLE I: Quantitative comparisons with 27 methods on three RGB-T datasets. The top three results in each column are marked with red , blue , green color, respectively. ↑ and ↓ denote that larger value is better and smaller value is better, respectively.
RGB-Thermal (RGB-T) salient object detection leverages complementary cues from visible and thermal modalities to improve robustness in challenging environments. However, in real-world scenarios, the reliability of each modality is inherently unstable: RGB images degrade under low illumination, motion blur, and noise, while thermal imagery often suffers from contrast compression and sensor artifacts. Such degradation introduces unreliable perceptual evidence that can mislead cross-modal fusion and significantly deteriorate detection performance. To address this challenge, we propose RA-SOD, a reliability-aware RGB-T salient object detection framework that explicitly models modality reliability and integrates it into feature learning and cross-modal fusion. First, we introduce a reliability-conditioned representation that adaptively compensates degraded modality features while preserving structural cues. Second, an uncertainty-guided dual-stream refinement strategy progressively corrects cross-modal representations while suppressing unreliable evidence. Finally, we propose a pixel-wise modality competition mechanism that dynamically selects modality cues according to spatial reliability for fine-grained fusion. Extensive experiments on four benchmarks (VT821, VT1000, VT5000, and VT-IMAG) demonstrate that RA-SOD achieves state-of-the-art performance and exhibits strong robustness under severe modality degradation. Code and models are available at https://github.com/zaoxienian/RA-SOD.
Hongbo Gao, Zhengyu Li, Xueru Nie +4
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, No. 92 West Dazhi Street, Nangang District, Harbin, Heilongjiang 150006, China · School of Computer Science, University of Sydney, Camperdown, Sydney, NSW 2006, Australia
RGB-thermal (RGB-T) object detection aims to fuse complementary information from visible and thermal modalities to achieve robust detection under varying illumination and weather conditions. Current methods typically employ attention mechanisms or transformers to perform cross-modal fusion independently at each feature scale, directly combining RGB and thermal features in the spatial domain. However, they still face significant limitations: cross-level knowledge inheritance caused by independent fusion at each scale,suppressing noise continuously due to the lack of bidirectional optimization, and information degradation induced by the absence of frequency-spatial collaboration. To address these issues, we propose DRPFNet, a Dual-domain Residual Progressive Fusion Network that constructs a unified information flow optimization system from three synergistic levels:structure, feature, and enhancement. At the structural level, we establish cross-scale propagation through bottom-up knowledge accumulation and bidirectional enhancement,ensuring smooth information flow. At the feature level, we collaboratively extract RGB high-frequency edges and thermal low-frequency structures via frequency band separation and edge guidance, guaranteeing representation quality. At the enhancement level, we enhance foreground-background discrimination through edge-guided dual-domain refinement,achieving precise object localization.Extensive experiments on two public RGB-T datasets demonstrate that our method achieves competitive performance with competitive efficiency, validating the effectiveness of this hierarchical collaborative strategy.
Zian Wang, Changchun Li
College of Computer Science and Technology, Jilin University, China
RGB-T detectors leverage the complementary strengths of visible and thermal infrared modalities, achieving robust performance under challenging conditions. Many of them resort to heavy dual backbones and exhaustive cross-modality fusion across the entire image, leading to impractically high computational costs. We observe that most image regions are smooth backgrounds (e.g., sky, ground) that can be easily handled by lightweight single-modality models. In light of this observation, we propose a sparse fusion mechanism for efficient RGB-T detection: first rapidly scanning the image to identify the proposals and then carefully examining the remaining sparse proposals via feature fusion. We propose a two-stage framework to instantiate this mechanism, which performs detection in two stages: 1) a lightweight and modality-specific detection stage that produces high-recall RoIs, and 2) a fusion-driven examination and refinement stage that filters out the false positives and refines the bounding boxes. This design enables the detector to adaptively allocate more computational resources to the potential foregrounds, improving the efficiency while ensuring detection accuracy. Extensive experiments show that our method achieves competitive performance with substantially fewer parameters and lower cost, while maintaining strong scalability to high-resolution images.
Chao Tian, Zikun Zhou, Chao Yang +2
Harbin Institute of Technology, Shenzhen, Guangdong, China