MIMONet: Multi-scale Input and Multi-scale Output Network for Salient Object Detection

Zhaojian Yao; Wei Gao; Tiesong Zhao; Hui Yuan; Sam Kwong

cs.CVAug 26, 2026

MIMONet: Multi-scale Input and Multi-scale Output Network for Salient Object Detection

Authors: Zhaojian Yao, Wei Gao, Tiesong Zhao, Hui Yuan, Sam Kwong

Organizations: School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, Shenzhen 518055, China · Peng Cheng Laboratory, Shenzhen 518066, China · Fujian Key Laboratory for Intelligent Processing and Wireless Transmission of Media Information, College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China · School of Control Science and Engineering, Shandong University, Jinan, China ·

Abstract

The existing methods for saliency detection task focus on the application of multi-level features, aiming to take advantage of the respective strengths of high- and low-level features. However, because the inputs of these models are single-size images, their multi-level features have difficulty in learning the knowledge of size variations of salient objects. Object-scale variation learning has great potential for detecting multi-scale objects, which has not been fully explored by existing methods. To improve the recognition ability of a model for objects with different sizes, we are inspired by the image pyramid to propose a Multi-scale Input and Multi-scale Output Network (MIMONet). In MIMONet, we extract multi-level features for three images with different resolutions to form three encoder branches, and information will be exchanged between the branches. The advantage of this approach is that the features of one branch can learn the knowledge of target size variation from the features of the other two branches. In addition, we design a Multi-scale Perception (MSP) module, in which the input feature layer is divided into several sub-layers with different resolutions. Capturing the multi-level structure information of the objects in these sub-layers can make the objects more fully perceived. For network training, we propose a Joint Saliency Loss (JSL), which can constrain multiple saliency maps output by the network to identify the same foreground objects, and induce their boundaries to be preserved clearly. Experimental results show that MIMONet has stronger detection capabilities and harvests better evaluation scores on multiple datasets compared to existing models. The code of our model will be released.

Figures & tables

Fig. 1: Categorisation of saliency detection models according to the number of inputs and outputs. (a) The original image is taken as input, and the detection network outputs its saliency map. (b) In the training stage, the original image is down-sampled and up-sampled, and multiple images with different resolutions are generated. These images are fed into the SISO model in turn and the network processes each image individually. Then, their corresponding saliency maps are output in turn. This framework is simply an improvement on the training strategy of the original SISO model. (c) The original image is resized to produce images of different resolutions. These images are fed into the detection network in parallel. The network simultaneously infers their saliency and outputs their saliency maps in parallel. During the inference process, multi-scale information collaborates with each other.

Panel of Figure 2 — Fig. 2: Saliency maps generated by different models. MIMONet is our proposed network. EDN and EGN belong to the modified and original versions of SISO respectively.

Fig. 3: Comparison of supervision fashions for saliency map. (a) Some SISO models impose additional constraints on specific feature layers during the training process, and output several saliency maps. There is no correlation between these saliency maps. (b) Our MIMONet outputs multiple saliency maps. Enhancing their correlation can further ensure that the same salient regions are predicted.

Fig. 4: The construction of MSP module. The feature layer is first separated into 3 sub-layers at a specific proportion. For the first sub-layer, it is down-sampled and its features are extracted using $3\times 3$ convolution. The second sub-layer is processed with $5\times 5$ convolution. After the third sub-layer is up-sampled, the larger $7\times 7$ convolution is adopted to collect its useful information. By aggregating the processing results of the three sub-layers, the saliency information is enhanced.

Panel of Figure 5 — Fig. 5: Feature map visualization. Obviously, salient regions receive more attention in $K$ .

Fig. 6: The saliency inference steps of MIMONet. First, an original image is down-sampled and up-sampled, with factors of 0.5 and 1.5, respectively, to obtain three images of different resolutions. These images are fed into an identical backbone (shared parameters), and then four feature layers will be generated. For each branch feature layer, it absorbs scale change information from the features of the other two branches. After the exchange of scale information, the multi-level features of each branch are fused and fed into an MSP module for feature enhancement. Finally, the optimized features are applied to detect salient regions. Thus, each branch outputs a saliency map.

Fig. 7: Multiple predictions of multi-scale inputs by MIMO.

Fig. 8: The process of generating common boundaries. Specifically, the low- and high-resolution saliency maps are upsampled and downsampled, respectively, and subsequently they are multiplied with the original resolution saliency map. In this way, a common saliency map is generated. We extract its structural information in parallel with different convolution kernels to obtain multiple foreground-background separation maps. By combining these results, a boundary map can be formed.

Panel of Figure 9 — Fig. 9: Separation effect between foreground and background. FBSM stands for foreground and background separation map. Numbers 1, 2, 3, and 4 indicate the results processed by different branches, corresponding to the branches of Conv 1 , Conv 3 , Conv 5 , and Conv 7 , respectively.

Method	#Params	Speed	ECSSD						PASCAL-S						HKU-IS
Method	(M)	(FPS)	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$
CPD 2019	47.85	45	0.939	0.917	0.898	0.918	0.949	0.037	0.866	0.826	0.800	0.848	0.887	0.071	0.925	0.891	0.875	0.906	0.950	0.034
BAS 2019	87.06	42	0.943	0.880	0.904	0.916	0.921	0.037	0.860	0.777	0.799	0.838	0.853	0.075	0.928	0.896	0.889	0.909	0.946	0.032
PoolNet 2019	68.26	38	0.944	0.915	0.896	0.921	0.945	0.039	0.869	0.821	0.799	0.849	0.876	0.074	0.932	0.899	0.881	0.916	0.954	0.033
BANet 2019	55.92	10	0.945	0.923	0.908	0.924	0.953	0.035	0.870	0.830	0.808	0.852	0.886	0.070	0.931	0.900	0.886	0.913	0.954	0.032
SCRN 2019	25.23	35	0.950	0.918	0.900	0.927	0.942	0.038	0.883	0.833	0.812	0.868	0.887	0.063	0.934	0.897	0.876	0.916	0.953	0.034

TABLE I: Quantitative comparisons on six datasets. The best two scores are highlighted in red and green , respectively. The subscript for each method indicates its year of publication. “*” means using a transformer backbone.

\uparrow

and

\downarrow

indicate higher and lower scores, respectively, for better performance

Panel of Figure 11 — Fig. 10: Visual comparisons of saliency maps generated by our proposed approach and other state-of-the-art approaches.

Setting	PASCAL-S						DUT-OMRON						SOC-test
Setting	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$
$N=1$
Low	0.867	0.811	0.805	0.849	0.888	0.065	0.797	0.736	0.725	0.821	0.866	0.060	0.751	0.663	0.651	0.759	0.802	0.098
Original	0.881	0.835	0.826	0.861	0.897	0.061	0.802	0.739	0.729	0.823	0.861	0.063	0.766	0.693	0.669	0.768	0.811	0.096
High	0.880	0.840	0.823	0.859	0.900	0.062	0.802	0.741	0.727	0.821	0.861	0.063	0.771	0.700	0.671	0.772	0.813	0.095
$N=2$

TABLE II: Performance comparison of network architectures.

N

represents the number of input images. “Original”indicates an original resolution image. “Original+Low” means that the network inputs contain the original resolution image and the low resolution image. “Original+High” is defined in the same way. Note that except “Original” and MISO, other architectures adopt the MIMO framework. The best scores are marked in bold

Panel of Figure 13 — Fig. 11: Detection results produced by different combinations of input size and quantity. “L.O.H” represents a combination of low-, original- and high-resolution images. “O.H” indicates original- and high-resolution images are input. “L.O” means the inputs of the model are low- and original-resolution images. “H”, “O” and “L” represent using only high, original, and low resolution images, respectively.

Setting	PASCAL-S						DUT-OMRON						SOC-test
Setting	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$
2:1:1	0.888	0.843	0.834	0.868	0.906	0.057	0.815	0.756	0.745	0.833	0.868	0.060	0.777	0.700	0.678	0.775	0.814	0.095
1:1:2	0.889	0.850	0.836	0.869	0.910	0.056	0.816	0.762	0.751	0.836	0.874	0.058	0.777	0.707	0.683	0.777	0.819	0.093
1:2:1	0.890	0.859	0.843	0.869	0.913	0.054	0.825	0.784	0.769	0.845	0.888	0.048	0.790	0.736	0.707	0.790	0.846	0.081
Conv	0.886	0.845	0.833	0.867	0.906	0.056	0.813	0.754	0.744	0.833	0.871	0.058	0.771	0.702	0.677	0.774	0.815	0.093
PPM	0.888	0.846	0.834	0.867	0.905	0.058	0.819	0.761	0.750	0.837	0.874	0.057	0.779	0.703	0.684	0.779	0.815	0.092

TABLE III: Evaluation results of MSP and other alternatives. “2:1:1”, “1:1:2” and “1:2:1” denote three different proportions for splitting the feature layer into low-, middle- and high-resolution sub-layers, respectively. The best scores are marked in bold

Panel of Figure 15 — Fig. 12: Saliency maps resulting from MIMONet equipped with different modules.

Setting	PASCAL-S						DUT-OMRON						SOC-test
Setting	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$	$F^{m}_{\beta}\uparrow$	$F^{a}_{\beta}\uparrow$	$F^{w}_{\beta}\uparrow$	$S_{m}\uparrow$	$E_{m}\uparrow$	$\mathcal{M}\downarrow$
w/o JSL	0.886	0.839	0.833	0.866	0.901	0.057	0.812	0.754	0.743	0.832	0.867	0.060	0.774	0.702	0.677	0.775	0.815	0.093
w/ JSL	0.890	0.859	0.843	0.869	0.913	0.054	0.825	0.784	0.769	0.845	0.888	0.048	0.790	0.736	0.707	0.790	0.846	0.081
$R=1$	0.886	0.841	0.832	0.866	0.903	0.058	0.815	0.755	0.745	0.834	0.868	0.060	0.777	0.703	0.684	0.779	0.815	0.092
$R=2$	0.886	0.844	0.832	0.867	0.905	0.057	0.816	0.759	0.750	0.836	0.871	0.058	0.779	0.705	0.682	0.777	0.817	0.093
$R=3$	0.887	0.851	0.838	0.867	0.910	0.055	0.820	0.774	0.763	0.843	0.880	0.052	0.785	0.728	0.702	0.787	0.837	0.084

TABLE IV: Performance analysis on JSL.

R

indicates the number of branches required to generate the common boundaries. The best scores are marked in bold

Panel of Figure 17 — Fig. 13: The boundary effects generated by different branch structures.

Salient Object Detection

Salient Object Detection finds visually prominent objects in images or video and predicts their pixel-level foreground masks.

See all 37 papers

U $^2$ Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection
Jun 18, 2026Junhui Li, Jialu Li, Youshan ZhangMamba U-Net
School of Electronics and Information Engineering, University of Science and Technology Liaoning, China · School of Artificial Intelligence, Chuzhou University, Anhui, China · MS in Artificial Intelligence, Yeshiva University, New York, USA
HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection
Jun 30, 2026Jiawei Xu, Qiangqiang Zhou, Zhouping Li +3Efficient Multimodal Inference Salient Object Detection
aSchool of Artificial Intelligence, Jiangxi Normal University, Street, Nanchang, 330000, State, China · bFaculty of Business Information, Shanghai Business School, Zhongshan West Road, Shanghai, 200235, China · cSchool of Computer Science and Information Engineering, Shanghai Institute of Technology, Street, Shanghai, 201418, China
G2HFNet: GeoGran-Aware Hierarchical Feature Fusion Network for Salient Object Detection in Optical Remote Sensing Images
Mar 13, 2026Bin Wan, Runmin Cong, Xiaofei Zhou +3Salient Object Detection Remote Sensing
the School of Control Science and Engineering, Shandong University, Jinan 250061, China, and also with State Key Laboratory of Autonomous Intelligent Unmanned Systems, Shanghai 201210, China · School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China · School of information engineering, Huzhou University, Huzhou 313000, China +1

Jun 18, 2026J/K move · Enter open · S save

U $^2$ Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection

Junhui Li, Jialu Li, Youshan Zhang

School of Electronics and Information Engineering, University of Science and Technology Liaoning, China · School of Artificial Intelligence, Chuzhou University, Anhui, China · MS in Artificial Intelligence, Yeshiva University, New York, USA

Mamba U-Net Salient Object Detection Multi-Scale Feature Fusion Object Detection

Open paper AskPDF arXiv

Mamba-based models have emerged as a promising alternative for salient object detection (SOD), offering significant advantages in modeling long sequences. However, existing models often fail to explore contextual information and the depth of the entire architecture. This paper introduces U

^2

Mamba, a powerful and innovative U-structured network for salient object detection. We propose multiscale Mamba U-blocks (MMUBs) that enhance the model depth to improve local feature extraction capabilities. Our newly developed nested U-structure, incorporating MMUBs, enables the network to integrate various receptive fields from shallow and deep layers, thereby collecting richer contextual information and longer-range data without being constrained by resolution. Instead of using the traditional deep supervision scheme and top-level supervised training, we propose a hierarchical training supervision method where the loss is computed at each level during the training process. Extensive experiments demonstrate that U

^2

Mamba achieves highly competitive performance against state-of-the-art methods. The source code is available at https://github.com/JL021/U2Mamba.

Abstract

Salient Object Detection

Explore similar work

U2^22Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection

HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection

G2HFNet: GeoGran-Aware Hierarchical Feature Fusion Network for Salient Object Detection in Optical Remote Sensing Images

U $^2$ Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection