MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models

Puskal Khadka; KC Santosh

cs.CVMar 20, 2026

MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models

Authors: Puskal Khadka, KC Santosh

Organizations: USD Artifical Intelligence Research, Department of Computer Science, University of South Dakota, Vermillion, 57069, USA.

Abstract

State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at https://github.com/puskal-khadka/MFil-Mamba.

Figures & tables

Fig. 1: Top-1 Validation Accuracy versus Model Parameters comparison on Imagenet-1k [ 7 ] datasets. MFil-Mamba demonstrates superior performance compared to baseline state-of-the-art models with similar parameter counts.

Fig. 2: (Top) Overview of the MFil-Mamba. (Bottom Left) Illustration of Single MFil-Mamba Block. (Bottom Middle) Illustration of MFil-SSM block with filter-based scanning across four input representations. Each representation is independently filtered and then its patches are concatenated and passed through a selective state space block. (Bottom Right) Illustration of State space model.

Layer	Output Size	MFil-Mamba-T	MFil-Mamba-S	MFil-Mamba-B
Stem	56 $\times$ 56	dim 94	dim 94	dim 128
Stage 1	28 $\times$ 28	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 94}\end{bmatrix}$ $\times$ 1	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 94}\end{bmatrix}$ $\times$ 2	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 128}\end{bmatrix}$ $\times$ 2
Stage 1	28 $\times$ 28	Downsample, conv 2 $\times$ 2, stride 2, dim 188	Downsample, conv 2 $\times$ 2, stride 2, dim 188	Downsample, conv 2 $\times$ 2, stride 2, dim 256
Stage 2	14 $\times$ 14	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 188}\end{bmatrix}$ $\times$ 3	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 188}\end{bmatrix}$ $\times$ 2	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 256}\end{bmatrix}$ $\times$ 2
Stage 2	14 $\times$ 14	Downsample, conv 2 $\times$ 2, stride 2, dim 376	Downsample, conv 2 $\times$ 2, stride 2, dim 376	Downsample, conv 2 $\times$ 2, stride 2, dim 512
Stage 3	7 $\times$ 7	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 376}\end{bmatrix}$ $\times$ 8	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 376}\end{bmatrix}$ $\times$ 18	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 512}\end{bmatrix}$ $\times$ 18

TABLE I: Detailed architecture overview of the MFil-Mamba variants.

Hyperparameter	Tiny	Small	Base
Input Size	224
Epochs	300
Batch Size	1024
Optimizer	AdamW
Weight Decay	0.05
Learning Rate Decay	Cosine

TABLE II: Hyperparameters of MFil-Mamba models for ImageNet-1k classification task.

Method	Venue	Img Size	#Param	FLOPs	Top-1 Acc(%)
RegNetY-4G [ 25 ]	cvpr’20	224 2	21M	4.0G	80.0
RegNetY-8G [ 25 ]		224 2	39M	8.0G	81.7
RegNetY-16G [ 25 ]		224 2	84M	16.0G	82.9
ConvNeXt-T [ 26 ]	cvpr’22	224 2	29M	4.5G	82.1
ConvNeXt-S [ 26 ]		224 2	50M	8.7G	83.1
ConvNeXt-B [ 26 ]		224 2	89M	15.4G	83.8

TABLE III: Model performances on ImageNet-1K Classification . FLOPs are calculated with an input resolution of 224

\times

224.

Backbone	#Param	FLOPs	Object Detection			Instance Segmentation
Backbone	#Param	FLOPs	AP box	AP ${}_{50}^{\rm box}$	AP ${}_{75}^{\rm box}$	AP mask	AP ${}_{50}^{\rm mask}$	AP ${}_{75}^{\rm mask}$
Swin-T [ 6 ]	48M	267G	42.7	65.2	46.8	39.3	62.2	42.2
ConvNeXt-T [ 26 ]	48M	262G	44.2	66.6	48.3	40.1	63.3	42.8
PVT-S [ 39 ]	44M	245G	40.4	62.9	43.8	37.8	60.1	40.3
DAT-T [ 41 ]	48M	272G	44.4	67.6	48.5	40.4	64.2	43.1
ViT-Adapter-S [ 40 ]	48M	403G	44.7	65.8	48.3	39.9	62.5	42.8

TABLE IV: Object detection and instance segmentation performance on MS COCO dataset [ 62 ] using the Mask RCNN framework [ 30 ] with 1

\times

schedule. FLOPs are calculated with an input resolution of 1280

\times

800. AP b and AP m represents mean average precision (AP) of bounding box and mask, respectively.

Fig. 3: Qualitative visualizations of MFil-Mamba-T for object detection (1st row) and instance segmentation tasks (2nd row) on the MS COCO dataset [ 62 ] , and for semantic segmentation task (3rd row) on the ADE20K dataset [ 63 ] . Detailed visualization is presented in Sec IV-G .

Backbone	#Param	FLOPs	mIoU (SS)	mIoU (MS)
ResNet-50 [ 3 ]	67M	953G	42.1	42.8
Swin-T [ 6 ]	60M	945G	44.4	45.8
Focal-T [ 81 ]	62M	998G	45.8	47.0
DAT-T [ 41 ]	60M	957G	45.5	46.4
ConvNeXT-T [ 26 ]	60M	939G	46.0	46.7
NAT-T [ 82 ]	58M	934G	47.1	48.4

TABLE V: Semantic segmentation performance of different backbones on ADE20K [ 63 ] using UPerNet [ 80 ] . FLOPs are computed for an input resolution of 512

\times

2048. SS and MS denote single-scale and multi-scale evaluation, respectively.

Fig. 4: Comparison of MFil-Mamba’s effective receptive field with existing state-of-the-art baselines.

Fig. 5: Grad-CAM visualization on the ImageNet-1k validation set, illustrating model’s ability to focus on the main subject of the image.

Fig. 6: Object detection and instance segmentation visualizations on the MS COCO validation set. In each group of three successive images, the left image shows the original input, the middle shows object detection results with bounding boxes, and the right shows instance segmentation results with masks and bounding boxes.

Fig. 7: Semantic segmentation visualizations on the ADE20k dataset.

Strategy	#Params	FLOPs	Top-1 Acc.
Single Scan (Flatten)	30.93M	4.94G	82.79
Cross Scan (4-Direction)	30.93M	5.14G	82.95
Original + Single Filter	33.36M	5.41G	83.06
Multi Filter Scan	33.46M	5.56G	83.22

TABLE VI: Ablation of scanning modules in the Tiny variant.

Merging	#Param	FLOPs	Top-1 Acc.
Without Adpt. Weighting	33.46M	5.56G	83.10
With Adpt. Weighting	33.46M	5.56G	83.22

TABLE VII: Ablation study of adaptive weighting implementation.

$\mathbf{d\_state}$	$\mathbf{ssm\_ratio}$	#Params	FLOPs	Top-1 Acc.
1	1.0	33.46M	5.56G	83.22
1	2.0	48.27M	8.21G	83.08
2	2.0	48.31M	8.34G	83.10

TABLE VIII: Ablation study on (different) model configurations.

Figure 16

Figure 17

Explore similar work

SF-Mamba: Rethinking State Space Model for Vision
Mar 17, 2026Masakazu Yoshimura, Teruaki Hayashi, Yuki Hoshino +2Visual SSMs
Sony Group Corporation, Tokyo, Japan.
Deformba: Vision State Space Model with Adaptive State Fusion
May 20, 2026Hongyu Ke, Jack Morris, Yongkang Liu +4Visual SSMs State Space Models
Department of Computer Science, Georgia State University · University of Tennessee Knoxville · InfoTech Labs, Toyota Motor North America R&D.
Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics
Jul 3, 2026Anvitha Ramachandran, Dhruv Parikh, Haoyang Fan +2Visual SSMs State Space Models
University of Southern California · DEVCOM Army Research Office

Mar 17, 2026J/K move · Enter open · S save

SF-Mamba: Rethinking State Space Model for Vision

Masakazu Yoshimura, Teruaki Hayashi, Yuki Hoshino, Wei-Yao Wang+1

Sony Group Corporation, Tokyo, Japan.

Visual SSMs

Open paper AskPDF arXiv

The realm of Mamba for vision has been advanced in recent years to strike for the alternatives of Vision Transformers (ViTs) that suffer from the quadratic complexity. While the recurrent scanning mechanism of Mamba offers computational efficiency, it inherently limits non-causal interactions between image patches. Prior works have attempted to address this limitation through various multi-scan strategies; however, these approaches suffer from inefficiencies due to suboptimal scan designs and frequent data rearrangement. Moreover, Mamba exhibits relatively slow computational speed under short token lengths, commonly used in visual tasks. In pursuit of a truly efficient vision encoder, we rethink the scan operation for vision and the computational efficiency of Mamba. To this end, we propose SF-Mamba, a novel visual Mamba with two key proposals: auxiliary patch swapping for encoding bidirectional information flow under an unidirectional scan and batch folding with periodic state reset for advanced GPU parallelism. Extensive experiments on image classification, object detection, and instance and semantic segmentation consistently demonstrate that our proposed SF-Mamba significantly outperforms state-of-the-art baselines while improving throughput across different model sizes. We will release the source code after publication.

Abstract

Explore similar work

SF-Mamba: Rethinking State Space Model for Vision

Deformba: Vision State Space Model with Adaptive State Fusion

Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics