MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models

Puskal Khadka; KC Santosh

cs.CVMar 20, 2026

MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models

Authors: Puskal Khadka, KC Santosh

Organizations: USD Artifical Intelligence Research, Department of Computer Science, University of South Dakota, Vermillion, 57069, USA.

Abstract

State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at https://github.com/puskal-khadka/MFil-Mamba.

Figures & tables

Fig. 1: Top-1 Validation Accuracy versus Model Parameters comparison on Imagenet-1k [ 7 ] datasets. MFil-Mamba demonstrates superior performance compared to baseline state-of-the-art models with similar parameter counts.

Fig. 2: (Top) Overview of the MFil-Mamba. (Bottom Left) Illustration of Single MFil-Mamba Block. (Bottom Middle) Illustration of MFil-SSM block with filter-based scanning across four input representations. Each representation is independently filtered and then its patches are concatenated and passed through a selective state space block. (Bottom Right) Illustration of State space model.

Layer	Output Size	MFil-Mamba-T	MFil-Mamba-S	MFil-Mamba-B
Stem	56 $\times$ 56	dim 94	dim 94	dim 128
Stage 1	28 $\times$ 28	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 94}\end{bmatrix}$ $\times$ 1	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 94}\end{bmatrix}$ $\times$ 2	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 128}\end{bmatrix}$ $\times$ 2
Stage 1	28 $\times$ 28	Downsample, conv 2 $\times$ 2, stride 2, dim 188	Downsample, conv 2 $\times$ 2, stride 2, dim 188	Downsample, conv 2 $\times$ 2, stride 2, dim 256
Stage 2	14 $\times$ 14	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 188}\end{bmatrix}$ $\times$ 3	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 188}\end{bmatrix}$ $\times$ 2	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 256}\end{bmatrix}$ $\times$ 2
Stage 2	14 $\times$ 14	Downsample, conv 2 $\times$ 2, stride 2, dim 376	Downsample, conv 2 $\times$ 2, stride 2, dim 376	Downsample, conv 2 $\times$ 2, stride 2, dim 512
Stage 3	7 $\times$ 7	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 376}\end{bmatrix}$ $\times$ 8	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 376}\end{bmatrix}$ $\times$ 18	$\begin{bmatrix}\text{MFil-Mamba Block,}\\ \text{dim 512}\end{bmatrix}$ $\times$ 18

TABLE I: Detailed architecture overview of the MFil-Mamba variants.

Hyperparameter	Tiny	Small	Base
Input Size	224
Epochs	300
Batch Size	1024
Optimizer	AdamW
Weight Decay	0.05
Learning Rate Decay	Cosine

TABLE II: Hyperparameters of MFil-Mamba models for ImageNet-1k classification task.

Method	Venue	Img Size	#Param	FLOPs	Top-1 Acc(%)
RegNetY-4G [ 25 ]	cvpr’20	224 2	21M	4.0G	80.0
RegNetY-8G [ 25 ]		224 2	39M	8.0G	81.7
RegNetY-16G [ 25 ]		224 2	84M	16.0G	82.9
ConvNeXt-T [ 26 ]	cvpr’22	224 2	29M	4.5G	82.1
ConvNeXt-S [ 26 ]		224 2	50M	8.7G	83.1
ConvNeXt-B [ 26 ]		224 2	89M	15.4G	83.8

TABLE III: Model performances on ImageNet-1K Classification . FLOPs are calculated with an input resolution of 224

\times

224.

Backbone	#Param	FLOPs	Object Detection			Instance Segmentation
Backbone	#Param	FLOPs	AP box	AP ${}_{50}^{\rm box}$	AP ${}_{75}^{\rm box}$	AP mask	AP ${}_{50}^{\rm mask}$	AP ${}_{75}^{\rm mask}$
Swin-T [ 6 ]	48M	267G	42.7	65.2	46.8	39.3	62.2	42.2
ConvNeXt-T [ 26 ]	48M	262G	44.2	66.6	48.3	40.1	63.3	42.8
PVT-S [ 39 ]	44M	245G	40.4	62.9	43.8	37.8	60.1	40.3
DAT-T [ 41 ]	48M	272G	44.4	67.6	48.5	40.4	64.2	43.1
ViT-Adapter-S [ 40 ]	48M	403G	44.7	65.8	48.3	39.9	62.5	42.8

TABLE IV: Object detection and instance segmentation performance on MS COCO dataset [ 62 ] using the Mask RCNN framework [ 30 ] with 1

\times

schedule. FLOPs are calculated with an input resolution of 1280

\times

800. AP b and AP m represents mean average precision (AP) of bounding box and mask, respectively.

Fig. 3: Qualitative visualizations of MFil-Mamba-T for object detection (1st row) and instance segmentation tasks (2nd row) on the MS COCO dataset [ 62 ] , and for semantic segmentation task (3rd row) on the ADE20K dataset [ 63 ] . Detailed visualization is presented in Sec IV-G .

Backbone	#Param	FLOPs	mIoU (SS)	mIoU (MS)
ResNet-50 [ 3 ]	67M	953G	42.1	42.8
Swin-T [ 6 ]	60M	945G	44.4	45.8
Focal-T [ 81 ]	62M	998G	45.8	47.0
DAT-T [ 41 ]	60M	957G	45.5	46.4
ConvNeXT-T [ 26 ]	60M	939G	46.0	46.7
NAT-T [ 82 ]	58M	934G	47.1	48.4

TABLE V: Semantic segmentation performance of different backbones on ADE20K [ 63 ] using UPerNet [ 80 ] . FLOPs are computed for an input resolution of 512

\times

2048. SS and MS denote single-scale and multi-scale evaluation, respectively.

Fig. 4: Comparison of MFil-Mamba’s effective receptive field with existing state-of-the-art baselines.

Fig. 5: Grad-CAM visualization on the ImageNet-1k validation set, illustrating model’s ability to focus on the main subject of the image.

Fig. 6: Object detection and instance segmentation visualizations on the MS COCO validation set. In each group of three successive images, the left image shows the original input, the middle shows object detection results with bounding boxes, and the right shows instance segmentation results with masks and bounding boxes.

Fig. 7: Semantic segmentation visualizations on the ADE20k dataset.

Strategy	#Params	FLOPs	Top-1 Acc.
Single Scan (Flatten)	30.93M	4.94G	82.79
Cross Scan (4-Direction)	30.93M	5.14G	82.95
Original + Single Filter	33.36M	5.41G	83.06
Multi Filter Scan	33.46M	5.56G	83.22

TABLE VI: Ablation of scanning modules in the Tiny variant.

Merging	#Param	FLOPs	Top-1 Acc.
Without Adpt. Weighting	33.46M	5.56G	83.10
With Adpt. Weighting	33.46M	5.56G	83.22

TABLE VII: Ablation study of adaptive weighting implementation.

$\mathbf{d\_state}$	$\mathbf{ssm\_ratio}$	#Params	FLOPs	Top-1 Acc.
1	1.0	33.46M	5.56G	83.22
1	2.0	48.27M	8.21G	83.08
2	2.0	48.31M	8.34G	83.10

TABLE VIII: Ablation study on (different) model configurations.

Figure 16

Figure 17

Explore similar work

Mar 17, 2026

SF-Mamba: Rethinking State Space Model for Vision

The realm of Mamba for vision has been advanced in recent years to strike for the alternatives of Vision Transformers (ViTs) that suffer from the quadratic complexity. While the recurrent scanning mechanism of Mamba offers computational efficiency, it inherently limits non-causal interactions between image patches. Prior works have attempted to address this limitation through various multi-scan strategies; however, these approaches suffer from inefficiencies due to suboptimal scan designs and frequent data rearrangement. Moreover, Mamba exhibits relatively slow computational speed under short token lengths, commonly used in visual tasks. In pursuit of a truly efficient vision encoder, we rethink the scan operation for vision and the computational efficiency of Mamba. To this end, we propose SF-Mamba, a novel visual Mamba with two key proposals: auxiliary patch swapping for encoding bidirectional information flow under an unidirectional scan and batch folding with periodic state reset for advanced GPU parallelism. Extensive experiments on image classification, object detection, and instance and semantic segmentation consistently demonstrate that our proposed SF-Mamba significantly outperforms state-of-the-art baselines while improving throughput across different model sizes. We will release the source code after publication.

Masakazu Yoshimura, Teruaki Hayashi, Yuki Hoshino +2

Sony Group Corporation, Tokyo, Japan.

Visual SSMs

Ask PDF

May 20, 2026

Deformba: Vision State Space Model with Adaptive State Fusion

State Space Models (SSMs) have emerged as a powerful and efficient alternative to Transformers, demonstrating linear-time complexity and exceptional sequence modeling capabilities. However, their application to vision tasks remains challenging. First, existing vision SSMs largely depend on manually designed fixed scanning methods to flatten image patches into sequences, which imposes predefined geometric structures and increases the complexity. Second, the broader adoption of vision SSMs is hindered in domains that require query-based interactions between distinct information streams. This is a result of the inherently causal and self-referential nature of SSMs designed for 1D sequence modeling tasks. This fusion mechanism is indispensable for critical perception tasks such as multi-view 3D fusion. To address these limitations, we propose Deformba, a context adaptive method that dynamically augments the spatial structural information while maintaining the linear complexity of SSMs. Deformba also allows multi-modal fusion like cross attention. To demonstrate the effectiveness and general applicability of Deformba, we test its performance on general 2D vision tasks such as image classification, object detection, and segmentation, as well as 3D vision tasks like BEV perception. Extensive experiments show that Deformba achieves strong performance across various visual perception benchmarks.

Hongyu Ke, Jack Morris, Yongkang Liu +4

Department of Computer Science, Georgia State University · University of Tennessee Knoxville · InfoTech Labs, Toyota Motor North America R&D.

Visual SSMs State Space Models

Ask PDF

Jul 3, 2026

Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics

State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.

Anvitha Ramachandran, Dhruv Parikh, Haoyang Fan +2

University of Southern California · DEVCOM Army Research Office

Visual SSMs State Space Models

Ask PDF