Organizations: School of Computer Science, Beijing Institute of Technology, Beijing, China · School of Aerospace Engineering, Beijing Institute of Technology, Beijing, China · Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
When applied to two-dimensional features such as images, Mamba-based methods either flatten the feature map into 1D sequences and fuse multiple scans, which breaks spatial adjacency and multiplies computation, or recurse directly on the 2D grid but rescale the recurrence with a fixed input-independent factor and aggregate only the predecessors on a single anti-diagonal, leaving the discrete step unconstrained by 2D stability theory. We propose FM-Mamba, a native 2D selective state space model built on the general Fornasini--Marchesini (FM) model. Its coupled recurrence evolves along both spatial axes; cross-axis coupling induces a discrete diagonal-predecessor term A~0, and two independent input-dependent step sizes Δ1(i,j), Δ2(i,j) govern propagation along the two axes, so that the coefficients adapt per position and per direction. A normalized ratio rk=a0,k/(a1,ka2,k) yields a closed-form piecewise bound on the two step sizes; when a configuration falls outside the stable interval the steps are projected onto this bound, and the criterion is re-verified per position after training, which suppresses systematic local amplification without global damping. The recurrence is executed by a dual-diagonal wavefront scan: each update gathers information from two anti-diagonals, and the input enters the current state through a single projection, so that neither training nor inference flattens the feature map. Code is available at https://github.com/BitSsm/FM-Mamba-Runtime
Figures & tables
Figure 1: Dual-Diagonal Wavefront Scan: black marks the anti-diagonal i+j on which h(i,j) resides, red the anti-diagonal i+j−1 read through A~1(i,j)h(i−1,j) and A~2(i,j)h(i,j−1) , and blue the anti-diagonal i+j−2 read through A~0(i,j)h(i−1,j−1)
Condition
a0,k
rk
Step constraint
I
−k
[−1,−0.0625]
none
II
−k2
−1
none
III
−2.5k2
−2.5
≤0.9ln(5)/16
Table 1: The three conditions of A0
Figure 2: Training process comparison. Left: the flattening baselines, 2M2D2AT and the three conditions of 2MFM2AT ; right: 2MFM2AFM and its PSD variant in three conditions. Circles, squares, triangles and stars mark epochs 5, 10, 20 and 40, respectively; the horizontal axis is cumulative training time.
Training cost
Validation quality
Architecture
Condition
Params (M)
Time/ epoch (s)
Total (h)
Peak mem. (GiB)
GPU energy (kWh)
Best epoch
mAP (%)
micro-F1 (%)
macro-F1 (%)
Val. BCE
Vision Mamba [ 29 ]
–
24.720
252.53
2.806
14.63
1.545
20
51.260
57.176
46.372
0.0807
VMamba [ 17 ]
–
24.365
205.87
2.287
9.76
1.097
17
51.366
56.654
44.310
0.0782
2DMamba [ 27 ]
–
24.365
197.91
2.199
9.85
1.096
17
51.433
56.351
44.067
0.0781
2M2D2AT [ 3 ]
–
24.638
497.98
5.533
21.99
2.892
22
52.034
58.293
48.151
0.0807
2M2D2A2D
–
22.063
694.18
7.713
25.03
4.081
34
52.581
58.550
48.617
0.0847
Table 2: Training cost and validation quality
Architecture
Condition
mAP (%)
micro-F1 (%)
macro-F1 (%)
micro-P (%)
micro-R (%)
FPS
Compute (GFLOPs/img)
Peak mem. (MiB)
Vision Mamba [ 29 ]
–
51.281
57.647
46.870
71.092
48.479
72.96
9.094
125.32
VMamba [ 17 ]
–
50.748
56.938
44.257
72.947
46.692
63.71
7.864
123.85
2DMamba [ 27 ]
–
50.832
56.970
44.219
73.140
46.655
74.09
7.871
123.85
2M2D2AT [ 3 ]
–
52.517
58.685
48.409
71.257
49.884
53.76
8.700
137.88
2M2D2A2D
–
52.289
59.065
48.814
70.418
50.864
64.15
7.686
126.67
2MFM2AT
I
52.395
58.614
48.320
71.443
49.692
64.45
8.775
126.48
Table 3: Test-set classification quality and inference performance
State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at https://github.com/puskal-khadka/MFil-Mamba.
Puskal Khadka, KC Santosh
USD Artifical Intelligence Research, Department of Computer Science, University of South Dakota, Vermillion, 57069, USA.
Balancing fine-grained local modeling with long-range dependency capture under computational constraints remains a central challenge in sequence modeling. While Transformers provide strong token mixing, they suffer from quadratic complexity, whereas Mamba-style selective state-space models (SSMs) scale linearly but often struggle to capture high-rank and synchronous global interactions. We present a consistency boundary analysis that characterizes when diagonal short-memory SSMs can approximate causal attention and identifies structural gaps that remain. Motivated by this analysis, we propose InfoMamba, an attention-free hybrid architecture. InfoMamba replaces token-level self-attention with a concept bottleneck linear filtering layer that serves as a minimal-bandwidth global interface and integrates it with a selective recurrent stream through information-maximizing fusion (IMF). IMF dynamically injects global context into the SSM dynamics and encourages complementary information usage through a mutual-information-inspired objective. Extensive experiments on classification, dense prediction, and non-vision tasks show that InfoMamba consistently outperforms strong Transformer and SSM baselines, achieving competitive accuracy-efficiency trade-offs while maintaining near-linear scaling.
Youjin Wang, Jiaqiao Zhao, Rong Fu +5
Renmin University of China, Beijing, China · University of Macau, Macau SAR, China · Central South University, Changsha, China +1
Scaling inference-time compute has emerged as an important driver of LLM performance, making inference efficiency a central focus of model design alongside model quality. While the current Transformer-based models deliver strong model quality, their quadratic compute and linear memory make inference expensive. This has spurred the development of sub-quadratic models with reduced linear compute and constant memory requirements. However, many recent linear models trade off model quality and capability for algorithmic efficiency, failing on tasks such as state tracking. Moreover, their theoretically linear inference remains hardware-inefficient in practice. Guided by an inference-first perspective, we introduce three core methodological improvements inspired by the state space model (SSM) viewpoint of linear models. We combine: (1) a more expressive recurrence derived from SSM discretization, (2) a complex-valued state update rule that enables richer state tracking, and (3) a multi-input, multi-output (MIMO) formulation for better model performance without increasing decode latency. Together with architectural refinements, our Mamba-3 model achieves significant gains across retrieval, state-tracking, and downstream language modeling tasks. At the 1.5B scale, Mamba-3 improves average downstream accuracy by 0.6 percentage points compared to the next best model (Gated DeltaNet), with Mamba-3's MIMO variant further improving accuracy by another 1.2 points for a total 1.8 point gain. Across state-size experiments, Mamba-3 achieves comparable perplexity to Mamba-2 despite using half of its predecessor's state size. Our evaluations demonstrate Mamba-3's ability to advance the performance-efficiency Pareto frontier.
Aakash Lahoti, Kevin Y. Li, Berlin Chen +5
Carnegie Mellon University · Princeton University · Together AI +1