MambaVF: State Space Model for Efficient Video Fusion

Zixiang Zhao; Yukun Cui; Lilun Deng; Haowen Bai; Haotong Qin; Tao Feng; Konrad Schindler

cs.CVFeb 5, 2026

MambaVF: State Space Model for Efficient Video Fusion

Authors: Zixiang Zhao, Yukun Cui, Lilun Deng, Haowen Bai, Haotong Qin, Tao Feng, Konrad Schindler

Organizations: ETH Zürich · Xi’an Jiaotong University · Nanyang Technological University · Tsinghua University

Abstract

Video fusion aims to integrate complementary information from multiple source videos while preserving temporal consistency. Effective modeling of temporal dynamics is essential to this goal, yet existing methods incur substantial computational overhead from optical flow estimation and feature warping. In this paper, we present MambaVF, an efficient video fusion framework that uses state space model (SSM) to achieve temporal modeling without explicit motion estimation. First, by formulating video fusion as a sequential state update process, MambaVF captures long-range temporal dependencies with linear complexity, significantly reducing computation and memory costs. Second, the lightweight SSM-based fusion module eliminates conventional flow-guided alignment. Instead, it introduces a mutual state fusion module and a spatio-temporal bidirectional scanning mechanism to enable information aggregation across video streams. Experiments on multiple benchmarks confirm that MambaVF reaches state-of-the-art performance in different video fusion applications (multi-exposure, multi-focus, infrared-visible, medical), while reducing parameters by >90% and FLOPs by >80%, resulting in >50% shorter runtime. Project page: https://mambavf.github.io

Figures & tables

Figure 1: Compared with UniVF ( Zhao et al., 2025 ) , our MambaVF not only attains state-of-the-art performance on VF-Bench ( Zhao et al., 2025 ) , but requires only 5.7% of the parameters and 13.4% of the FLOPs, while achieving a 2.1× speedup.

Figure 2: Parameter, FLOPs, and runtime contribution of optical flow and feature warping modules in UniVF ( Zhao et al., 2025 ) .

Figure 3: An overview of our MambaVF architecture. The MuFuse Mamba Encoder processes the embedded source features through VSS Blocks, whose MuFuse modules exchange complementary information via reciprocal state interaction. STB scanning supplies eight matching paths for the two sources: four forward orders with their full reversals. Tuples list axes from fastest to slowest. Spatial downsampling reduces encoding cost, and the decoder reconstructs the fused center frame.

Figure 4: Visual comparison of fused results for multi-exposure video fusion.

Figure 5: Visual comparison of fused results for multi-focus video fusion.

Figure 6: Visual comparison of fused results for infrared-visible video fusion.

Figure 7: Visual comparison of fused results for MVF.

	VF-Bench Multi-Exposure Fusion Branch (540p)							VF-Bench Multi-Focus Fusion Branch (480p)
	VIF $\uparrow$	SSIM $\uparrow$	MI $\uparrow$	Qabf $\uparrow$	BiSWE $\downarrow$	MS2R $\downarrow$		VIF $\uparrow$	SSIM $\uparrow$	MI $\uparrow$	Qabf $\uparrow$	BiSWE $\downarrow$	MS2R $\downarrow$
CUNet	0.50	0.85	1.85	0.39	7.55	0.20	CUNet	0.53	0.86	3.52	0.68	10.23	0.42
HoLoCo	0.50	0.86	2.56	0.42	8.22	0.19	RFL	0.77	0.90	6.31	0.78	8.46	0.28
CRMEF	0.62	0.94	2.60	0.63	8.72	0.19	EPT	0.76	0.90	6.33	0.78	8.50	0.29
TC-MoA	0.74	0.99	2.93	0.72	7.82	0.16	TC-MoA	0.75	0.90	5.27	0.77	8.39	0.28
FILM	0.77	0.99	4.35	0.72	8.28	0.17	FILM	0.75	0.89	5.06	0.78	8.61	0.33

Table 1: Quantitative evaluation results for the low-resolution MEF and MFF tasks. The red and blue highlights indicate the best and second-best scores.

Table 9

Descriptions	Configurations			Metrics
Descriptions	Scan type	3D Decoder	Multi-frame	VIF $\uparrow$	SSIM $\uparrow$	MI $\uparrow$	Qabf $\uparrow$	BiSWE $\downarrow$	MS2R $\downarrow$
Exp. I: 2D spatial scan	2D	✗	✓	0.74	0.98	3.83	0.70	8.02	0.20
Exp. II: 1D temporal scan	1D	✗	✓	0.62	0.95	2.97	0.58	8.35	0.17
Exp. III: w/o state interaction	3D	✗	✓	0.79	0.99	4.16	0.73	7.87	0.16
Exp. IV: Single-frame input	2D	✗	✗	0.68	0.98	3.49	0.70	7.94	0.18
Exp. V: 3D Decoder	3D	✓	✓	0.79	0.99	3.86	0.73	7.68	0.16

Table 4: Results of the ablation experiments, with the best-performing values highlighted in red .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Visual comparison of fused results for multi-exposure video fusion.

Figure 10: Visual comparison of fused results for multi-focus video fusion.

Figure 11: Visual comparison of fused results for infrared-visible video fusion.

Figure 12: Visual comparison of fused results for medical video fusion.

Explore similar work

MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction
Apr 2, 2026Xilai Li, Weijun Jiang, Xiaosong Li +5Visible Image Fusion Optical Flow
Foshan University, Foshan, China · Kunming University of Science and Technology, Kunming, China · China University of Mining and Technology, Beijing, China
TTF: Temporal Token Fusion for Efficient Video-Language Model
May 8, 2026Simin Huo, Ning LIVideo-Language Models Qwen3
Shanghai Jiao Tong University
FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection
May 23, 2025Damith Chamalke Senadeera, Muhammad Awais, Shibo Li +2Fine-Grained Video Understanding Violence Detection
Queen Mary University of London, UK · Digital Environment Research Institute (DERI), London, UK · Queen’s University Belfast, UK

Apr 2, 2026, cs.CVJ/K move · Enter open · S save

MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

Xilai Li, Weijun Jiang, Xiaosong Li, Yang Liu+4

Foshan University, Foshan, China · Kunming University of Science and Technology, Kunming, China · China University of Mining and Technology, Beijing, China

Visible Image Fusion Optical Flow Motion-Based Perception Infrared Cross-Modal Attention

Open paper AskPDF arXiv

Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results. However, most existing methods are designed for static image fusion and cannot effectively handle frame-to-frame motion in videos. Current video fusion methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost. To mitigate these challenges, we propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism that enhances efficiency while maintaining superior fusion quality. Specifically, we leverage optical flow to identify dynamic regions in multi-modal sequences, adaptively allocating computationally intensive cross-modal attention to these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweight weak interaction module is employed to maintain structural and appearance integrity. By decoupling the processing of dynamic and static regions, MAVFusion simultaneously preserves temporal consistency and fine-grained details while significantly accelerating inference. Extensive experiments demonstrate that MAVFusion achieves state-of-the-art performance on multiple infrared and visible video benchmarks, achieving a speed of 14.16 FPS at

640 \times 480

resolution. The source code will be available at https://github.com/ixilai/MAVFusion.

Abstract

Explore similar work

MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

TTF: Temporal Token Fusion for Efficient Video-Language Model

FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection