CAGMamba: Context-Aware Gated Cross-Modal Mamba Network for Multimodal Sentiment Analysis

Minghai Jiao; Jing Xiao; Peng Xiao; Ende Zhang; Shuang Kan; Wenyan Jiang; Jinyao Li; Yixian Liu

cs.CLApr 4, 2026

CAGMamba: Context-Aware Gated Cross-Modal Mamba Network for Multimodal Sentiment Analysis

Authors: Minghai Jiao, Jing Xiao, Peng Xiao, Ende Zhang, Shuang Kan, Wenyan Jiang, Jinyao Li, Yixian Liu,

Organizations: School of Computer Science and Engineering, Northeastern University, Shenyang, China · Xingning Power Supply Bureau, Guangdong Power Grid Co., Ltd., Meizhou, China

Abstract

Multimodal Sentiment Analysis (MSA) requires effective modeling of cross-modal interactions and contextual dependencies while remaining computationally efficient. Existing fusion approaches predominantly rely on Transformer-based cross-modal attention, which incurs quadratic complexity with respect to sequence length and limits scalability. Moreover, contextual information from preceding utterances is often incorporated through concatenation or independent fusion, without explicit temporal modeling that captures sentiment evolution across dialogue turns. To address these limitations, we propose CAGMamba, a context-aware gated cross-modal Mamba framework for dialogue-based sentiment analysis. Specifically, we organize the contextual and the current-utterance features into a temporally ordered binary sequence, which provides Mamba with explicit temporal structure for modeling sentiment evolution. To further enable controllable cross-modal integration, we propose a Gated Cross-Modal Mamba Network (GCMN) that integrates cross-modal and unimodal paths via learnable gating to balance information fusion and modality preservation, and is trained with a three-branch multi-task objective over text, audio, and fused predictions. Experiments on three benchmark datasets demonstrate that CAGMamba achieves state-of-the-art or competitive results across multiple evaluation metrics. All codes are available at https://github.com/User2024-xj/CAGMamba.

Figures & tables

Figure 1: Overall Framework of the Proposed CAGMamba Model.

Figure 2: Architecture of the GCMN module. The upper part shows the three-stream gated fusion structure. The lower part illustrates the internal architecture of the BSSM.

Method	CMU-MOSI					CMU-MOSEI
Method	Acc-2	F1	Acc-7	MAE	Corr	Acc-2	F1	Acc-7	MAE	Corr
MISA	81.80/83.40	81.70/83.60	42.30	0.78	0.78	83.60/85.50	83.80/85.30	52.20	0.56	0.76
Self-MM	83.44/85.46	83.36/85.43	46.67	0.71	0.80	83.76/85.15	83.82/84.90	53.87	0.53	0.77
MMIM	84.14/86.06	84.00/85.98	46.65	0.70	0.80	82.24/85.97	82.66/85.94	54.24	0.53	0.77
ConFEDE	84.17/85.52	84.13/85.52	42.27	0.74	0.78	81.65/85.82	82.17/85.83	54.86	0.52	0.78
CLGSI	83.97/86.43	83.63/86.25	49.67	0.70	0.79	84.01/86.32	84.21/86.18	54.56	0.53	0.76

Table 1: Comparison with representative baselines. Acc-2 reports two values: negative/non-negative and negative/positive classification accuracy, respectively. The best results are marked in bold , while the second-best results are underlined.

Method	Acc-5 $\uparrow$	Acc-3 $\uparrow$	Acc-2 $\uparrow$	F1 $\uparrow$	MAE $\downarrow$	Corr $\uparrow$
LMF	40.53	64.68	77.77	77.88	0.441	0.576
TFN	39.30	65.12	78.38	78.62	0.432	0.591
MulT	37.94	64.77	78.56	79.66	0.453	0.564
MISA	-	-	76.54	76.59	0.447	0.563
MAG-BERT	-	-	74.44	71.75	0.492	0.399
Self-MM	41.53	65.47	80.04	80.44	0.425	0.595

Table 2: Comparison with representative baselines. The best results are marked in bold , while the second-best results are underlined.

Experiment	Configuration	Acc-2 $\uparrow$
Context	MMML (no context)	86.31
	MMML + concat	87.17
	GCMN + concat	87.31
	GCMN + gated seq. (ours)	88.19
Fusion	Single-path (no gate)	86.21
	Dual-path + fixed gate	87.31

Table 3: Ablation study evaluating the impact of contextual modeling and fusion strategies. The best results are marked in bold , while the second-best results are underlined.

$(K_{\text{t}},\,K_{\text{a}})$	(0, 0)	(1, 0)	(0, 1)	(1, 1)	(1, 2)	(2, 2)	(3, 1)	(2, 1)
Acc-2 $\uparrow$	86.33	86.98	86.72	87.54	87.15	87.63	87.95	88.19

Table 4: Ablation study on context window size on CMU-MOSI.

K_{\text{t}}

and

K_{\text{a}}

denote the number of preceding utterances used as context for text and audio, respectively. The best result is marked in bold and the second-best is underlined .

Method	Params	FLOPs	Acc-2 $\uparrow$
Vision Mamba	1.40M	0.12G	84.97/85.49
Video Mamba	1.41M	0.13G	85.74/86.89
Transformer	4.45M	0.38G	84.74/86.20
MSAmba	1.41M	0.13G	85.99 / 87.43
GCMN (ours)	0.80M	0.10G	88.19 / 88.15

Table 5: Efficiency comparison in terms of model size (Params), computational cost (FLOPs), and performance (Acc-2). The best results are marked in bold , while the second-best results are underlined.

Panel of Figure 3 — Figure 3: T-SNE visualization of the learned representations on CMU-MOSI. Compared to baseline methods, our model produces more compact intra-class clusters and better inter-class separation, indicating improved representation quality.

Multimodal Sentiment Analysis

Multimodal sentiment analysis combines language, voice, and visual cues to infer sentiment, polarity, and emotional intensity.

See all 54 papers

AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba
Mar 19, 2026Yan Li, Yifei Xing, Xiangyuan Lan +3Multimodal Sentiment Analysis Multimodal Fusion
Pengcheng Laboratory, Shenzhen, China · Shaanxi University of Science & Technology, Xi’an, China · School of Computer Science, Northwestern Polytechnical University, Xi’an, China
Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis
Feb 23, 2026Chunlei Meng, Jiabin Luo, Zhenglin Yan +4Multimodal Sentiment Analysis Multimodal Disentangled Representation Learning
Fudan University · Peking University · University of Macau
Prediction-Layer Branch Calibration for Multimodal Sentiment Analysis
Sep 29, 2026Yulin Sun, Kele Xu, Yong DouMultimodal Sentiment Analysis Multimodal Fusion
National University of Defense Technology

Mar 19, 2026J/K move · Enter open · S save

AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba

Yan Li, Yifei Xing, Xiangyuan Lan, Xin Li+2

Pengcheng Laboratory, Shenzhen, China · Shaanxi University of Science & Technology, Xi’an, China · School of Computer Science, Northwestern Polytechnical University, Xi’an, China

Multimodal Sentiment Analysis Multimodal Fusion Cross-Modal Alignment State Space Models

Open paper AskPDF arXiv

In the era of large-scale pre-trained models, effectively adapting general knowledge to specific affective computing tasks remains a challenge, particularly regarding computational efficiency and multimodal heterogeneity. While Transformer-based methods have excelled at modeling inter-modal dependencies, their quadratic computational complexity limits their use with long-sequence data. Mamba-based models have emerged as a computationally efficient alternative; however, their inherent sequential scanning mechanism struggles to capture the global, non-sequential relationships that are crucial for effective cross-modal alignment. To address these limitations, we propose AlignMamba-2, an effective and efficient framework for multimodal fusion and sentiment analysis. Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy, promoting geometric and statistical consistency between modalities without incurring any inference-time overhead. More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts to explicitly handle data heterogeneity during the fusion process. Extensive experiments on four challenging benchmarks, including dynamic time-series (on the CMU-MOSI and CMU-MOSEI datasets) and static image-related tasks (on the NYU-Depth V2 and MVSA-Single datasets), demonstrate that AlignMamba-2 establishes a new state-of-the-art in both effectiveness and efficiency across diverse pattern recognition tasks, ranging from dynamic time-series analysis to static image-text classification.

Abstract

Multimodal Sentiment Analysis

Explore similar work

AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba

Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis

Prediction-Layer Branch Calibration for Multimodal Sentiment Analysis