BanglaMamba: Exploring State Space Models for Bangla Fake News Detection

M. K. Khalidi Siam

cs.CLAug 25, 2026

BanglaMamba: Exploring State Space Models for Bangla Fake News Detection

Authors: M. K. Khalidi Siam

Organizations: Department of Computer Science and Engineering BRAC University Dhaka, Bangladesh

Abstract

Fake news detection has become an important Natural Language Processing (NLP) task due to the rapid spread of misinformation through online news platforms and social media. While transformer-based models such as BanglaBERT achieve strong performance for Bangla text classification, their quadratic computational complexity makes them less suitable for long-document processing in resource-constrained environments. This paper investigates Mamba-based State Space Models (SSMs) as an efficient alternative for Bangla fake news detection. We propose BanglaMamba and compare it with pre-trained BanglaBERT and a similarly configured BERT model trained from scratch. Experimental results show that BanglaBERT achieves the highest Macro-F1 score (0.9260), while BanglaMamba (0.9029) achieves performance comparable to the from-scratch CustomBERT (0.9057) despite using a different architecture. Meanwhile, BanglaMamba achieves approximately $2.2\times$ higher inference throughput and 49% lower inference peak GPU memory usage than the BERT-based models. Cross-dataset evaluation shows that BanglaBERT generalizes better to an external dataset, highlighting the importance of large-scale pretraining. These findings demonstrate that Mamba-based SSMs can provide a competitive and computationally efficient alternative to Transformer-based architectures for Bangla fake news detection, particularly in resource-constrained settings.

Figures & tables

Property	BanglaBERT	CustomBERT
Architecture	Transformer Encoder	Transformer Encoder
Parameters	13.7M	13.7M
Hidden Dimension	256	256
Hidden Layers	12	12
Maximum Context Length	512 Tokens	512 Tokens
Pretraining	Yes	No

Table 1: Configuration of BanglaBERT and CustomBERT.

Property	BanglaMamba
Architecture	Mamba
Parameters	12.61M
Hidden Dimension (d_model)	256
Hidden Layer (n_layer)	10
Maximum Context Length	512 Tokens

Table 2: Configuration of the BanglaMamba.

Figure 1: BanglaMamba model architecture.

Metric	BanglaBERT	BanglaMamba	CustomBERT
Macro F1	0.9260 $\pm$ 0.0008	0.9029 $\pm$ 0.0034	0.9057 $\pm$ 0.0078
AUC-ROC	0.9632 $\pm$ 0.0028	0.9420 $\pm$ 0.0103	0.9407 $\pm$ 0.0235
MCC	0.8534 $\pm$ 0.0024	0.8086 $\pm$ 0.0085	0.8137 $\pm$ 0.0144
PR-AUC	0.9896 $\pm$ 0.0013	0.9820 $\pm$ 0.0049	0.9810 $\pm$ 0.0097
ECE	0.0507 $\pm$ 0.0371	0.0457 $\pm$ 0.0134	0.0537 $\pm$ 0.0176
Fake F1	0.8753 $\pm$ 0.0011	0.8361 $\pm$ 0.0057	0.8408 $\pm$ 0.0136

Table 3: Classification performance comparison among BanglaBERT, BanglaMamba, and CustomBERT.

Metric	BanglaBERT	BanglaMamba	CustomBERT
P50 Latency (ms)	37.0767 $\pm$ 0.2701	14.8433 $\pm$ 2.2382	37.0333 $\pm$ 0.2845
P95 Latency (ms)	39.03 $\pm$ 3.0967	17.4267 $\pm$ 3.1683	39.05 $\pm$ 3.12
Throughput (sample/s)	1675.03 $\pm$ 24.8858	3623.2333 $\pm$ 125.4971	1680.13 $\pm$ 23.3548
Inference Peak VRAM (MB)	1011.9 $\pm$ 0	514.5 $\pm$ 0	1011.9 $\pm$ 0
Training Peak VRAM (MB)	5141.6 $\pm$ 0	1145.4 $\pm$ 0	5141.6 $\pm$ 0

Table 4: Efficiency comparison among BanglaBERT, BanglaMamba, and CustomBERT.

Metric	BanglaBERT		BanglaMamba		CustomBERT
	Short	Long	Short	Long	Short	Long
Macro F1	0.9309 $\pm$ 0.0047	0.9054 $\pm$ 0.0175	0.9030 $\pm$ 0.0012	0.9012 $\pm$ 0.0217	0.9049 $\pm$ 0.0063	0.9080 $\pm$ 0.0149
AUC-ROC	0.9662 $\pm$ 0.0062	0.9466 $\pm$ 0.0193	0.9435 $\pm$ 0.0088	0.9351 $\pm$ 0.0201	0.9424 $\pm$ 0.0213	0.9336 $\pm$ 0.0363
MCC	0.8633 $\pm$ 0.0099	0.8127 $\pm$ 0.0342	0.8092 $\pm$ 0.0046	0.8042 $\pm$ 0.0427	0.8120 $\pm$ 0.0122	0.8196 $\pm$ 0.0253
Fake F1	0.8831 $\pm$ 0.0082	0.8427 $\pm$ 0.0323	0.8357 $\pm$ 0.0024	0.8351 $\pm$ 0.0391	0.8390 $\pm$ 0.0106	0.8462 $\pm$ 0.0284

Table 5: Performance comparison of BanglaBERT, BanglaMamba, and CustomBERT on short and long articles.

Metric	BanglaBERT	BanglaMamba	CustomBERT
Macro F1	0.7914 $\pm$ 0.0348	0.5753 $\pm$ 0.0703	0.7227 $\pm$ 0.0892
AUC-ROC	0.8693 $\pm$ 0.0185	0.7598 $\pm$ 0.0164	0.7777 $\pm$ 0.1218
MCC	0.5913 $\pm$ 0.0708	0.2590 $\pm$ 0.0767	0.4746 $\pm$ 0.1696
PR-AUC	0.8622 $\pm$ 0.0076	0.7845 $\pm$ 0.0172	0.7197 $\pm$ 0.1502
ECE	0.0929 $\pm$ 0.0423	0.2564 $\pm$ 0.0472	0.1448 $\pm$ 0.0605
Fake F1	0.7763 $\pm$ 0.0365	0.4564 $\pm$ 0.1253	0.6824 $\pm$ 0.1081

Table 6: Classification performance among BanglaBERT, BanglaMamba, and CustomBERT on the cross-dataset.

Metric	BanglaBERT (Mean Pooling)
Macro F1	0.9144 $\pm$ 0.0081
AUC-ROC	0.9565 $\pm$ 0.0077
MCC	0.8300 $\pm$ 0.0172
PR-AUC	0.9886 $\pm$ 0.0019
ECE	0.0866 $\pm$ 0.0182
Fake F1	0.8565 $\pm$ 0.0124

Table 7: Classification performance of BanglaBERT using mean pooling instead of the [CLS] representation.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Specification
Programming Language	Python 3.11
Deep Learning Framework	PyTorch 2.4.0 + CUDA 12.4
Transformer Library	Transformers 4.40.0
State Space Model Library	Mamba-SSM 2.2.2
GPU	NVIDIA A100 (40 GB)

Appendix

Table 8: Experimental setup.

Configuration	Value
$d_{\mathrm{model}}$	256
$head\_hidden\_dim$	128
$n_{\mathrm{layer}}$	10
Dropout	0.2
Parameters	12.61M

Appendix

Table 9: Configurations of the BanglaMamba model.

Hyperparameter	BanglaBERT	BanglaMamba	CustomBERT
Vocabulary Size	32,000	32,000	32,000
Maximum Sequence Length	512	512	512
Number of Labels	2	2	2
Epochs	5	5	5
Batch Size	32	32	32
Gradient Accumulation	2	2	2

Appendix

Table 10: Training configurations for BanglaBERT, BanglaMamba, and CustomBERT.

Figure 2: Validation Macro-F1 across training epochs for BanglaBERT, BanglaMamba, and CustomBERT over three random seeds.

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9255	0.9255	0.9269
AUC-ROC	0.9607	0.9626	0.9662
MCC	0.8523	0.8517	0.8561
PR-AUC	0.9885	0.9892	0.9910
ECE	0.0313	0.0934	0.0273
Fake F1	0.8746	0.8747	0.8766

Appendix

Table 11: BanglaBERT classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	37.35	36.81	37.07
P95 Latency (ms)	42.60	37.07	37.42
Throughput (samples/s)	1649.46	1699.17	1676.46
Inference Peak VRAM (MB)	1011.9	1011.9	1011.9
Training Peak VRAM (MB)	5141.6	5141.6	5141.6

Appendix

Table 12: BanglaBERT efficiency metrics across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9036	0.9060	0.8992
AUC-ROC	0.9539	0.9352	0.9369
MCC	0.8075	0.8176	0.8007
PR-AUC	0.9873	0.9776	0.9810
ECE	0.0313	0.0479	0.0578
Fake F1	0.8384	0.8403	0.8297

Appendix

Table 13: BanglaMamba classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	16.07	16.20	12.26
P95 Latency (ms)	17.87	20.35	14.06
Throughput (samples/s)	3744.20	3631.85	3493.65
Inference Peak VRAM (MB)	514.5	514.5	514.5
Training Peak VRAM (MB)	1145.4	1145.4	1145.4

Appendix

Table 14: BanglaMamba efficiency metrics across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9051	0.9137	0.8982
AUC-ROC	0.9504	0.9579	0.9139
MCC	0.8108	0.8294	0.8010
PR-AUC	0.9845	0.9884	0.9700
ECE	0.0406	0.0467	0.0737
Fake F1	0.8405	0.8545	0.8273

Appendix

Table 15: CustomBERT classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	36.90	36.84	37.36
P95 Latency (ms)	37.37	37.13	42.65
Throughput (samples/s)	1690.40	1696.59	1653.40
Inference Peak VRAM (MB)	1011.9	1011.9	1011.9
Training Peak VRAM (MB)	5141.6	5141.6	5141.6

Appendix

Table 16: CustomBERT efficiency metrics across three random seeds.

Bengali NLP

See all 23 papers

Addressing Data Scarcity in Bangla Fake News Detection: An LLM-Based Dataset Augmentation Approach
May 2, 2026Ahmed Alfey Sani, Kazi Akib Zaoad, Shefayat E Shams Adib +2Class-Imbalanced Learning Synthetic Data Augmentation
Islamic University of Technology Gazipur, Bangladesh
Hybrid Feature Combinations with CNN for Bangla Fake News Classification
May 17, 2026Md Gulzar Hussain, Babe Sultana, Md Rinku AliFake News Detection Feature Selection
School of Software, Nanjing University of Information Science and Technology, Nanjing, China · Department of Computer Science and Engineering, Green University of Bangladesh, Dhaka, Bangladesh · School of Computer Science and Artiﬁcial Intelligence, Changzhou University, Changzhou, China
BanglaRhet: Benchmarking Classical and Transformer Models for Rhetorical and Persuasion Detection in Bangla Political Speech
Oct 7, 2026Rohit Kumar Sen, Anik ChowdhuryBengali NLP Political Discourse Analysis
Department of Computer Science and Engineering North East University Bangladesh Sylhet, Bangladesh

May 2, 2026J/K move · Enter open · S save

Addressing Data Scarcity in Bangla Fake News Detection: An LLM-Based Dataset Augmentation Approach

Ahmed Alfey Sani, Kazi Akib Zaoad, Shefayat E Shams Adib, Md Abdul Muqtadir+1

Islamic University of Technology Gazipur, Bangladesh

Class-Imbalanced Learning Synthetic Data Augmentation Fake News Detection Text Classification Low-Resource Language Processing

Open paper AskPDF arXiv

The growing spread of misinformation in digital media highlights the need for reliable fake news detection systems, yet progress in under-resourced languages such as Bangla is limited by small and imbalanced datasets. This study investigates whether Large Language Model (LLM) based augmentation can effectively address this limitation and improve Bangla fake news classification. Existing datasets remain valuable but highly imbalanced, limiting model performance, and LLM based augmentation for Bangla has been scarcely explored. To fill this gap, we propose a systematic augmentation framework that generates synthetic Bangla news articles using the instruction tuned Gemma 3 27B IT model, supported by semantic filtering and controlled subsampling to preserve label consistency and diversity. We compare zero shot and few shot prompting, evaluate multiple augmentation rates, and examine random versus similarity-based selection strategies. Our experiments show that augmenting only the minority class with a high augmentation rate and random subsampling yields the strongest gains, raising the Fake News F1 score from 0.85 to 0.88. To support reproducibility and further research in this low-resource domain, we publicly release 4,545 synthetically generated Bangla fake news samples along with our full implementation. These findings demonstrate that well-designed LLM-driven augmentation can significantly improve fake news detection in low resource settings and provide a practical foundation for advancing multilingual misinformation research.

Abstract

Bengali NLP

Explore similar work

Addressing Data Scarcity in Bangla Fake News Detection: An LLM-Based Dataset Augmentation Approach

Hybrid Feature Combinations with CNN for Bangla Fake News Classification

BanglaRhet: Benchmarking Classical and Transformer Models for Rhetorical and Persuasion Detection in Bangla Political Speech