Detecting Hate and Inflammatory Content in Bengali Memes: A New Multimodal Dataset and Co-Attention Framework

Rakib Ullah; Mominul islam; Md Sanjid Hossain; Md Ismail Hossain

cs.CLFeb 25, 2026

Detecting Hate and Inflammatory Content in Bengali Memes: A New Multimodal Dataset and Co-Attention Framework

Authors: Rakib Ullah, Mominul islam, Md Sanjid Hossain, Md Ismail Hossain

Organizations: Sylhet Engineering College · Computer Science And Engineering, Sylhet Engineering College, Sylhet, Bangladesh · Daffodil International University · Computer Science And Engineering, Daffodil International University, Dhaka, Bangladesh

Abstract

Internet memes have become a dominant form of expression on social media, including within the Bengali speaking community. While often humorous, memes can also be exploited to spread offensive, harmful, and inflammatory content targeting individuals and groups. Detecting this type of content is exceptionally challenging due to its satirical, subtle, and culturally specific nature. This problem is magnified for low-resource languages like Bengali, as existing research predominantly focuses on high-resource languages. To address this critical research gap, we introduce Bn-HIB (Bangla Hate Inflammatory Benign), a novel dataset containing 3,247 manually annotated Bengali memes categorized as Benign, Hate, or Inflammatory. Significantly, Bn- HIB is the first dataset to distinguish inflammatory content from direct hate speech in Bengali memes. Furthermore, we propose the MCFM (Multi-Modal Co-Attention Fusion Model), a simple yet effective architecture that mutually analyses both the visual and textual elements of a meme. MCFM employs a co-attention mechanism to identify and fuse the most critical features from each modality, leading to a more accurate classification. Our experiments show that MCFM significantly outperforms several state-of-the-art models on the Bn-HIB dataset, demonstrating its effectiveness in this nuanced task. To facilitate reproducibility and future research, the Bn-HIB dataset has been made publicly available through Mendeley Data. Warning: This work contains material that may be disturbing to some audience members. Viewer discretion is advised

Figures & tables

Class	Inclusion Criteria	Exclusion Criteria
Inflammatory	– Controversial statements or imagery; norm-challenging discourse; inflammatory tone or humor; boundary-pushing content.	– Accidental controversy; mild disagreement; primarily educational focus.
Hate	– Specific names or groups; group-identifiable traits; pronouns with clear referents; direct or indirect addressing.	– Generic mentions; self-referential content.
Benign	– Positive or neutral tone; everyday relatable humor; inclusive, non-divisive content.	– Any provocative or targeting cue; implicit controversial element.

TABLE I : Annotation Inclusion And Exclusion Criteria

Panel of Figure 2 — Fig. 1 : Meme (a) portrays Hindus as hypocrites for allegedly destroying their own temples and blaming Muslims, while Meme (b) specifically targets an organization and individual, making it more directed but less representative of the broader Hindu population.

Fig. 2 : Data Collection and Pre-Processing Overview

Fig. 3 : Proposed Multi-Modal Co-Attention Fusion Model (MCFM)

Category	Class Split			Language Split			Text Length (Words)
	Train	Val	Test	Train	Val	Test	Mean / Max / Min / Mode / Med.
Hate (HM)	811	174	173	–	–	–	–
Inflammatory (IM)	773	166	167	–	–	–	–
Benign (BM)	688	147	148	–	–	–	–
Bengali	–	–	–	1190	255	256	–
Code-switched	–	–	–	332	72	71	–

TABLE II: Dataset Statistics

Fig. 4 : Annotation Decision Tree

Category	Model / Strategy	Ref.
Textual	Banglish-BERT	[ 28 ]
	mBERT	[ 29 ]
	XLM-RoBERTa	[ 30 ]
	MuRIL	[ 31 ]
	mDeBERTa-v3	[ 32 , 33 ]
Visual	ResNet-50	[ 34 ]

TABLE III : Textual, Visual, and Multi-Modal fusion Models

Textual Models			Visual Models
Model	Acc.	F1	Model	Acc.	F1
Banglish BERT	0.7070	0.7109	ResNet-50	0.5553	0.5509
XLM-R	0.7152	0.7162	EffNet-B3	0.5205	0.5205
M-BERT	0.6865	0.6906	ViT-B16	0.4734	0.4790
Muril	0.6475	0.6462	ConvNext	0.5738	0.5731
MDeBERTa-V3	0.7193	0.7235	Swin-V2	0.5922	0.5909

TABLE IV : Model Performance (Accuracy / Macro F1)

Fig. 5 : Class-Wise Performance Of Top Models

Panel of Figure 10 — Fig. 6 : Error analysis: confusion matrix and performance trends.

Component	Textual (MDeBERTa-V3)	Visual (Swin-B)	Multi-Modal (Fusion)
Base Model	microsoft/ mdeberta-v3-base	Swin-B ( IMAGENET1K_V1 )	Transformer + ImageNet-pretrained CNN
Hidden Size	768	–	Text: 1024
Input Size	Max token length = 256	$224\times 224$	$224\times 224$
Tokenization / Preprocessing	Native tokenizer	Flip, rotation, jitter (train); resize + normalize (eval)	Tokenizer + image preprocessing
Special Tokens	[CLS], [SEP], [PAD]	–	–
Classification Head	ReLU $\rightarrow$ Dropout(0.5) $\rightarrow$ Linear (3 classes)	Dropout(0.5) $\rightarrow$ Linear (3 classes)	Dropout(0.5) $\rightarrow$ Linear

TABLE V : Experimental Details of Textual, Visual and Multi-Modal Models

Harmful Meme Detection

See all 23 papers

Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection
Feb 22, 2026Raihan Tanvir, Md. Golam Rabiul AlamHate Speech Detection Low-Resource Language Processing
Department of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh · Department of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka 1208, Bangladesh
BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes
Jul 4, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman +3Low-Resource Language Processing
Ahsanullah University of Science and Technology, Dhaka-1208, Bangladesh
AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes
Jul 29, 2026Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat +3Arabic NLP Hate Speech Detection
Qatar Computing Research Institute, Qatar · 2APAVI.AI, France · 3Hamad Bin Khalifa University, Qatar +1

Feb 22, 2026J/K move · Enter open · S save

Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection

Raihan Tanvir, Md. Golam Rabiul Alam

Department of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh · Department of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka 1208, Bangladesh

Hate Speech Detection Low-Resource Language Processing Cross-Modal Attention

Open paper AskPDF arXiv

Hateful content on social media increasingly appears as multimodal memes that combine images and text to convey harmful narratives. In low-resource languages such as Bengali, automated detection remains challenging due to limited annotated data, class imbalance, and pervasive code-mixing. To address these issues, we augment the Bengali Hateful Memes (BHM) dataset with semantically aligned samples from the Multimodal Aggression Dataset in Bengali (MIMOSA), improving both class balance and semantic diversity. We propose the Enhanced Dual Co-attention Framework (xDORA), integrating vision encoders (CLIP, DINOv2) and multilingual text encoders (XGLM, XLM-R) via weighted attention pooling to learn robust cross-modal representations. Building on these embeddings, we develop a FAISS-based k-nearest neighbor classifier for non-parametric inference and introduce RAG-Fused DORA, which incorporates retrieval-driven contextual reasoning. We further evaluate LLaVA under zero-shot, few-shot, and retrieval-augmented prompting settings. Experiments on the extended dataset show that xDORA (CLIP + XLM-R) achieves macro-average F1-scores of 0.78 for hateful meme identification and 0.71 for target entity detection, while RAG-Fused DORA improves performance to 0.79 and 0.74, yielding gains over the DORA baseline. The FAISS-based classifier performs competitively and demonstrates robustness for rare classes through semantic similarity modeling. In contrast, LLaVA exhibits limited effectiveness in few-shot settings, with only modest improvements under retrieval augmentation, highlighting constraints of pretrained vision-language models for code-mixed Bengali content without fine-tuning. These findings demonstrate the effectiveness of supervised, retrieval-augmented, and non-parametric multimodal frameworks for addressing linguistic and cultural complexities in low-resource hate speech detection.

Abstract

Harmful Meme Detection

Explore similar work

Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection

BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes

AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes