Addressing Overcommitment in the Reasoning of Gendered Economic Memes under Multimodal Ambiguity

Kushal Kanwar; Dushyant Singh Chauhan; Kapil Rana; Gopendra Vikram Singh; Nils Lukas

cs.LGOct 8, 2026

Addressing Overcommitment in the Reasoning of Gendered Economic Memes under Multimodal Ambiguity

Authors: Kushal Kanwar, Dushyant Singh Chauhan, Kapil Rana, Gopendra Vikram Singh, Nils Lukas

Organizations: Jaypee University of Information Technology · Mohamed bin Zayed University of Artificial Intelligence · Thapar Institute of Engineering & Technology · Dr. B. R. Ambedkar National Institute of Technology Jalandhar

Abstract

Multimodal meme understanding is increasingly used to analyze socially sensitive content, yet existing models often exhibit biased behavior when interpreting economic dependence and social roles under ambiguity. Many memes express economic relationships through sparse text or symbolic visual cues, providing insufficient evidence for gendered attribution. In such underspecified settings, models tend to rely on pretraining correlations, leading to hallucinated and stereotypical economic role assignments. In this work, we study gendered economic dependence in image-text memes through the lens of contextual sufficiency and identify epistemic overcommitment-inferring roles without adequate evidence-as a primary source of bias. We propose CGER-Net, a context-grounded multimodal framework that estimates whether the input provides sufficient evidence for gendered economic reasoning and applies evidence-gated inference to enable confident attribution when cues are explicit while favoring principled abstention otherwise. We evaluate CGER-Net on EconMeme-GE, a curated dataset of image-text memes annotated as Men, Women, Neutral, or Ambiguous. Across strong contemporary multimodal baselines, CGER-Net reduces Gender Overcommitment Rate by up to 44% on ambiguous instances while maintaining comparable accuracy on unambiguous cases. Human evaluation further shows that 79% of generated rationales are judged as epistemically aligned with the available evidence. These results highlight the importance of modeling when not to infer for reliable and responsible multimodal analysis.

Figures & tables

Panel of Figure 1 — Figure 1: Illustrative examples from EconMeme-GE , showing image description, textual content, annotation labels, and corresponding human annotation rationales.

Model	Type	Acc	F1	AR	GOR $\downarrow$	RAR $\uparrow$
CLIP ViT-L/14	VLM	70.4	66.9	0.44	0.37	–
BLIP-2	VLM	72.1	68.3	0.48	0.31	–
InstructBLIP	VLM	73.2	69.1	0.50	0.28	–
LLaVA-1.6	LMM	74.5	70.6	0.52	0.34	0.46
Qwen-VL-Chat	LMM	75.1	71.3	0.51	0.32	0.49
InternVL-Chat	LMM	75.6	71.8	0.53	0.30	0.51

Table 1: The performance comparison of MLM baseline models with CGER-Net shows that under ambiguity, GOR measures (lower is better) unjustified gendered economic attribution. The consistency between generated rationales and predictions is measured by RAR (with higher values indicating better consistency).

Panel of Table 5 — Table 5: Qualitative analysis with rationale inspection for gendered economic dependence reasoning. The table contrasts predictions from LLaVA-1.6 (a strong multimodal large language model baseline) with CGER-Net outputs and their corresponding rationales. Color coding highlights epistemically appropriate abstention ( blue ), correct grounded attribution ( green ), and unjustified or incorrect inference ( red ). Rationales are shown in abbreviated form for clarity.

Explore similar work

Jun 4, 2026cs.CV

Multimodal Sexism Identification and Characterization using Large Language Models and Gradient Boosting

We present the AILS-NTUA submission to the EXIST 2026 Lab at CLEF, addressing multimodal sexism identification and characterization in memes (Task 2) and short-form videos (Task 3). Our system follows a feature-engineered late-fusion pipeline built around gradient-boosted regression models and hierarchical post-processing. For memes, we combine visual, textual, demographic, biometric, and LLM-derived semantic indicators designed to capture high-level cues such as stereotyping, objectification, irony, and misogyny. For videos, we investigate the effect of feature selection, frame-based visual representations, OCR-based textual features, acoustic descriptors, and sensor-derived metadata. Development results show that focused LLM-derived semantic cues improve meme sexism identification, while video performance is highly sensitive to feature dimensionality and cross-modal noise. For videos, development results favor compact feature selection, but official test results show that this conclusion does not fully transfer to unseen data, where the unfiltered representation generalizes better. Overall, our findings highlight the usefulness of targeted semantic feature engineering for static memes and the need for more robust temporal modeling in noisy short-form video settings.

Kyriakos Chaviaras, Maria Lymperaiou, Athanasios Voulodimos

1Artificial Intelligence and Learning Systems Laboratory School of Electrical and Computer Engineering National Technical University of Athens

Hate Speech Detection Multimodal Classification

Ask PDF

Jul 20, 2026cs.CV

RMS@CC-MMD 2026: Multimodal Misogyny Detection via Geometric Interaction and Multi-View Consensus

The proliferation of internet memes has introduced new complexities to automated content moderation, particularly in detecting misogyny. Memes often rely on a semantic clash between visual and textual modalities, where hateful intent is implicit and culturally grounded. This paper presents GeoMVC (Geometric Interaction and Multi-View Consensus), developed for the CC-MMD Grand Challenge at ICMI 2026. To address the limitations of static feature concatenation, a Geometric Interaction Layer is proposed that models cross-modal alignment via Hadamard products and cosine similarity between frozen visual and textual embeddings. We further mitigate distribution shifts caused by noisy OCR and code-mixed transliteration through a Multi-View Consensus strategy, aggregating predictions across raw, length-filtered, and English-translated text views. The system achieved Rank 2 in the Malayalam partition (Macro F1: 0.892) and Rank 3 in the Chinese partition (Macro F1: 0.895) on Task A, while securing Rank 5 in the Tamil partition (Macro F1: 0.521). A detailed error analysis on the development partition highlights open challenges in modeling localized transliteration and code-mixed sarcasm across Dravidian and Chinese cultural contexts.

Md. Ajwad Hossain

Department of Electronics and Telecommunication Engineering Chittagong University of Engineering & Technology (CUET) Chattogram, Bangladesh

Cross-Modal Alignment Hate Speech Detection Multimodal Classification

Ask PDF

Jul 5, 2026cs.CL

AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes

We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder subtasks. We model them hierarchically as conditional soft-label prediction over empirical annotator distributions. Our system maps fixed Gemini Embedding 2 vision-language representations through a lightweight Gated MLP trained with KL divergence and homoscedastic uncertainty weighting. Our submissions ranked first on Task 2.3 and fourth on Tasks 2.1 and 2.2 on the official Soft-Soft leaderboards. The code is available at https://github.com/NLP-AI-Wizards/EXIST-2026

Matteo Fasulo, Antonio Gravina, Luca Tedeschini +1

Swiss Data Science Center, ETH Zürich, Andreasturm, Andreasstrasse 5, 8092 Zürich, Switzerland · Everest Systems GmbH, Max-Jarecki-Straße 21, 69115 Heidelberg, Germany · Villanova.ai S.P.A, Località Sa Illetta, SS 195 KM 2.3, 09123 Cagliari, Italy +1

Soft-Label Learning Hierarchical Classification Multimodal Classification

Ask PDF