UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes

Nazar Andrushko; Vera Schmitt; Veronika Solopova

cs.AIOct 9, 2026

UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes

Authors: Nazar Andrushko, Vera Schmitt, Veronika Solopova

Organizations: Ukrainian Catholic University, Lviv, Ukraine · Johannes Gutenberg University Mainz, Mainz, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany · Centre for European Research in Trusted AI (CERTAIN)

Abstract

Memes play a central role in modern online communication, especially in contexts of political conflict where visual and textual cues combine to convey persuasive messages. We study persuasive technique detection in Ukrainian memes using vision-language models (VLMs). We introduce UkrMeme, a dataset of 200 organically produced Ukrainian wartime memes annotated for persuasion techniques, and PTM-UA, a Ukrainian translation of a subset of the Propaganda 950 benchmark that preserves original visual content while translating meme text. We evaluate VLMs in a zero-shot prompting setup across image, text, and multimodal inputs using hierarchical F1 metrics. Finally, we compare Layer-wise Relevance Propagation (LRP) and model-generated token attributions and analyze the semantic categories of evidence emphasized by each explanation method.

Figures & tables

Figure 1: Example from the UkrMeme dataset. Gold labels: Smears , Loaded Language . en: "A monument to “Vladimir Lenin founding Ukraine” bronze, granite, Alzheimer’s"

Figure 2: Example from PTM-UA: original English meme (left) and its Ukrainian translation (right). Text is inpainted and re-rendered while preserving the visual content. Gold labels: Appeal to Authority , Smears .

MASI Krippendorff’s $\alpha$ : 0.406
Measure	Observed	Expected	Adjusted
Boot-match	0.764	0.303	0.661
Boot-precision	0.628	0.184	0.544
Boot-recall	0.698	0.227	0.610
Boot-F1	0.643	0.180	0.565
Augmented- $\kappa$	0.535	0.238	0.390

Table 1: Set-level inter-annotator agreement for UkrMeme .

Label	$\kappa$
No Propaganda	0.776
Smears	0.511
Appeal to Fear/Prejudice	0.506
Exaggeration/Minimisation	0.415
Causal Oversimplification	0.409
Glittering Generalities (Virtue)	0.394

Table 2: Per-class Cohen’s

\kappa

for UkrMeme labels with at least 5 occurrences from either annotator.

Figure 3: Label distribution for the UkrMeme dataset (200 samples).

Figure 4: Label distribution for the translated PTM-UA (158 samples).

			hF1
Dataset	Averaging	Mode	Qwen3-VL-4B	Qwen3-VL-8B	GPT-4.1-mini †	Gemma 3 4B †
PTM-UA	Micro	text	0.440	0.416	0.556	0.498
		image	0.483	0.524	0.592	0.549
		image+text	0.537	0.523	0.613	0.569
	Macro	text	0.195	0.198	0.255	0.255
		image	0.219	0.257	0.350	0.297

Table 3: hF1 comparison across all four models on PTM-UA (

n{=}158

) and UkrMeme (

n{=}200

). Best hF1 per averaging method is bolded within each dataset and averaging type. † GPT-4.1-mini and Gemma 3 4B are detection-only baselines; explainability analysis is not conducted for these models (see § 3.2.2 ).

Model	Mode	Orig. EN	Translated
		Mi / Ma hF1	Mi / Ma hF1
GPT-4.1-mini	text	0.548 / 0.270	0.556 / 0.255
	image	0.615 / 0.362	0.592 / 0.350
	image+text	0.603 / 0.347	0.612 / 0.356
Gemma 3 4B	text	0.495 / 0.257	0.498 / 0.255
	image	0.542 / 0.309	0.549 / 0.297

Table 4: Original English vs. Ukrainian-translated PTM-UA, hierarchical micro/macro F1, on the 158 retained samples. Translated-condition values for Qwen3-VL-4B/8B are taken from Table 3 ; Mi = micro hF1, Ma = macro hF1.

Figure 5: Confusion matrix for UkrMeme (Qwen3-VL-4B, image+text). The model massively over-predicts Loaded Language and Slogans , particularly for gold No Propaganda samples.

Figure 6: Confusion matrix for PTM-UA (Qwen3-VL-4B, image+text). Loaded Language acts as the dominant attractor class.

Panel of Figure 7 — Figure 7: Token attribution distributions across dataset settings. Columns correspond to PTM-UA (translated memes) and UkrMeme (native Ukrainian memes), with both multimodal (image+text) and text-only variants.

Figure 8: Agreement between VLM-based and LRP token attributions explanations for low-hF1 samples across dataset settings. The distribution shows moderate overlap across conditions, with higher median agreement for the UkrMeme multimodal setting.

	PTM-UA			UkrMeme
	Sh.	VLM	LRP	Sh.	VLM	LRP
Entity	22.9	15.8	19.4	20.5	13.0	16.9
Emotional	33.1	33.7	12.7	25.9	32.4	12.7
Rhetoric	17.1	22.7	19.4	15.5	17.6	15.9
Content	23.4	26.4	34.3	33.2	30.9	33.1
Other	3.6	1.1	14.2	4.9	6.2	21.3

Table 5: Category distribution (%) of attributed tokens by source. Total word counts shown as

n

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Averaging	Mode	hP	hR	hF1	hP	hR	hF1
			Qwen3-VL-4B			Qwen3-VL-8B
PTM-UA	Micro	text	0.538	0.372	0.440	0.492	0.360	0.416
		image	0.570	0.419	0.483	0.536	0.512	0.524
		image+text	0.569	0.508	0.537	0.519	0.526	0.523
	Macro	text	0.308	0.175	0.195	0.328	0.210	0.198
		image	0.302	0.249	0.219	0.334	0.295	0.257

Appendix

Table 6: Hierarchical F1 results on PTM-UA (

n{=}158

) and UkrMeme (

n{=}200

) for Qwen3-VL models. Best hF1 per averaging method is bolded within each model and dataset.

Averaging	Mode	w/ NP	w/o NP	$\Delta$	w/ NP	w/o NP	$\Delta$
		Qwen3-VL-4B			Qwen3-VL-8B
Micro	text	0.440	0.475	+.035	0.416	0.432	+.016
	image	0.483	0.527	+.044	0.524	0.515	$-$ .009
	image+text	0.537	0.550	+.013	0.523	0.529	+.006
Macro	text	0.195	0.214	+.019	0.198	0.211	+.013
	image	0.219	0.229	+.010	0.257	0.276	+.019

Appendix

Table 7: PTM-UA results with and without the No Propaganda label (

n{=}158

). The largest improvements appear in text-only mode, where No Propaganda is most frequently predicted.

Figure 9: Confusion matrix for PTM-UA without No Propaganda (Qwen3-VL-4B, image+text). Former abstentions are redistributed into partially correct technique labels.

Explore similar work

Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors
Date pendingGyuwon Park, Hyounghun KimVLM Evaluation VLM Reasoning
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH
MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes
Jul 30, 2026Weihang Wang, Kainan Tu, Jielei Zhang +9VLM Evaluation Vision-Language Models
1Bilibili · 2Fudan University · 3Peking University
Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding
Jun 2, 2026Zhengyi Zhao, Shubo Zhang, Zezhong Wang +6Multimodal Disentangled Representation Learning Representation Disentanglement
The Chinese University of Hong Kong · Huawei · Central China Normal University +3

Date pending, cs.CLJ/K move · Enter open · S save

Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

Gyuwon Park, Hyounghun Kim

Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH

VLM Evaluation VLM Reasoning Vision-Language Alignment

Open paper AskPDF arXiv

Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context. Despite recent progress, it remains unclear whether Vision-Language Models (VLMs) understand visual persuasiveness. This motivates us to ask: can VLMs assess whether an image persuasively supports an intended message, which visual factors shape this judgment, and do they align with human judgments? Through empirical analyses on image-message pairs where human raters consistently agree on the persuasiveness judgment, we show that VLMs exhibit a recall-oriented bias: they over-predict images as persuasive while achieving high recall. We introduce Visual Persuasive Factors (VPFs), a taxonomy informed by cognitive psychology for quantifying visual cues that shape persuasive judgments. Our factor-level analysis reveals that VPFs distinguish human persuasiveness judgments, whereas VLMs only partially reproduce these patterns, often generating false positives by treating persuasion-relevant cues as sufficient evidence. Building on this insight, we evaluate VPF-guided interventions and find that properly framed VPF knowledge can improve performance, but merely specifying visual cues or adding step-by-step reasoning is insufficient. By analyzing model rationales at the level of functional reasoning steps, we further identify a central bottleneck in connecting object identification to semantic message alignment.

Abstract

Explore similar work

Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes

Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding