Online memes are often carriers of unsafe content such as disinformation and propaganda. Their rapid evolution in response to news events and strong dependence on contextual information make it difficult for platforms and moderators to detect and moderate them effectively. Moreover, existing moderation studies typically classify memes as safe or unsafe based on individual images, lacking the ability to perform broader provenance analysis, i.e., where a meme originates, how it spreads, what cultural symbolism it represents, whether it poses potential risks to society, etc. To bridge this gap, we introduce MemeTrace, a multimodal LLM-based agent that automatically traces the provenance and potential risks of emerging memes. We evaluate MemeTrace on Meme100, a dataset of 100 real-world memes, using both human annotation and automatic metrics. The results demonstrate that MemeTrace substantially outperforms the VLM baseline. More importantly, MemeTrace achieves higher accuracy in detecting unsafe memes, especially those involving disinformation and sensitive content, and remains robust under visual perturbations.
Figures & tables
Figure 1: Meme: Donald Trump with a bruise overlaid with Jeffrey Epstein’s face, invoking conspiracy theories about their alleged associations.
Figure 2: Overview of the MemeTrace agent architecture, illustrating the multimodal perception, evidence retrieval, and confidence-routed provenance reasoning loop.
Figure 4: Examples of three perturbation types. The figure shows the original meme (top left) and its three perturbed variants.
Metric
System
about
str.
ori.
spread
cul.
usage
aud.
emo.
cog.
per.
safe.
Informativeness
Baseline
3.407
4.493
3.117
3.783
4.147
4.077
4.303
4.153
4.117
4.473
4.483
MemeTrace
4.917
4.927
4.533
4.670
4.883
4.907
4.907
4.867
4.920
4.913
4.777
Correctness
Baseline
3.863
4.580
3.227
3.930
4.350
4.307
4.463
4.297
4.330
4.513
4.400
MemeTrace
4.880
4.957
4.553
4.733
4.903
4.913
4.927
4.917
4.903
4.917
4.670
Table 1: Per-field human evaluation on Meme100, averaged across three annotators. Abbreviations: str.=structure, ori.=origin, cul.=culture, aud.=target audience, emo.=emotional impact, cog.=cognitive load, per.=persuasive technique.
System
Info.
Corr.
Adjusted
Baseline
4.050
4.205
4.128
MemeTrace
4.838
4.843
4.841
Table 2: Overall human evaluation comparison between MemeTrace and baseline on Meme100. Info.=Informativeness, Corr.=Correctness.
Metric
MemeTrace
Baseline
Δ
digits_per100w
2.088
0.631
+1.457
years_per100w
0.880
0.168
+0.712
dates_per100w
0.380
0.020
+0.360
ner_person_unique
3.220
1.960
+1.260
ner_org_unique
2.720
1.490
+1.230
ner_gpe_unique
1.830
1.290
+0.540
Table 3: Automatic metric comparison between MemeTrace and baseline on Meme100.
System/Dataset
Info.
Corr.
Adj.
Baseline
3.886
4.035
3.961
MemeTrace (full)
4.859
4.847
4.853
MemeTrace (Gemini)
4.452
4.450
4.451
MemeTrace (Qwen)
4.591
4.559
4.575
w/o LocalDB
4.394
4.382
4.388
w/o RIS
4.335
4.300
4.317
Table 4: Human evaluation on the 20-meme Ablation and Robustness subsets . Info.=Informativeness, Corr.=Correctness, Adj.=Adjusted Score.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Main Category
Fields
Description
Semantics
About
Describes what the meme is about, including background information and contextual analysis.
Structure
Details the meme’s visual and textual format, including layout, template, and constituent elements.
Spreading Dynamics
Origin
Specifies when, where, and how the meme likely originated.
Spread
Explains how the meme spread across platforms or online communities.
Cultural Context
Culture
Identifies the subcultures involved and the cultural symbolism the meme represents.
Usage
Describes the pragmatic or communicative use of the meme, such as sarcasm, trolling, or critique.
Appendix
Table 5: 11 fields in the meme provenance report.
Figure 5: Coverage statistics of MemeDB (confirmed KYM entries, as of Jun. 2025): entries by year, types, origins, and regions.
Figure 7: Overview of all 100 memes in the Meme100 dataset.
Figure 9: Alignment between human evaluation and automatic metrics: (left) Pearson r , (right) Spearman ρ .
System
Var(info)
Var(corr)
Var(overall)
Baseline
0.6191
0.8285
0.6025
MemeTrace (full)
0.1448
0.1630
0.1076
MemeTrace (Gemini)
0.8470
0.8833
0.8148
MemeTrace (Qwen)
0.6636
0.7530
0.6663
w/o LocalDB
0.9318
1.0197
0.8992
w/o RIS
1.2242
1.3348
1.2049
Appendix
Table 6: Inter-annotator agreement measured by mean rating variance (lower is better).
System
Var(overall)
ICC(overall)
Baseline
0.6025
0.6004
MemeTrace (full)
0.1076
0.5021
Appendix
Table 7: Inter-annotator agreement measured by mean rating variance and ICC.
System
Info.
Corr.
Adjusted
Baseline
3.877
4.050
3.964
WebSearch
4.607
4.582
4.594
MemeTrace
4.800
4.780
4.790
Appendix
Table 8: Overall blinded human evaluation on the 20-meme ablation subset, averaged over the two external annotators. WebSearch = GPT-4o with native web search. Info.=Informativeness, Corr.=Correctness.
System
about
structure
origin
spread
culture
usage
aud.
emo.
cog.
pers.
safety
Informativeness
Baseline
3.133
4.500
2.950
3.850
3.900
4.000
4.200
4.000
3.767
4.183
4.267
MemeTrace (full)
4.967
4.917
4.733
4.700
4.817
4.950
4.883
4.833
4.983
4.900
4.767
MemeTrace (Gemini)
4.950
4.417
4.650
4.167
4.467
4.400
4.683
4.083
4.433
4.200
4.517
MemeTrace (Qwen)
4.983
4.700
4.583
4.317
4.550
4.667
4.700
4.450
4.683
4.383
4.483
w/o LocalDB
4.783
4.450
4.450
4.117
4.367
4.300
4.417
4.217
4.467
4.250
4.517
Appendix
Table 9: Per-field human evaluation under module ablations, model variants, and robustness perturbations. Abbreviations: aud.=target audience, emo.=emotional impact, cog.=cognitive load, pers.=persuasive technique, Robu=Robustness, Gauss=Gaussian, Crop=Cropping.
Figure 10: Qualitative comparison on the “immigration/Florida–turnpike” meme.
Figure 11: Qualitative comparison on the “penguin–tariff in the Oval Office” meme.
Figure 12: Qualitative comparison under different system configurations for the “Last Supper” meme.
Figure 13: Safety-flag statistics.
Figure 14: Example execution log trace from MemeTrace (Gemini 1.5 Pro) on memeid = 2 . Some intermediate steps have been omitted for brevity.
Figure 15: Confidence evolution averaged over the main MemeTrace experiments ( n=100 ). The solid line represents the mean confidence, and the shaded region denotes ± 1 standard deviation.