MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detection

Arkadiusz Modzelewski; Witold Sosnowski; Eleni Papadopulos; Elisa Sartori; Tiziano Labruna; Giovanni Da San Martino; Adam Wierzbicki

cs.CLMar 15, 2026

MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detection

Authors: Arkadiusz Modzelewski, Witold Sosnowski, Eleni Papadopulos, Elisa Sartori, Tiziano Labruna, Giovanni Da San Martino, Adam Wierzbicki

Organizations: University of Padua, Italy · Polish-Japanese Academy of Information Technology, Poland · NASK National Research Institute, Poland · Politecnico di Torino, Italy

Abstract

The intentional creation and spread of disinformation poses a significant threat to public discourse. However, existing English datasets and research rarely address the intentionality behind the disinformation. This work presents MALINT, the first human-annotated English corpus developed in collaboration with expert fact-checkers to capture disinformation and its malicious intent. We utilize our novel corpus to benchmark 12 language models, including small language models (SLMs) such as BERT and large language models (LLMs) like Llama 3.3, on binary and multilabel intent classification tasks. Moreover, inspired by inoculation theory from psychology and communication studies, we investigate whether incorporating knowledge of malicious intent can improve disinformation detection. To this end, we propose intent-based inoculation, an intent-augmented reasoning for LLMs that integrates intent analysis to mitigate the persuasive impact of disinformation. Analysis on six disinformation datasets, five LLMs, and seven languages shows that intent-augmented reasoning improves zero-shot disinformation detection. To support research in intent-aware disinformation detection, we release the MALINT dataset with annotations from each annotation step.

Figures & tables

Statistic	Value
Total No. of Articles	1,600
Avg. Article Length (words)	963
Avg. Article Length (characters)	6,045

Table 1: Overview of the MALINT corpus.

Statistic	UCPI	UIOA	PASV	PSSA	CPV
Count	321	234	154	222	197
%	20.06	14.63	9.63	13.88	12.31

Table 2: Malicious intent types distribution in MALINT.

Model	UCPI	UIOA	PASV	PSSA	CPV
Small Language Models
BERT Base	0.562	0.484	0.500	0.614	0.293
BERT Large	0.528	0.437	0.543	0.529	0.306
DeBERTa V3 Base	0.675	0.505	0.580	0.523	0.400
DeBERTa V3 Large	0.696	0.649	0.683	0.547	0.460
RoBERTa Base	0.693	0.547	0.674	0.515	0.486

Table 3: LMs’ F 1 scores on binary intent classification across five categories, compared to random and BoW logistic regression baselines.

Model	Micro F 1	Weighted F 1
Small Language Models
BERT Base	0.421	0.414
BERT Large	0.578	0.521
DeBERTa V3 Base	0.812	0.804
DeBERTa V3 Large	0.817	0.815
RoBERTa Base	0.813	0.821

Table 4: Performance of LMs on multilabel intent classification compared to baselines: a random classifier and a one-vs-rest logistic regression using BoW approach.

Dataset	Disinformation	Credible
MALINT	30%	70%
ISOT Fake News	55%	45%
CoAID	21%	79%
EUDisinfo	33%	67%
ECTF	41%	59%
EUvsDisinfo	49%	51%

Table 5: Class distribution across test datasets. The percentages represent the original distribution of disinformation and credible content within each dataset.

Language	Base	IBI
German	0.794	$0.911$ $\uparrow$ 15%
Spanish	0.683	$0.828$ $\uparrow$ 21%
Estonian	0.716	$0.892$ $\uparrow$ 25%
French	0.611	$0.749$ $\uparrow$ 23%
Polish	0.709	$0.846$ $\uparrow$ 19%
Russian	0.619	$0.735$ $\uparrow$ 19%

Table 6: Comparison of Base vs IBI performance across languages. Results present averaged F 1 scores over all models and methods. See Appendix I for the table with standard deviations.

	GPT 4o Mini		GPT 4.1 Mini		Gemini 2.0 Flash		Gemma 3 27b it		Llama 3.3 70B
	Base	IBI	Base	IBI	Base	IBI	Base	IBI	Base	IBI
VaN	0.815	$0.856$ $\uparrow$ 5% 0.856	0.825	$0.873$ $\uparrow$ 6%	0.789	$0.855$ $\uparrow$ 8%	0.783	$0.820$ $\uparrow$ 5%	0.836	$0.863$ $\uparrow$ 3%
Z-CoT	0.836	$0.849$ $\uparrow$ 2%	0.810	$0.861$ $\uparrow$ 6%	0.751	$0.837$ $\uparrow$ 11%	0.782	$0.806$ $\uparrow$ 3%	0.807	$0.865$ $\uparrow$ 7%
DeF_Spec	0.887	$0.877$ $\downarrow$ 1%	0.870	$0.879$ $\uparrow$ 1%	0.843	$0.881$ $\uparrow$ 5%	0.812	$0.846$ $\uparrow$ 4%	0.806	$0.871$ $\uparrow$ 8%

Table 7: F 1 scores on MALINT for competitive prompting methods and their improvement with IBI.

GPT 4o Mini
	Overall		Articles		Posts		Prior Cutoff		Post Cutoff
	Base	IBI	Base	IBI	Base	IBI	Base	IBI	Base	IBI
VaN	0.736	$0.828$ $\uparrow$ 13%	0.754	$0.862$ $\uparrow$ 14%	0.703	$0.755$ $\uparrow$ 7%	0.727	$0.821$ $\uparrow$ 13%	0.762	$0.846$ $\uparrow$ 11%
Z-CoT	0.740	$0.826$ $\uparrow$ 12%	0.764	$0.854$ $\uparrow$ 12%	0.692	$0.766$ $\uparrow$ 11%	0.724	$0.823$ $\uparrow$ 14%	0.786	$0.833$ $\uparrow$ 6%
DeF-SpeC	0.746	$0.792$ $\uparrow$ 6%	0.782	$0.817$ $\uparrow$ 4%	0.682	$0.742$ $\uparrow$ 9%	0.712	$0.771$ $\uparrow$ 8%	0.843	$0.850$ $\uparrow$ 1%
GPT 4.1 Mini

Table 8: Results with F 1 scores for five LLMs. The Base columns shows the competitive method results, while the IBI columns presents results for prompts adapted to the Intent-based Inoculation.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Identifier	Learning Rate	Weight Decay	Warmup Ratio
Undermining the Credibility of Public Institutions (UCPI)
BERT-base	google-bert/bert-base-uncased	2e-5	0.03	0.06
BERT-large	google-bert/bert-large-uncased	1e-5	0.05	0.06
RoBERTa-large	FacebookAI/roberta-large	2e-5	0.05	0.06
RoBERTa-base	FacebookAI/roberta-base	1e-5	0.05	0.1
DeBERTa-v3-large	microsoft/deberta-v3-large	1e-5	0.01	0.06

Appendix

Table 9: Optimal hyperparameters for binary classification of all malicious intent categories.

Model	Identifier	Learning Rate	Weight Decay	Warmup Ratio
BERT-base	google-bert/bert-base-uncased	2e-5	0.1	0.06
BERT-large	google-bert/bert-large-uncased	2e-5	0.01	0.1
RoBERTa-large	FacebookAI/roberta-large	1e-5	0.03	0.06
RoBERTa-base	FacebookAI/roberta-base	1e-5	0.05	0.1
DeBERTa-v3-large	microsoft/deberta-v3-large	1e-5	0.03	0.1
DeBERTa-v3-base	microsoft/deberta-v3-base	4e-5	0.05	0.1

Appendix

Table 10: Optimal hyperparameters for each model in multilabel malicious intent classification.

API Model Name	Knowledge Cutoff Date	Access Details	License	Model Size
gpt-4o-mini	October 2023	OpenAI API 07.2025	Commercial	Not Disclosed
gemini-2.0-flash	June 2024	Google API 07.2025	Commercial	Not Disclosed
gpt-4.1-mini-2025-04-14	June 2024	OpenAI API 07.2025	Commercial	Not Disclosed
meta-llama/Llama-3.3-70B-Instruct	December 2023	DeepInfra API 07.2025	Meta Llama 3 Community	70B
google/gemma-3-27b-it	August 2024	DeepInfra API 07.2025	Gemma Terms of Use	27B

Appendix

Table 11: Large Language Models used in our experiments.

Panel of Figure 2 — Figure 2: Prompt template used for binary classification of malicious intent categories with LLMs. In each instance, placeholders <Here name of the malicious intent category> and <[shortcut]> were replaced with one of the following categories and their respective abbreviations: Undermining the Credibility of Public Institutions [UCPI], Changing Political Views [CPV], Undermining International Organizations and Alliances [UIOA], Promoting Social Stereotypes/Antagonisms [PSSA], and Promoting Anti-scientific Views [PASV].

Panel of Figure 3 — Figure 3: Prompt used for multilabel classification of malicious intent with LLMs. The system is instructed to detect five predefined categories of malicious intent within a given text. The model evaluates all categories simultaneously and returns a dictionary of binary Yes / No decisions for each. The prompt emphasizes a conservative decision-making policy: the model is instructed to respond Yes only when confident.

Figure 4: The prompt template for each baseline method in disinformation detection, namely, VaN , Z-CoT , and DeF-SpeC . Each baseline method differs in the Baseline Specific Instructions block. Generally, it provides method-specific guidelines defining the task and requests for structured output. The text $T$ represents the content passed for disinformation evaluation.

Figure 5: The prompt template for first stage of IBI experiment, namely for intent analysis. The component $K_{I}$ encapsulates knowledge about a predefined set of malicious intent categories. Guidelines $G_{A}$ determine the task and specify the structure of the expected response. Finally, the text $T$ represents the content passed for intent analysis.

Figure 6: The prompt template for second final stage of IBI experiments, namely for disinformation detection step. The component $\theta$ provides threat against malicious intents and gives some task details. Next component is the generated analysis $A_{I}(T)$ from the output of first step of IBI experiment and finally, the text $T$ represents the content passed for disinformation evaluation. $G_{I}$ fully determine the task and specify the structure of the expected response. The Baseline Specific Instructions block is a part of guidelines and includes different instructions depending on which baseline method was adapted to IBI experiment, namely it can be instruction from VaN , Z-CoT , or DeF-SpeC

Method	Split	Gemini 2.0 Flash	GPT 4.1 Mini	GPT 4o Mini	Llama3.3 70B	Gemma 3 27B
VaN	Overall	0.010	0.010	0.010	0.050	0.010
VaN	News Article	0.010	0.010	0.010	0.010	0.010
VaN	Twitter Post	N.S.	0.010	0.010	N.S.	N.S.
VaN	Prior Cutoff	0.010	0.010	0.010	N.S.	0.010
VaN	Post Cutoff	0.010	0.010	0.010	0.050	N.S.
Z-CoT	Overall	0.010	0.010	0.010	0.010	0.010

Appendix

Table 12: Results of McNemar’s test, comparing each prompting method ( VaN , Z-CoT , and DeF-Spec ) against its IBI-adjusted counterpart across various language models. The values represent significance levels for different evaluation metrics, with N.S as Non-Significant indicating no statistically significant difference at the 0.05 threshold.

CoAID
	GPT 4o Mini		GPT 4.1 Mini		Gemini 2.0 Flash		Gemma 3 27b it		Llama 3.3 70B
	Base	IBI	Base	IBI	Base	IBI	Base	IBI	Base	IBI
VaN	0.531	0.627	0.480	0.599	0.631	0.607	0.618	0.650	0.654	0.680
Z-CoT	0.532	0.628	0.468	0.611	0.588	0.603	0.388	0.660	0.628	0.699
DeF_Spec	0.507	0.566	0.496	0.624	0.574	0.610	0.617	0.651	0.582	0.646
ECTF

Appendix

Table 13: F 1 scores on four disinformation datasets for competitive prompting methods and their enhancement with Intent-Based Inoculation.

Model	CPV		PSSA		UIOA		PASV		UCPI
Model	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
gpt-4o-mini	0.914	0.787	0.882	0.809	0.860	0.864	0.866	0.850	0.905	0.796
Llama-3.3-70B-Instruct	0.889	0.825	0.876	0.839	0.856	0.899	0.862	0.888	0.836	0.941
gpt-4.1-mini	0.872	0.869	0.886	0.831	0.862	0.908	0.862	0.905	0.876	0.863
gemini-2.0-flash	0.878	0.821	0.855	0.866	0.845	0.902	0.854	0.871	0.865	0.842
google/gemma-3-27b-it	0.853	0.806	0.834	0.799	0.824	0.824	0.815	0.862	0.876	0.722

Appendix

Table 14: Average F1 scores for disinformation detection across three methods (VaN, Z-CoT, DeF-Spec), split by correct and incorrect intent prediction for each intent.

	CPV		PSSA		UIOA		PASV		UCPI
Model	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
gpt-4o-mini	0.910	0.783	0.883	0.792	0.852	0.867	0.860	0.848	0.903	0.788
Llama-3.3-70B	0.881	0.829	0.870	0.843	0.855	0.889	0.857	0.889	0.834	0.933
gpt-4.1-mini	0.870	0.879	0.882	0.850	0.864	0.912	0.861	0.921	0.871	0.877
gemini-2.0-flash	0.869	0.830	0.849	0.871	0.843	0.896	0.856	0.853	0.871	0.821
gemma-3-27b-it	0.853	0.800	0.831	0.792	0.828	0.800	0.812	0.857	0.879	0.707

Appendix

Table 15: F 1 scores with VaN + IBI, split by intent prediction correctness.

	CPV		PSSA		UIOA		PASV		UCPI
Model	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
gpt-4o-mini	0.911	0.766	0.872	0.796	0.850	0.847	0.857	0.832	0.913	0.759
Llama-3.3-70B	0.891	0.818	0.873	0.843	0.854	0.904	0.860	0.889	0.833	0.945
gpt-4.1-mini	0.862	0.860	0.874	0.825	0.844	0.931	0.854	0.889	0.865	0.855
gemini-2.0-flash	0.847	0.819	0.833	0.847	0.825	0.879	0.828	0.866	0.840	0.830
gemma-3-27b-it	0.837	0.786	0.818	0.774	0.805	0.809	0.794	0.857	0.858	0.707

Appendix

Table 16: F 1 scores with Z-CoT + IBI, split by intent prediction correctness.

	CPV		PSSA		UIOA		PASV		UCPI
Model	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
gpt-4o-mini	0.921	0.812	0.892	0.839	0.876	0.878	0.880	0.869	0.899	0.841
Llama-3.3-70B	0.894	0.829	0.886	0.829	0.861	0.904	0.868	0.885	0.840	0.945
gpt-4.1-mini	0.885	0.867	0.901	0.819	0.878	0.881	0.871	0.906	0.892	0.857
gemini-2.0-flash	0.918	0.814	0.882	0.879	0.866	0.932	0.877	0.896	0.885	0.874
gemma-3-27b-it	0.868	0.832	0.852	0.830	0.840	0.864	0.840	0.871	0.892	0.752

Appendix

Table 17: F 1 scores with DeF-Spec + IBI, split by intent prediction correctness.

Language	Base	IBI
German	0.794 $\pm$ 0.06	0.911 $\pm$ 0.02
Spanish	0.683 $\pm$ 0.08	0.828 $\pm$ 0.04
Estonian	0.716 $\pm$ 0.11	0.892 $\pm$ 0.04
French	0.611 $\pm$ 0.06	0.749 $\pm$ 0.05
Polish	0.709 $\pm$ 0.1	0.846 $\pm$ 0.05
Russian	0.619 $\pm$ 0.07	0.735 $\pm$ 0.02

Appendix

Table 18: Comparison of Base vs IBI performance across languages. Results present averaged F 1 scores together with standard deviations over all models and methods.

GPT 4o Mini
	Spanish		German		Polish
	Base	IBI	Base	IBI	Base	IBI
VaN	0.715	$0.868$ $\uparrow$ 21%	0.806	$0.925$ $\uparrow$ 15%	0.734	$0.842$ $\uparrow$ 15%
Z-CoT	0.710	$0.877$ $\uparrow$ 24%	0.813	$0.932$ $\uparrow$ 15%	0.727	$0.840$ $\uparrow$ 16%
DeF-Spec	0.742	$0.825$ $\uparrow$ 11%	0.850	$0.913$ $\uparrow$ 7%	0.696	$0.775$ $\uparrow$ 11%
Llama 3.3 70B

Appendix

Table 19: F 1 scores on the EUvsDisinfo dataset separated by six languages computed on about 500 test texts per language. Results report F 1 for competitive prompting methods as the baseline ( Base ) and their enhancement with intent-based reasoning ( IBI ).

Paper	Dataset	Intent Categories	Fully Annotated	Language	Algorithmic Annotation	Annotation from Each Step
Zhou et al. (2022)	Enhanced from Existing	Intentional / Unintentional	No	English	Yes	No
Modzelewski et al. (2024)	Original	9 categories focused on Polish disinformation	Yes	Polish	Fully annotated by humans	No
Wang et al. (2024a)	No dataset presented	Hierarchy of expressed intents of articles (lowest-level labels: Popularize, Clout, Smear, Conflict, Connect)	No	English	Yes	No
Our MALINT	Original	6 categories focused on current global disinformation	Yes	English	Fully annotated by humans	Yes

Appendix

Table 20: Comparison of datasets related to MALINT, highlighting the novelty of the MALINT dataset

Fake News Detection

Fake news detection identifies false or misleading claims by analyzing content, context, evidence, and how information spreads.

See all 98 papers

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts
Aug 10, 2026Kevin Thomas, Milosz Kasprzyk, Reuel C Igbokwe Onuigbo +5Fake News Detection AI-Generated Text Detection
University of Sheffield, Department of Computer Science, Regent Court, 211 Portobello Street, Sheffield, S1 4DP, United Kingdom
Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats
Jun 29, 2026Sebastian Kula, Martin TamajkaMulti-Agent LLM Systems Fake News Detection
Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · West Pomeranian University of Technology in Szczecin, Szczecin, Poland
To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs
Apr 8, 2026Zohaib Khan, Mustafa Dogan, Ifeoma Okoh +6Deception in Language Models Language Model Safety Evaluation
Fatima Fellowship · Stanford University · Georgia Institute of Technology

Aug 10, 2026J/K move · Enter open · S save

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts

Kevin Thomas, Milosz Kasprzyk, Reuel C Igbokwe Onuigbo, Elliott Pert+4

University of Sheffield, Department of Computer Science, Regent Court, 211 Portobello Street, Sheffield, S1 4DP, United Kingdom

Fake News Detection AI-Generated Text Detection Adversarial Robustness of Language Models Robustness of Machine-Generated Text Detection Adversarial Text Attacks

Open paper AskPDF arXiv

Detecting machine-generated disinformation on social media is increasingly difficult as large language models (LLMs) make it easier to generate and rewrite misleading content at scale. Static benchmark evaluations, measuring detector performance on fixed held-out datasets, do not capture how detectors behave when posts are deliberately transformed to evade classification. This paper adapts the Build it, Break it, Fix it framework into Build it, Break it, Repeat (BiBiR): iterative sessions designed to stress-test detectors' robustness under iterative adversarial conditions, evaluating whether models remain reliable when disinformation posts are systematically transformed to evade classification. Across five iterations, the findings show that the best adversarial breakers' transformations came from a combination of back-translation and LLM persona-based rewriting, with the best performing technique achieving a 95% label flip rate (LFR), whilst still preserving the meaning of the original posts. The best builders' model was a triplet contrastive model with a dynamic anchor switching (DASS) architecture, which achieved an average accuracy of 72.68%, outperforming the strong baseline (a fine-tuned e5-small-LoRA) by 15 percentage points on the most robust set of breakers' adversarial attacks. The results demonstrate that an iterative framework best exposes detector weaknesses and pushes robustness improvements; however, it may still require semantic preservation analysis to distinguish valid adversarial evasion from transformations that changed the original disinformation claims' meaning.

Abstract

Fake News Detection

Explore similar work

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts

Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats

To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs