Reasoning for Propaganda Detection via Label-Guided CoT Optimisation

Zeba Afroz; Harsh Vardhan; Md. Shad Akhtar

cs.AIOct 12, 2026

Reasoning for Propaganda Detection via Label-Guided CoT Optimisation

Authors: Zeba Afroz, Harsh Vardhan, Md. Shad Akhtar

Organizations: Indraprastha Institute of Information Technology Delhi

Abstract

Propaganda detection systems usually operate as black boxes, lacking explanations. To address this, we hypothesise that persuasive techniques contain definitional characteristics with respect to their social-political setting. We introduce ReCoT, a reasoning-conditioned detection framework that captures definitional cues of persuasive techniques via population-level refinement and performance-guided resampling, without requiring additional explanation annotations. On INDI-PROPspan, ReCoT achieves a macro F1_LCS of 0.364, surpassing baselines and reaching 88% of the SFT upper bound on Avg F1_LCS. Similarly, on PTC, ReCoT reaches 85% of the SFT upper bound on Avg F1_LCS, closing much of this gap while remaining interpretable. Importantly, the analysis of the reasoning pool's convergence and update pattern for ReCoT shows effective refinement of reasoning CoTs.

Figures & tables

Figure 1: Conceptual comparison of Post-Hoc vs. Intrinsic Explanation frameworks. In the Post-Hoc setup (top), the Persuasive Span influences the LLM’s CoT Reasoning. In the Intrinsic setup (bottom), the LLM generates CoT Reasoning directly from the input news, which then identifies the relevant Persuasive Span.

ID	Technique	Train	Test	Total
T1	Causal Oversimplification	99	21	120
T2	Black-and-white Fallacy	28	4	32
T3	Straw Man	55	17	72
T4	Whataboutism	32	10	42
T5	Reductio ad Hitlerum	22	6	28
T6	Red Herring	31	13	44

Table 1: Frequency distribution of propaganda techniques in INDI-PROPspan .

Figure 2: Overview of the proposed ReCoT framework. Block 1: illustrates the initialisation stage, where initial CoT-predictions pairs are generated, Block 2: presents the sampling process & Block 3: shows the iterative population-based refinement process via resampled in-context learning.

Figure 3: ReCoT ’s refinement and update density convergence curve across

25

iterations for i-bias & k=2 configuration on the INDI-PROPspan dataset

Config	Backbone	IndiProp		PTC
Config	Backbone	Avg	W.Avg	Avg	W.Avg
SFT (Upper Bound)	Mistral	0.412	0.538	0.235	0.266
Hitachi	—	0.180	0.232	0.093	0.118
Aschern	—	0.205	0.231	0.174	0.203
0-shot	Mistral	0.263	0.273	0.187	0.184
1-shot		0.318	0.317	0.200	0.192

Table 2:

F1_{LCS}

Performance comparison on INDI-PROPspan and PTC for i-bias & K=2 setup .

Setup	R	Mistral				Qwen
Setup	R	IndiProp		PTC		IndiProp		PTC
		Avg	W.Avg	Avg	W.Avg	Avg	W.Avg	Avg	W.Avg
$K=1$	R=1	0.323	0.336	0.171	0.185	0.323	0.339	0.184	0.201
	R=2	0.340	0.337	0.171	0.185	0.313	0.344	0.190	0.205
	R=3	0.354	0.360	0.182	0.196	0.321	0.358	0.187	0.199
	R=4	0.334	0.345	0.193	0.205	0.341	0.352	0.177	0.192

Table 3: ReCoT Performance across different values of

K

using

i_{bias}

strategy

Config	Backbone	IndiProp		PTC
Config	Backbone	Avg	W.Avg	Avg	W.Avg
SFT (Upper Bound)	Mistral	0.210	0.214	0.127	0.195
Hitachi	—	0.161	0.207	0.092	0.143
Aschern	—	0.152	0.193	0.089	0.141
GEPA	Mistral	0.247	0.256	0.158	0.185
0-shot	Mistral	0.201	0.213	0.111	0.139

Table 4: Performance comparison on the SemEval metrics - Avg F1 and Weighted Average (W.Avg) F1 are reported for the IndiProp and PTC tasks.

Category of error	Remark / Error Analysis
GT: [ORG] 4 4 4 The names of the organisation and policy are intentionally omitted to avoid explicit political content in the paper. was misleading people by saying that [POLICY] is against minorities Pred: [ORG] was misleading people by saying that [POLICY] is against minorities, and asked them to show proof for that claim [Over-prediction]	Model added extra explanatory clause (“asked them to show proof”), extending beyond the gold boundary, another over-prediction error.
GT: The [ORG] is targeting a particular community by carrying out the [POLICY] in North and South 24 Pargana districts [Under-prediction] Pred: targeting a particular community	Model extracted only the core phrase, ignoring key qualifiers (“[ORG]”, “[POLICY]”), leading to a truncated reasoning span.
GT: No Propaganda span present. Pred: what they did was ‘unlawful’ and ‘not legally correct’ [False Positive]	Model predicted a span even though no persuasive content was annotated. False trigger due to lexical cues like “unlawful” and “not legally correct”.
GT: They won’t be able to give citizenship to anyone in two days. This is just a lollipop and a show-off. [False Negative] Pred: No Propaganda span present.	Model failed to detect explicit ridicule and exaggeration, missing emotionally charged language recall failure on loaded or sarcastic tone.

Table 5: Representative examples of common error categories in span-level propaganda detection, including over-prediction, under-prediction, false positives, and false negatives.

Iter	Human Evaluation									Expert Evaluation
	CR			RT			OC			TP			LS
	poor	fair	good	poor	fair	good	poor	fair	good	poor	fair	good	poor	fair	good
0	41	45	15	24	55	21	26	49	25	60	40	0	53	47	0
25	3	30	67	1	14	85	4	23	74	0	17	83	0	33	67

Table 6: Human and expert evaluation for assessing reasoning quality(in % values)

R	PTC		IndiProp
R	Avg F1	W. Avg F1	Avg F1	W. Avg F1
1	0.188	0.196	0.286	0.270
2	0.209	0.207	0.317	0.295
3	0.185	0.189	0.327	0.299
4	0.194	0.190	0.315	0.286
5	0.224	0.205	0.306	0.279

Table 7: Cross-dataset inference performance for the Mistral (K2) configuration. PTC was evaluated using CoT refined on IndiProp, while IndiProp was evaluated using CoT refined on PTC.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: ReCoT ’s refinement and update density convergence curve across

25

iterations for i-bias & k=2 configuration on the PTC dataset

Figure 5: Inter-bias comparison of technique wise inference results across

R

values for

K=2

on Mistral backbone

Phase	Configuration	Mistral (s)	Qwen (s)
Inference	$R=3$	6.05	11.70
Refinement	$K=1$	2.99	5.63
	$K=2$	4.70	9.01
	$K=3$	6.12	12.00

Appendix

Table 8: Inference and refinement time (in seconds) for Mistral and Qwen.

Config	Backbone	IndiProp		PTC
Config	Backbone	Avg	W.Avg	Avg	W.Avg
0-shot	GPT	0.231	0.212	0.114	0.170
1-shot		0.176	0.201	0.170	0.203
CoT		0.241	0.244	0.152	0.194
RePrompt	GPT	0.106	0.158	0.140	0.184
ReCoT R1		0.229	0.246	0.100	0.141

Appendix

Table 9: Performance comparison of the GPT backbone on the SemEval benchmark. Average F1 (Avg) and Weighted Average F1 (W.Avg) are reported for the IndiProp and PTC tasks.

Technique	R1	R2	R3	R4	R5
Causal Oversimplification	0.308	0.291	0.258	0.208	0.256
Black-and-white Fallacy	0.347	0.384	0.327	0.371	0.356
Straw Man	0.280	0.183	0.269	0.160	0.291
Whataboutism	0.309	0.302	0.375	0.331	0.340
Reductio ad Hitlerum	0.299	0.577	0.625	0.465	0.625
Red Herring	0.346	0.483	0.377	0.523	0.448

Appendix

Table 10: IndiProp per-technique F1 scores across refinement rounds (Mistral - K2(i-bias)).

Setup	R	Mistral				Qwen
Setup	R	IndiProp		PTC		IndiProp		PTC
		Avg	W.Avg	Avg	W.Avg	Avg	W.Avg	Avg	W.Avg
I - BIAS	R=1	0.320	0.311	0.167	0.180	0.296	0.326	0.172	0.191
	R=2	0.343	0.300	0.166	0.182	0.316	0.348	0.181	0.196
	R=3	0.364	0.347	0.155	0.172	0.345	0.362	0.184	0.198
	R=4	0.344	0.317	0.172	0.170	0.326	0.356	0.200	0.211

Appendix

Table 11: ReCoT performance across bias strategies with

K=2

configuration.

Technique	R1	R2	R3	R4	R5
Appeal to Authority	0.128	0.137	0.147	0.153	0.128
Appeal to fear/prejudice	0.176	0.211	0.173	0.195	0.202
Bandwagon, Reductio ad Hitlerum	0.253	0.167	0.167	0.167	0.167
Black-and-white Fallacy	0.047	0.106	0.089	0.059	0.084
Causal Oversimplification	0.182	0.157	0.111	0.224	0.197
Doubt	0.142	0.194	0.189	0.262	0.218

Appendix

Table 12: PTC per-technique F1 scores across refinement rounds (Mistral - k2(i-bias)).

Setup	Backbone	Config	IndiProp						PTC
			$R_{\text{LCS}}$		$P_{\text{LCS}}$		$F1_{\text{LCS}}$		$R_{\text{LCS}}$		$P_{\text{LCS}}$		$F1_{\text{LCS}}$
			Avg	W.Avg	Avg	W.Avg	Avg	W.Avg	Avg	W.Avg	Avg	W.Avg	Avg	W.Avg
SFT	Hitachi	–	0.381	0.438	0.150	0.201	0.180	0.232	0.179	0.217	0.084	0.108	0.093	0.118
SFT	Aschern	–	0.311	0.345	0.203	0.236	0.205	0.231	0.248	0.296	0.206	0.219	0.174	0.203
Prompting	Mistral	0-shot	0.270	0.276	0.319	0.345	0.263	0.273	0.259	0.243	0.199	0.211	0.187	0.184
		1-shot	0.365	0.346	0.340	0.365	0.318	0.317	0.244	0.215	0.214	0.225	0.200	0.192

Appendix

Table 13:

F1_{LCS}

R_{LCS}

, and

P_{LCS}

for ReCoT with

I_{bias}

K\in\{1,2,3\}

across Mistral, Qwen, and GPT on INDI-PROPspan and PTC

Variant	Evaluation Details
Initialization ( $n=0$ )	<CoT> Step 1: Identify the main arguments presented in the article. The article discusses the contrasting actions of citizens during a public appreciation event for frontline workers and the protests against CAA, NRC, and NPR led by Suranya Aiyar. Step 2: Look for instances where an opposing viewpoint is misrepresented. The article portrays Suranya Aiyar’s protests as being out of touch or inappropriate given the context of a pandemic and the collective appreciation of healthcare workers, suggesting that those who protest are disregarding the situation at hand. Step 3: Analyze specific statements that may reduce the opposing viewpoint to an unfairly simple or easily attackable position. The article claims "partisan political interests even in the face of an apocalypse." This implies that any dissent against the government’s practices during a crisis is wrong or irrelevant, which misrepresents the nuanced argument that some may hold against these policies, portraying them as entirely neglectful of the situation. Step 4: The statement about residents of Jamia Nagar raising slogans while others rallied for healthcare workers further simplifies their stance to mere opposition, not recognizing the validity of their protest or their right to express it. </CoT> Extracted Span(s): 1. “some people would not abandon their partisan political interests even in the face of an apocalypse.” 2. “On a similar note, when the nation demonstrated their appreciation towards those providing people with healthcare in times of this major crisis, residents of Jamia Nagar’s Abul Fazal colony raised ‘Azaadi’ slogans instead.” F1 Score: 0.0841
I-Bias ( $n=25$ )	<CoT> Step 1: The article describes the actions of Suranya Aiyar and others during a nationwide “Janta Curfew” to show appreciation for frontline workers battling the pandemic. Step 2: Suranya Aiyar is portrayed as screaming slogans demanding the rollback of CAA, NRC, and NPR, which are contentious issues in India. Step 3: The article criticizes her actions by stating that she “riled against the people as the nation came together as one,” implying that her actions were disruptive and inappropriate during a time of national unity. Step 4: To identify Straw Man propaganda, we need to check if the article misrepresents or exaggerates Suranya Aiyar’s actions or intentions. The statement “riled against the people as the nation came together as one” could be interpreted as a Straw Man if it attributes an extreme or simplistic viewpoint to her actions that may not be accurate. Conclusion: Propaganda spans exist </CoT> <span>riled against the people as the nation came together as one</span> Extracted Span(s): “riled against the people as the nation came together as one” F1 Score: 0.7060
J-Bias ( $n=25$ )	<CoT> Step 1: To determine if there are any “Straw Man” arguments in the article, I need to identify whether the article misrepresents or oversimplifies the position of any opposing side or group. A Straw Man argument often involves refuting a weaker version of the actual argument presented by the opposing side. Step 2: Analyzing the article, it describes the actions of Suranya Aiyar and other individuals who raised slogans during the ‘Janta Curfew’. The article portrays these actions as disruptive and partisan, contrasting them with the nationwide effort to show appreciation for frontline workers battling the pandemic. Step 3: I look for any indication that the article misrepresents the motivations or arguments of Suranya Aiyar or the other individuals mentioned. The article states that Suranya Aiyar and others “riled against the people as the nation came together as one,” and that their actions demonstrate a refusal to abandon “partisan political interests even in the face of an apocalypse.” This could be seen as a misrepresentation or oversimplification of their actions, as it frames their political statements as inherently disruptive and selfish, without exploring their underlying motivations or the context of their slogans. Conclusion: Propaganda spans exist. </CoT> <span>riled against the people as the nation came together as one</span> <span>some people would not abandon their partisan political interests even in the face of an apocalypse</span> Extracted Span(s): 1. “riled against the people as the nation came together as one” 2. “some people would not abandon their partisan political interests even in the face of an apocalypse” F1 Score: 0.5500
Random ( $n=25$ )	<CoT> Step 1: To identify if there are any statements that qualify as straw man propaganda, I need to check if the article misrepresents an opponent’s position in order to make it easier to attack or refute. Step 2: The article discusses Suranya Aiyar’s actions and slogans during the ‘Janta Curfew’. It mentions her demands for the rollback of CAA, NRC, and NPR, and her slogans in support of various causes. Step 3: The article states, “Suranya Aiyar was walking around her lawn presumably while she raised these slogans at 5 pm on Sunday.” This could be interpreted as a misrepresentation if the context or intent behind her actions is not accurately portrayed. Additionally, the phrase “some people would not abandon their partisan political interests even in the face of an apocalypse” could be seen as a straw man argument if it oversimplifies or misrepresents the motivations of those opposing CAA, NRC, and NPR. Conclusion: Propaganda spans exist. </CoT> <span>some people would not abandon their partisan political interests even in the face of an apocalypse.</span> Extracted Span(s): “some people would not abandon their partisan political interests even in the face of an apocalypse.” F1 Score: 0.1000

Appendix

Table 14: Propaganda Span and Reasoning (CoT) Results

Panel of Figure 6 — Figure 6: Prompts used in our framework: (top) iterative refinement prompt, (bottom) inference prompt.

Explore similar work

Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas
May 13, 2026Lukas Stähelin, Veronika Solopova, Max Upravitelev +6LLM Prompting LLM Fine-Tuning
Technische Universität Berlin, QU Lab, XplaiNLP Group, Berlin, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany · Centre for European Research in Trusted AI (CERTAIN)
ProBel: Propaganda Detection with Techniques, Spans, and Explanations
Aug 23, 2026Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori +2Text Classification Cross-Lingual Transfer
Qatar Computing Research Institute, Qatar · University of Padova, Italy
ExTax: Explainable Disinformation Detection via Persuasion, Emotion, and Narrative Role Taxonomies
May 26, 2026Shang Luo, Yingguang Yang, Zhenchen Sun +8Fake News Detection Explainable Artificial Intelligence
Peking University · University of Science and Technology of China · North China University of Science and Technology +6

May 13, 2026J/K move · Enter open · S save

Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas

Lukas Stähelin, Veronika Solopova, Max Upravitelev, David Kaplan+5

Technische Universität Berlin, QU Lab, XplaiNLP Group, Berlin, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany · Centre for European Research in Trusted AI (CERTAIN)

LLM Prompting LLM Fine-Tuning Language Model Robustness

Open paper AskPDF arXiv

Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements. We introduce a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema. Along three dimensions (model portfolio, schema effects, and prompting strategy) we evaluate the taxonomies as a classification task with the help of four language models (GPT-4.1-nano, Phi-4 14B, Qwen2.5-14B, Qwen3-14B). Our results show that fine-tuning is essential, since it transforms weak zero-shot baselines into competitive systems and reveals methodological differences that are hidden using base models. Across schemas, the Qwen models achieve the strongest overall performance, and Phi-4 14B consistently outperforms GPT-4.1-nano. Our hierarchical prompting method (HiPP), which predicts fine-grained techniques before aggregating them, is especially beneficial after fine-tuning and on the more ambiguous, low-agreement taxonomy, while remaining competitive on the simpler schema. The HQP dataset, annotated with the new intent-based labels, provides a richer lens on propaganda's strategic goals and a challenging benchmark for future work on robust, real-world detection.

Abstract

Explore similar work

Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas

ProBel: Propaganda Detection with Techniques, Spans, and Explanations

ExTax: Explainable Disinformation Detection via Persuasion, Emotion, and Narrative Role Taxonomies