Modeling Discourse Profiles in Climate Misinformation Discourse

Ge Gao; Zhengyang Shan; Bryan A. Plummer; Margrit Betke; Derry Wijaya

cs.CLOct 17, 2026

Modeling Discourse Profiles in Climate Misinformation Discourse

Authors: Ge Gao, Zhengyang Shan, Bryan A. Plummer, Margrit Betke, Derry Wijaya

Organizations: Boston University

Abstract

Prior work on climate misinformation detection typically classifies individual comments or claims using text alone. This framing overlooks a key property of online climate discourse: misinformation is often produced, reinforced, and sustained through recurring rhetorical patterns that reappear across comments and threads. Using a large-scale TikTok climate discourse dataset, we construct discourse profiles by clustering climate-related comments into recurring rhetorical patterns. We then incorporate these profiles into fine-grained climate misinformation classifiers using a late-fusion architecture. Under video-disjoint evaluation across three training seeds, discourse-profile-aware models significantly improve macro-F1 over text-only baselines for bag-of-words, TF-IDF, and MPNet representations. RoBERTa-large exhibits a positive mean improvement but substantially greater variability, and the difference is not statistically significant. Beyond classification, our results show discourse profiles shape misinformation reception and propagation. Discourse-profile-conditioned language model agents exhibit misinformation acceptance patterns aligned with profile-level misinformation prevalence in ClimateDisc, and the relationship between misinformation prevalence and engagement persistence differs across dominant discourse profiles.

Figures & tables

Figure 1: Comparison of text-only vs. discourse-profile-aware misinformation modeling. (a) Prior work Islam et al. (2020) ; Coan et al. (2021) ; Upadhyaya et al. (2023) ; Choi et al. (2023) ; Rojas et al. (2024) typically predicts a misinformation category from comment text alone, treating each comment independently. (b) We construct discourse profiles by clustering comments and use discourse profile as additional context for misinformation-type prediction; the same discourse profile abstraction also supports discourse-profile-conditioned analyses of LLM susceptibility and engagement persistence.

Figure 2: Offline discourse profile construction: encode comments, cluster embeddings, and use GPT-5-based cluster consolidation to assign the resulting clusters to higher-level discourse profiles

d_{i}

(Section 3.1 ).

Figure 3: Late-fusion discourse-profile-aware misinformation modeling: obtain misinformation-type labels and evaluate prediction models that concatenate contextual text representations with discourse-profile embeddings before classification (Section 3.2 ). Details of how we obtain comment-level misinformation labels are in Appendix A .

Local / Experiential Observers	Ideological / Political Combatants
1. Local Weather Anecdotes	0. Anti-Trump/GOP/Biden
12. Storms & Flooding	11. Government Blame / Anarchy
13. Geographical Self-Mentions	18. Electoral Mobilization
Systemic / Structural Critics	Pro-Science / Rationalist
4. Consumption / Plastics Skepticism	14. Pro-Science / Anti-Misinformation
17. Energy / Policy Debate

Table 1: Discourse profiles obtained by grouping the 21

k

-means clusters of comment embeddings into semantically coherent discourse-profiles. The Noise / Low-Signal group is excluded from downstream discourse-profile-sensitive analyses.

Representation	Text Acc	Prof Acc	$\Delta$ Acc	Text F1	Prof F1	$\Delta$ F1	Paired $p$
[MLP]
Bag-of-Words	0.690	0.709	+0.019	0.586	0.616	+0.030	0.028
TF-IDF	0.675	0.709	+0.034	0.522	0.601	+0.080	0.006
MPNet	0.799	0.808	+0.010	0.726	0.739	+0.014	0.015
RoBERTa-large	0.845	0.856	+0.011	0.794	0.810	+0.015	0.381

Table 2: Comparison of text-only ( Text ) and discourse-profile-aware ( Prof ) misinformation classification under video-disjoint evaluation. Accuracy and macro-F1 values are means across three training seeds.

p

-values are from two-sided paired t-tests comparing seed-level macro-F1 scores between the correct-profile and corresponding text-only models. Overall, correct-profile conditioning improves macro-F1 for bag-of-words, TF-IDF, and MPNet, while the RoBERTa-large difference is not statistically significant across seeds.

Figure 4: Within-Cluster Sum of Squares (WCSS; lower is better) and Silhouette Score (higher is better) across varying numbers of clusters $k$ . The Silhouette score reaches a local maximum at $k=21$ .

Representation	Text F1	Shuffled F1	Correct F1	Correct $-$ Shuffled	Significant seeds
[MLP]
Bag-of-Words	0.586	0.582	0.616	+0.034	3 / 3
TF-IDF	0.522	0.561	0.601	+0.041	3 / 3
MPNet	0.726	0.736	0.739	+0.003	1 / 3
RoBERTa-large	0.794	0.776	0.810	+0.033	1 / 3

Table 3: Permutation-test evaluation of discourse-profile conditioning across text representations. Shuffled-profile scores are first averaged across ten independent profile shuffles within each training seed; reported values are then averaged across the three training seeds. The last column reports how many of the three random seeds show a significant correct-vs-shuffled difference under the one-sample t-test. The correct-versus-shuffled advantage is most consistent for bag-of-words and TF-IDF, indicating that the specific profile assignment contributes most clearly in sparse lexical settings.

Discourse Profile	Susceptible (%)
Local / Experiential Observers	29.4
Ideological / Political Combatants	4.9
Systemic / Structural Critics	8.0
Pro-Science / Rationalist	10.4
Doomism / Nihilism	22.0
Conspiratorial / Anti-Climate	96.3

Table 4: Overall misinformation susceptibility rate by discourse profile. Conspiratorial / Anti-Climate profile exhibits the highest acceptance rate.

Figure 5: Mean persistence hours across misinformation level (Low, Mid, High) for each profile type.

Discourse Profile	$H$ Statistic	Raw $p$	BH-FDR $p$
Systemic Critique	20.501	$<0.001$	$\mathbf{0.003}$
Local Experiential	16.336	$<0.001$	$\mathbf{0.003}$
Conspiracy	9.237	0.010	$\mathbf{0.020}$
Pro Science	4.875	0.047	0.071
Political Combat	3.850	0.146	0.175
Doomism	2.912	0.233	0.233

Table 5: Within-profile Kruskal–Wallis tests examining whether persistence differs across misinformation tertiles (Low, Mid, High). BH-FDR denotes Benjamini–Hochberg correction across the six profile-level tests. Bold values remain significant at

\alpha=0.05

. After BH-FDR correction, persistence differs significantly across misinformation tertiles for Systemic Critique, Local Experiential, and Conspiracy.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Comment (ClimateDisc)	Matched Claim (QuotaClimat)
Climate change is a HOAX	Climate change is a complete hoax always has been.
Yes, climate change is real to what extent is arguable, but definitely changing	Climate change is definitely real. It changes every year from winter to autumn to spring to summer.
Climate change is very real. The climate has always changed, but in recent years it is changing too fast	Climate change is definitely real. It changes every year from winter to autumn to spring to summer.

Appendix

Table 6: Representative matched comment–claim pairs obtained using a cosine-similarity threshold of 0.85.

Cohen’s Kappa	Quality
>0.8	Almost Perfect Agreement
>0.6	Substantial Agreement
>0.4	Moderate Agreement
>0.2	Fair Agreement
0-0.2	Slight Agreement
<0	Almost No Agreement

Appendix

Table 7: Interpretation for different ranges of the Cohen’s Kappa values.

Comparison	Cohen’s $\kappa$
Human 1 $\leftrightarrow$ Human 2	0.76
Human 1 $\leftrightarrow$ classifier	0.57
Human 2 $\leftrightarrow$ classifier	0.55

Appendix

Table 8: Cross-domain agreement on 100 randomly sampled ClimateDisc comments independently labeled by two human annotators.

Panel of Figure 6 — Figure 6: Distribution of cluster/discourse profile sizes.

Category	BOW	TF-IDF	MPNet	RoBERTa
Science Unreliable	+6.8	+23.4	+0.5	+5.4
Not Bad	+5.7	+11.6	+1.8	+0.8
Not Happening	+4.3	+3.2	+0.6	+0.8
Not Human-Caused	+2.5	+2.6	+1.4	$-$ 0.2
Fossil Fuels Needed	+1.2	+8.6	+3.6	+1.3
Solutions Harmful	+0.7	+4.0	+0.0	+1.7

Appendix

Table 9: Mean category-level F1 change from the text-only model to the correct-profile model, averaged across three training seeds. The misinformation categories in this analysis are inherited from the QuotaClimat taxonomy. Values are F1 percentage-point differences. RoBERTa denotes RoBERTa-large.

Fake News Detection

Fake news detection identifies false or misleading claims by analyzing content, context, evidence, and how information spreads.

See all 93 papers

Retrieving Climate Change Disinformation by Narrative
Mar 23, 2026Max Upravitelev, Veronika Solopova, Charlott Jakob +3Information Retrieval Fake News Detection
Technische Universität Berlin · German Research Center for Artificial Intelligence (DFKI) · BIFOLD – Berlin Institute for the Foundations of Learning and Data +1
Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit
Jun 2, 2026JooYoung Lee, Lin Tian, Angela Brillantes +2LLM Evaluation Zero-Shot Text Classification
University of Technology Sydney, Sydney, Australia.
Angry but Accurate: Detecting and Profiling the Counter-Misinformation Ecosystem on Twitter
Jul 3, 2026Eun Cheol Choi, Emilio FerraraSocial Media Analysis User Modeling
University of Southern California, Annenberg School of Communication, Los Angeles, CA, United States · University of Southern California, Thomas Lord Department of Computer Science, Los Angeles, CA, United States

Mar 23, 2026J/K move · Enter open · S save

Retrieving Climate Change Disinformation by Narrative

Max Upravitelev, Veronika Solopova, Charlott Jakob, Premtim Sahitaj+2

Technische Universität Berlin · German Research Center for Artificial Intelligence (DFKI) · BIFOLD – Berlin Institute for the Foundations of Learning and Data+1

Information Retrieval Fake News Detection

Open paper AskPDF arXiv

Detecting climate disinformation narratives typically relies on fixed taxonomies, which do not accommodate emerging narratives. Thus, we re-frame narrative detection as a retrieval task: given a narrative's core message as a query, rank texts from a corpus by alignment with that narrative. This formulation requires no predefined label set and can accommodate emerging narratives. We repurpose three climate disinformation datasets (CARDS, Climate Obstruction, climate change subset of PolyNarrative) for retrieval evaluation and propose SpecFi, a framework that generates hypothetical documents to bridge the gap between abstract narrative descriptions and their concrete textual instantiations. SpecFi uses community summaries from graph-based community detection as few-shot examples for generation, achieving a MAP of 0.505 on CARDS without access to narrative labels. We further introduce narrative variance, an embedding-based difficulty metric, and show via partial correlation analysis that standard retrieval degrades on high-variance narratives (BM25 loses 63.4% of MAP), while SpecFi-CS remains robust (32.7% loss). Our analysis also reveals that unsupervised community summaries converge on descriptions close to expert-crafted taxonomies, suggesting that graph-based methods can surface narrative structure from unlabeled text.

Abstract

Fake News Detection

Explore similar work

Retrieving Climate Change Disinformation by Narrative

Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit

Angry but Accurate: Detecting and Profiling the Counter-Misinformation Ecosystem on Twitter