Real-World Challenges in Fake News Detection: Dealing with Posts by Cold Users

Sai Keerthana Karnam; Abhirup Kundu; Jashn Arora; Manish Jain; Animesh Mukherjee

cs.SIMar 30, 2026

Real-World Challenges in Fake News Detection: Dealing with Posts by Cold Users

Authors: Sai Keerthana Karnam, Abhirup Kundu, Jashn Arora, Manish Jain, Animesh Mukherjee

Organizations: Indian Institute of Technology Kharagpur · Google DeepMind

Abstract

Social media serves as a primary source of information in the current digital era. Many people consume a vast range of information in a very short span, yet, amidst the stream of genuine information, fake news and rumors continue to spread. The need for effective detection models is becoming increasingly critical. Past user behavior and user engagement on a post are strong signals that SOTA approaches leverage for fake news detection and other post classification tasks. However, these approaches lean too heavily on knowing this past behavior, and thus suffer from a cold user problem, or users that are new or have minimal footprint on the platform. In this paper, we make three core contributions. We first establish the value of user behavior, both content and user-user interactions, in the task of fake news and rumor detection. We then establish the extensive prevalence of cold users in the real-world datasets, and show the need for newer algorithms considering cold users. We next propose a novel socially-aware context representation scheme - USER EVIDENCE NETWORK (UEN) - to detect the spread of misinformation and unverified information while efficiently navigating this cold user challenge. We introduce techniques that approximate missing or absent behavior data of a new user from existing users' interactions. By carefully addressing the cold user challenge, our work provides robust approaches targeting fake news and rumor detection for real-world platforms.

Figures & tables

Paper	Content-based	Propagation patterns	Historical user behaviour	Handles cold users
Rashkin et al. (2017) , Miyazaki et al. (2023) , Nakamura et al. (2020)	✓	✗	✗	✗
Han et al. (2020) , Bian et al. (2020) , Nakamura et al. (2020) , Xu et al. (2023) , Wei et al. (2024b)	✓	✓	✗	✗
Yuan et al. (2019a) , Sun et al. (2023) , Su et al. (2023)	✓	✓	✓	✗
Wei et al. (2024a)	✓	✓	✗	✗ (Handles cold start problem but doesn’t include user features)
Our work	✓	✓	✓	✓

Table 1: Prior works leverage content features, model propagation patterns, and incorporate historical user behaviour, but lack the ability to handle cold users. In contrast, our work addresses all four aspects, with a particular emphasis on cold user generalization.

Statistic	Fakeddit dataset (text)	Gossipcop dataset (text)
Total #samples	1,063,106	5,464
Text #samples	971,806	5,464
Text #samples ( $\|$ comments $\|$ $\geq 1$ )	583,305	5,464
Comments	9,291,298	294,288
Fake #samples	179,961	2,732
True #samples	402,753	2,732

Table 2: Dataset statistics: The time period between the training set and testing set do not overlap ensuring that there is no data leakage.

Panel of Figure 1 — Figure 1: Graph representation of the social context.

Figure 2: The overall architecture of the UeN framework. For training , first the global interaction-based graph is constructed and trained to generate user representation in the first module. These embeddings are passed to the second module, which captures content and user behavior features. These features are fed to the third module to obtain a robust graph representation, which is ultimately classified in the fourth module. For testing , each user in the sample is first checked to determine if they are a cold user. Cold user representation is obtained from the cold user behavior mapper module (shown in Figure 3 ); while other users’ representation is obtained from the first module. Remaining steps follow the training phase.

Panel of Figure 3 — Figure 3: Cold user behavior mapper module.

Dataset	Hyperparameters	GCN	GraphSage	GAT
Fakeddit	$\lambda$	0.62	0.89	0.87
	$k_{1}$	19	28	11
	$k_{2}$	72	96	51
Gossipcop	k	39	4	11

Table 3: Hyperparameters used for each of the GNN models.

Dataset	Method	Metrics	GCN	GraphSage	GAT
Fakeddit	UeN ${}_{w/o\text{\ user }}$	Accuracy	0.77	0.81	0.79
	UeN ${}_{w/o\text{\ user }}$	Macro-F1	0.73	0.77	0.75
	UeN ${}_{w/o\text{\ mapper }}$	Accuracy	0.85	0.87	0.86
	UeN ${}_{w/o\text{\ mapper }}$	Macro-F1	0.81	0.83	0.81
	UeN	Accuracy	0.87	0.87	0.87
	UeN	Macro-F1	0.84	0.85	0.83

Table 4: Evaluation of the UeN frameworks. The variant with the best metric values are noted in boldface . Best accuracy is highlighted in green and best macro-F1 is highlighted in purple .

Model	Fakeddit ( $p$ -value)	Gossipcop ( $p$ -value)
GCN	$1.8342\times 10^{-7}$	$2.1947\times 10^{-54}$
GraphSage	$7.0411\times 10^{-96}$	$3.284\times 10^{-19}$
GAT	$4.6118\times 10^{-12}$	$6.2534\times 10^{-90}$

Table 5:

p

-values comparing the prediction outcomes of UeN

{}_{w/o\text{\ mapper }}

with UeN using the Mann-Whitney U test.

Category	Fakeddit			Gossipcop
	Total	True	Fake	Total	True	Fake
Total	116,639	86,443	30,196	1,094	557	537
(0.5,1]	31,327	23,080	8,247	838	509	329
(0,0.5]	63,915	50,758	13,157	240	44	196
0	21,397	12,605	8,792	16	4	12

Table 6: Distribution of test samples based on overlap ratio between the users in the sample and the global interaction-based graph. Even the last row, representing the existence of solely cold users , has a significant number of samples for Fakeddit.

GCN
Category	$\mathbb{\textsc{UeN}_{\textbf{w/o \ user}}}$		$\mathbb{\textsc{UeN}_{\textbf{w/o \ mapper}}}$		UeN
Category	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
(0.5,1]	0.83	0.79	0.93	0.91	0.93	0.92
(0,0.5]	0.78	0.72	0.89	0.84	0.89	0.84
0	0.66	0.65	0.63	0.61	0.72	0.71
GraphSage

Table 7: Results for Fakeddit dataset, based on overlap ratio based buckets. The variant with the best metric values for the zero overlap buckets are noted in boldface . Best accuracy for this bucket is highlighted in green and best macro-F1 for this bucket is highlighted in purple .

GCN
Category	$\mathbb{\textsc{UeN}_{\textbf{w/o \ user}}}$		$\mathbb{\textsc{UeN}_{\textbf{w/o \ mapper}}}$		UeN
Category	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
(0.5,1]	0.92	0.91	0.95	0.94	0.95	0.94
(0,0.5]	0.80	0.70	0.85	0.78	0.85	0.79
GraphSAGE
(0.5,5]	0.94	0.93	0.96	0.96	0.96	0.96

Table 8: Results for Gossipcop dataset, based on overlap ratio based buckets. The variant with the best metric values for the (0,0.5] category are noted in boldface . Best accuracy for this bucket is highlighted in green and best macro-F1 for this bucket is highlighted in purple .

GCN
Category	$\mathbb{\textsc{UeN}_{\textbf{H1}}}$		$\mathbb{\textsc{UeN}_{\textbf{H1+H2}}}$		$\mathbb{\textsc{UeN}_{\textbf{H1+H2+H3}}}$
Category	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
Overall	0.86	0.82	0.86	0.83	0.87	0.84
(0.5,1]	0.93	0.92	0.93	0.92	0.93	0.92
(0,0.5]	0.88	0.85	0.89	0.83	0.89	0.84
0	0.67	0.64	0.71	0.70	0.72	0.71

Table 9: Results for Fakeddit dataset, based on overlap ratio based buckets. The variant with the best metric values for the (0) category are noted in boldface . Best accuracy for this bucket is highlighted in green and best macro-F1 for this bucket is highlighted in purple .

GCN
Category	$\mathbb{\textsc{UeN}_{\textbf{H1}}}$		$\mathbb{\textsc{UeN}_{\textbf{H2}}}$
Category	Accuracy	Macro-F1	Accuracy	Macro-F1
Overall	0.92	0.92	0.92	0.92
(0.5,1]	0.95	0.94	0.95	0.94
(0,0.5]	0.84	0.76	0.85	0.79
GraphSAGE

Table 10: Results for Gossipcop dataset, based on overlap ratio based buckets. The variant with the best metric values for the (0,0.5] category are noted in boldface . Best accuracy for this bucket is highlighted in green and best macro-F1 for this bucket is highlighted in purple .

Model	$k_{1}=5$	$k_{1}=10$	$k_{1}=15$
GCN	74.88 $\pm$ 0.33	76.54 $\pm$ 0.45	76.63 $\pm$ 0.42
GraphSAGE	75.84 $\pm$ 0.48	77.02 $\pm$ 0.50	77.29 $\pm$ 0.53
GAT	77.29 $\pm$ 0.25	77.89 $\pm$ 0.34	77.63 $\pm$ 0.29

Table 11: Validation accuracy (%) of different models for varying

k_{1}

values on the Fakeddit dataset. Values are reported as mean ± standard deviation.

Dataset	Category	Metrics	UeN			PSGT	Llama		Qwen		GPT-OSS
			GCN	GraphSage	GAT		$\text{LLM}_{\text{cnt}}$	$\text{LLM}_{\text{cnt+cmt}}$	$\text{LLM}_{\text{cnt}}$	$\text{LLM}_{\text{cnt+cmt}}$	$\text{LLM}_{\text{cnt}}$	$\text{LLM}_{\text{cnt+cmt}}$
Fakeddit	Overall	Accuracy	0.87	0.87	0.87	0.81	0.63	0.73	0.60	0.65	0.61	0.64
	Overall	Macro-F1	0.84	0.85	0.83	0.76	0.52	0.54	0.51	0.55	0.45	0.44
	(0.5,1]	Accuracy	0.93	0.93	0.93	0.86	0.63	0.73	0.62	0.67	0.60	0.64
	(0.5,1]	Macro-F1	0.92	0.91	0.90	0.83	0.52	0.53	0.55	0.59	0.44	0.43
	(0,0.5]	Accuracy	0.89	0.88	0.89	0.83	0.68	0.79	0.60	0.65	0.65	0.69

Table 12: Comparison of results of UeN with other baselines. Best accuracy for each baseline is highlighted in green and best macro-F1 for this bucket is highlighted in purple .

Fake News Detection

Fake news detection identifies false or misleading claims by analyzing content, context, evidence, and how information spreads.

See all 98 papers

Jul 7, 2026

Unsupervised Anomaly Detection of Information Operations Users via Behavioral and Language Patterns

Information Operations on social media networks have been identified as a significant threat to democracy and modern society, but they are challenging and expensive to detect by humans. Existing supervised IO detection methods fail to capture the dynamic nature of evolving IO user behavior, while existing unsupervised approaches rely on oversimplified assumptions of coordination among IO users that may not exist in practice. To overcome the limitations of existing methods, we formulate IO user detection as an anomaly detection problem and propose a novel unsupervised IO user detection approach called Temporal-bEhavior-laNguage Signals for information Operation Recognition (TENSOR), which leverages multimodal data, including temporal online user behavior, such as message posting activities, and the textual content of the messages. The motivation is that IO users are typically a very small fraction of all online users and have unique temporal behavioral and language patterns. Specifically, we train a Temporal Point Process (TPP) to capture abnormal temporal behavioral patterns of IO users because they are known to behave in a coordinated manner for IO campaigns. We further introduce a novel evidence function that converts LLM responses, which are generated from user post timelines, into quantitative scores to adjust the TPP outputs for better IO user detection. Experimental results show that TENSOR outperforms the baselines on five real-world IO datasets. Code is available at https://github.com/xiuzhenzhang/TENSOR.

Sishun Liu, Sajal Halder, Ke Deng +2

RMIT University, Melbourne, Victoria 3000, Australia · Macquarie University, Sydney, New South Wales 2000, Australia

Social Media Analysis Temporal Point Processes Unsupervised Anomaly Detection

Ask PDF

May 4, 2026

An Information-theoretic Propagation Denoising and Fusion Framework for Fake News Detection

Incomplete propagation data significantly hinders robust fake news detection. Recent approaches leverage large language models to simulate missing user interactions via role-playing, thereby enriching propagation with synthetic signals. However, such propagation data is intrinsically unreliable, and directly fusing it can lead to biased representations, leading to limited detection performance. In this paper, we alleviate the unreliability of synthetic propagation from the mutual information perspective and propose a novel information-theoretic propagation denoising and fusion (InfoPDF) framework to learn effective representations from both real and synthetic propagation. Specifically, we first generate attribute-specific synthetic propagation using large language models. Then we model each synthetic propagation graph as a probabilistic latent distribution to guide reliability-aware adaptive fusion with real propagation. During training, we design a mutual information-based objective to learn compressed and task-sufficient propagation representations. It jointly suppresses noisy signals across attribute-specific synthetic propagation, maintains consistency between real and synthetic propagation representations, and ensures task sufficiency for fake news detection and attribute prediction. Experiments on three real-world datasets show that InfoPDF consistently achieves superior performance across various fake news detection tasks. Further analysis demonstrates that InfoPDF can estimate attribute-level reliabilities and learn more discriminative propagation representations.

Mengyang Chen, Lingwei Wei, Wei Zhou +1

Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

Synthetic Data Augmentation Fake News Detection Automated Fact-Checking

Ask PDF

May 7, 2026

COVID-19 Infodemic. Understanding content features in detecting fake news using a machine learning approach

The use of content features, particularly textual and linguistic for fake news detection is under-researched, despite empirical evidence showing the features could contribute to differentiating real and fake news. To this end, this study investigates a selection of content features such as word bigrams, part of speech distribution etc. to improve fake news detection. We performed a series of experiments on a new dataset gathered during the COVID-19 pandemic and using Decision Tree, K-Nearest Neighbor, Logistic Regression, Support Vector Machine and Random Forest. Random Forest yielded the best results, followed closely by Support Vector Machine, across all setups. In general, both the textual and linguistic features were found to improve fake news detection when used separately, however, combining them into a single model did not improve the detection significantly. Differences were also noted between the use of bigrams and part of speech tags. The study shows that textual and linguistic features can be used successfully in detecting fake news using the traditional machine learning approach as opposed to deep learning.

Vimala Balakrishnan, Lee Zing Hii, Eric Laporte

Faculty of Computer Science & Information Technology, Universiti Malaya, 50603, Kuala Lumpur, Malaysia · LIGM, Univ Gustave Eiffel, CNRS, F-77454 Marne-la-Vallée, France

Fake News Detection Text Classification Automated Fact-Checking

Ask PDF

Abstract

Fake News Detection

Explore similar work

Unsupervised Anomaly Detection of Information Operations Users via Behavioral and Language Patterns

An Information-theoretic Propagation Denoising and Fusion Framework for Fake News Detection

COVID-19 Infodemic. Understanding content features in detecting fake news using a machine learning approach