The spread of misinformation on social networks poses a significant challenge to online communities and society at large. Not all users contribute equally to this phenomenon: a small number of highly effective individuals can exert outsized influence, amplifying false narratives and contributing to significant societal harm. This paper seeks to mitigate the spread of misinformation by enabling proactive interventions, identifying and ranking users according to key behavioral indicators associated with harmful content dissemination. We examine three user archetypes -- amplifiers, super-spreaders, and coordinated accounts -- each characterized by distinct behavioral patterns in the dissemination of misinformation. These are not mutually exclusive, and individual users may exhibit characteristics of multiple archetypes. We develop and evaluate several user ranking models, each aligned with a specific archetype, and find that super-spreader traits consistently dominate the top ranks among the most influential misinformation spreaders. As we move down the ranking, however, the interplay of multiple archetypes becomes more prominent. Additionally, we demonstrate the critical role of temporal dynamics in predictive performance, and introduce methods that reduce data requirements by minimizing the observation window needed for accurate forecasting. Finally, we demonstrate the utility and benefits of explainable AI (XAI) techniques, integrating multiple archetypal traits into a unified model to enhance interpretability and offer deeper insight into the key factors driving misinformation propagation. Our findings provide actionable tools for identifying potentially harmful users and guiding content moderation strategies, enabling platforms to monitor accounts of concern more effectively.
Figures & tables
Figure 1: Credibility distribution for labeled retweets for each dataset under scrutiny : VaccinItaly (left) and COVID-19 Multilanguage (right). Credibility score is assigned to each tweet based on NewsGuard’s ratings, as described in Section 4.1
Figure 2: Illustration of the network dismantling process. The figure illustrates how the removal of highly ranked users impacts the overall volume of misinformation. Each point on the curve corresponds to one user removed from the network, in descending order according to their ranking. The observed trend reveals that a small subset of users is responsible for a disproportionately large share of misinformation diffusion.
Figure 3: Performance of the best performing ranking methods for each archetype. Each marker represents a user removed from the misinformation reshare network of the VaccinItaly dataset. The x-axis represents the user’s rank, while the y-axis indicates the estimated reduction in misinformation achieved by removing that user.
ML
TASH-Index
H-Index
TAI-Score
Influence Score
@k
nDCG
Quality
nDCG
Quality
nDCG
Quality
nDCG
Quality
nDCG
Quality
3
0.963
34.0%
0.949
33.3%
0.892
32.6%
0.938
32.6%
0.892
32.6%
5
0.967
39.8%
0.912
35.7%
0.858
35.0%
0.901
35.0%
0.865
35.7%
10
0.939
45.8%
0.931
45.8%
0.832
40.4%
0.873
40.3%
0.824
40.4%
20
0.890
49.0%
0.906
52.8%
0.790
43.7%
0.848
49.2%
0.783
43.2%
100
0.848
62.3%
0.832
60.0%
0.800
66.7%
0.829
65.8%
0.793
66.9%
Table 1: Comparison of methods using nDCG and Quality scores at different @k levels on the VaccinItaly Dataset. The maximum Quality score remains identical across methods when all the nodes are removed from the misinformation reshare network, except those that appear after the training phase; consequently, 2.9% of misinformation cannot be mitigated due to the emergence of new users in the test set. Bold values denote the best results, underlined values indicate the second best.
Figure 4: SHAP summary plot: The plot displays the impact of each feature on the machine learning model’s predictions. Each dot represents a SHAP value for a particular user-feature instance, with color indicating the actual feature value (from low in blue to high in red). SHAP values quantify the direction and magnitude of a feature’s contribution to the model’s output. Values are shown on a logarithmic scale to enhance visibility and interpretability.
Figure 5: SHAP decision plots showing how archetypal features contribute to model predictions for users in different ranking intervals. Each line represents a user, starting from the base value and accumulating SHAP values (bottom up) to reach the final prediction on the x-axis. Line color indicates the total SHAP contribution (log-scaled), revealing how the influence of different archetypes varies across the ranking.
Figure 6: Performance of the top-performing models under data scarcity conditions. Each point corresponds to a training of the model with X days of data, obtaining Y as nDCG in evaluation over the same period.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Optimization grid of α and δ for the TASH-Index using nDCG as the evaluation metric.
Figure 8: Performance comparison of amplifiers -ranking strategies.
Figure 9: Performance of coordinated user detection strategies based on similarity networks.
Figure 10: Comparison of misinformation removal effectiveness across super-spreader identification methods.
ML
TASH-Index
H-Index
TAI-Score
Influence Score
@k
nDCG
Quality
nDCG
Quality
nDCG
Quality
nDCG
Quality
nDCG
Quality
3
0.986
41.7%
0.986
41.7%
0.880
34.1%
0.914
36.2%
0.932
37.6%
5
0.939
42.7%
0.932
42.0%
0.905
41.5%
0.932
43.0%
0.886
38.4%
10
0.918
46.8%
0.904
45.2%
0.919
51.3%
0.931
49.5%
0.918
49.0%
20
0.932
56.9%
0.926
56.8%
0.899
54.9%
0.925
56.7%
0.904
54.0%
100
0.896
68.6%
0.880
66.2%
0.872
68.4%
0.890
69.5%
0.884
70.7%
Appendix
Table 2: Comparison of methods using nDCG and Quality scores at different @k levels on COVID-19 Multilanguage dataset. The maximum Quality score remains identical across methods because all the nodes are removed from the misinformation reshare network except those that appear after the training phase; consequently, 4.8% of misinformation cannot be mitigated due to the emergence of new users in the test phase. Bold values denote the best results, underlined values indicate the second best.
Figure 11: Impact of varying the credibility threshold during training. The x-axis indicates the threshold applied, and the y-axis shows model performance (nDCG). For each threshold x , only training data involving content with credibility scores ≤x was used.
Figure 12: Credibility profiles of top-100 ranked users across different ranking methods. Each column represents a ranking method, with users shown as bubbles positioned according to their average credibility scores for original posts (orange) and reshares (blue). For the Optimal ranking, we use distinct colors (green for posts, pink for reshares) to emphasize its role as the ground-truth baseline. Bubble size reflects ranking position, with larger bubbles indicating higher ranks. While all credibility scores are computed from training data, only the Optimal ranking is derived using test-set information based on actual future misinformation spread. This visualization reveals how each method prioritizes users with different credibility profiles, providing insight into the types of actors emphasized by different ranking strategies.
Figure 13: Sliding window procedure for model tuning and evaluation.
Model
MAE ↓
MSE ↓
nDCG@3 ↑
nDCG@5 ↑
nDCG@10 ↑
nDCG ↑
Linear Model
1.073
343.34
0.955
0.920
0.851
0.890
Random Forest
1.108
397.55
0.970
0.968
0.939
0.910
XGBoost
1.079
641.62
0.750
0.860
0.850
0.864
Neural Network
1.287
361.50
0.970
0.928
0.910
0.900
Appendix
Table 3: Performance comparison of tested models on the test set. Random Forest emerges as the most stable and reliable model overall, especially in managing high target skewness and inter-feature correlation.
Social media platforms generate vast volumes of user-generated content that often obscure meaningful discourse, a phenomenon conceptualized as social noise. While existing misinformation detection methods primarily focus on factual verification, they largely overlook the role of social noise in amplifying misinformation. This study assesses the performance of six text-based models and a graph neural network (GNN) in mitigating the effects of misinformation using predefined social noise constructs. The dataset consists of 500,000 tweets collected from the X platform using the Ukraine hashtag during the Russia-Ukraine conflict from March 2024 to March 2025. Advanced machine learning models achieve strong performance, with accuracies ranging from 79.94% to 97.17%. RoBERTa demonstrates the highest performance, with an accuracy of 97.17%, Cohen's kappa of 0.9660, MCC of 0.9660, and AUC of 0.9977. The GNN achieves an accuracy of 64.52% and an AUC of 0.899, demonstrating performance above random-chance classification by exploiting the relational structure between tweets. All text-based models achieve kappa values greater than 0.75, indicating substantial overall agreement. Explainable AI analysis using LIME reveals that 89.6% of predictive weight is attributed to contextual features rather than isolated keywords. The findings highlight that integrating social noise into AI pipelines enhances misinformation detection and supports more effective mitigation strategies in large-scale social media environments.
Manar Alsaid
East Texas A&M University, Department of Computer Science and Information Systems, Commerce, Texas, USA
Social media platforms increasingly use AI-based credibility indicators to help users judge misinformation. Unlike individual human-AI decision-making, these indicators are embedded in information spread: users see both an AI prediction and earlier judgments shaped by the same AI, and their own judgments may then enter the public history. Yet how to analytically characterize this process remains under-explored. We therefore introduce a social-learning lens for this setting by extending the classical Bayesian cascade model with the AI indicator as a shared public signal. The resulting Gateway condition compares the evidence from the AI prediction with users' private impressions. Through this view, we show that AI changes what public history means. Crowd agreement may reflect accumulated independent human evidence, or repeated dependence on the same AI prediction. This creates a preservation-correction trade-off: stronger reliance on AI can preserve correct predictions, but can also lock in incorrect ones by blocking corrective private impressions. We calibrate the model using human-subject data on news veracity judgments. Although the AI outperforms human users, the average user weights it below her own impression but above several peer judgments, while individual users vary from discounting the AI to relying on it enough to cascade. Simulations show that over-reliance on a weak AI is especially harmful, and that diversifying AI signals across users can better keep the crowd informative. We conclude with implications for understanding human-AI interaction in information spread and designing misinformation interventions.
Zhuoran Lu, Weilong Wang, Yangyang Yu +4
Purdue University West Lafayette, Indiana, USA · St John’s University New York, New York, USA · Stevens Institute of Technology Hoboken, New Jersey, USA +2
Social media serves as a primary source of information in the current digital era. Many people consume a vast range of information in a very short span, yet, amidst the stream of genuine information, fake news and rumors continue to spread. The need for effective detection models is becoming increasingly critical. Past user behavior and user engagement on a post are strong signals that SOTA approaches leverage for fake news detection and other post classification tasks. However, these approaches lean too heavily on knowing this past behavior, and thus suffer from a cold user problem, or users that are new or have minimal footprint on the platform. In this paper, we make three core contributions. We first establish the value of user behavior, both content and user-user interactions, in the task of fake news and rumor detection. We then establish the extensive prevalence of cold users in the real-world datasets, and show the need for newer algorithms considering cold users. We next propose a novel socially-aware context representation scheme - USER EVIDENCE NETWORK (UEN) - to detect the spread of misinformation and unverified information while efficiently navigating this cold user challenge. We introduce techniques that approximate missing or absent behavior data of a new user from existing users' interactions. By carefully addressing the cold user challenge, our work provides robust approaches targeting fake news and rumor detection for real-world platforms.
Sai Keerthana Karnam, Abhirup Kundu, Jashn Arora +2
Indian Institute of Technology Kharagpur · Google DeepMind