BERTopic-Virality Prioritisation: A Scalable Framework for Thematic and Comparative Analysis of COVID-19 and Monkeypox Misinformation on Twitter
Organizations: Department Of Computing And Mathematics, Manchester Metropolitan University, Manchester, UK
Abstract
Health misinformation circulating during pandemics can gain traction rapidly, creating harmful narratives that compete with public health guidance. Most topic-modelling pipelines treat engagement as an external outcome, limiting their ability to prioritise semantically coherent topics that are also rapidly diffusing. We introduce BERTopic-VP, a virality-prioritised topic-modelling framework that combines contextual embedding-based clustering (BERTopic) with a post hoc Virality Prioritisation (VP) layer. The pipeline is complemented by a two-stage hybrid misinformation detection module that fuses a supervised content-based classifier with an external verification signal derived from public-health knowledge bases. Applied to three benchmark datasets, COVID-19_FNIR, Monkeypox, and Constraint, the framework achieves strong classification performance, with F1 up to 0.950 and ROC-AUC up to 0.989, while identifying high-impact clusters under top 1%, 5%, and 10% VP thresholds. For datasets without native engagement metadata, prioritisation is based on a logistic propensity-to-spread score, used as an ordinal proxy for diffusion potential rather than a direct measure of engagement. The results show that integrating semantic structure, virality-aware ranking, and affective-linguistic profiling enables scalable and interpretable comparative analysis of misinformation across pandemics. The proposed framework supports monitoring-oriented early warning by surfacing low-volume but high-risk narratives for analyst review.
Figures & tables
| Dataset | Observed engagement? | VP signal used | Interpretation |
|---|---|---|---|
| Monkeypox | Yes | Observed | Realised interaction counts drive VP ranking. |
| COVID–19_FNIR | No | Proxy from , equivalently | VP ranking reflects predicted diffusion propensity, not realised engagement. |
| Constraint | No | Proxy from , equivalently | VP ranking reflects predicted diffusion propensity, not realised engagement. |
| Component | Hyperparameter | Value |
|---|---|---|
| UMAP | n_neighbors | 15 |
| UMAP | min_dist | 0.1 |
| UMAP | n_components | 5 |
| HDBSCAN | min_cluster_size | 50 |
| HDBSCAN | min_samples | 10 |
| Dataset | Purity | NMI | ARI | # Clusters | Coherence ( ) |
|---|---|---|---|---|---|
| COVID–19_FNIR | 0.742 | 0.210 | 0.181 | 112 | 0.084 |
| Monkeypox | 0.940 | 0.281 | 0.233 | 87 | 0.286 |
| Constraint | 0.522 | 0.036 | -0.002 | 5 | 0.463 |
| Dataset | Accuracy | Precision | Recall | F1 | ROC-AUC | PR-AUC |
|---|---|---|---|---|---|---|
| COVID–19_FNIR | 0.906 | 0.897 | 0.918 | 0.907 | 0.966 | 0.961 |
| Monkeypox | 0.956 | 0.937 | 0.963 | 0.950 | 0.989 | 0.985 |
| Constraint | 0.895 | 0.901 | 0.897 | 0.899 | 0.951 | 0.953 |
| Pandemic Source | Polarity | Joy | Fear | Anger | Sadness |
|---|---|---|---|---|---|
| COVID–19_FNIR | 0.051 | 0.202 | 0.150 | 0.132 | 0.118 |
| Constraint | 0.037 | 0.181 | 0.167 | 0.144 | 0.103 |
| Monkeypox | 0.022 | 0.135 | 0.226 | 0.193 | 0.152 |
| Group | TTR | Avg. Sentence Length | Flesch Reading Ease | Flesch–Kincaid Grade Level |
|---|---|---|---|---|
| COVID–19_FNIR | 0.89 | 18.55 | 30.26 | 12.95 |
| Monkeypox | 0.92 | 15.55 | 55.73 | 8.90 |
| Dataset | Top 1% | Top 5% | Top 10% |
|---|---|---|---|
| COVID–19_FNIR | 9/112 | 15/112 | 26/112 |
| Monkeypox | 5/87 | 11/87 | 16/87 |
| Mean ARI | ARI range | range | Noise range | |
|---|---|---|---|---|
| 10 | 0.709 | [0.654, 0.770] | [42, 48] | [0.368, 0.398] |
| Dataset | Cluster ID | Virality Score ( ) | Top Terms | Misinfo Presence |
|---|---|---|---|---|
| COVID–19_FNIR | 1 | 3.457 | masks, cure, prevent | Mostly factual |
| Monkeypox | 11 | 13.000 | scientists, losing fight | None detected |
| Monkeypox | 25 | 9.765 | cases, states, prevalence | None detected |
| Constraint | 2 | 2.681 | lagos, discharged, fct | All misinformation |
| Constraint | 4 | 2.649 | covid, cases, reported | Mixed |
| Dataset | Cluster theme | Representative tweets (paraphrased) | |
|---|---|---|---|
| COVID–19_FNIR (Low) | Mixed preventive and political claims | “The vaccine is useless, the virus is fake, and the government just wants control.” “Masks work but lockdowns destroy jobs.” The framing conflicts across posts. | 0.084 |
| Constraint (High) | Vaccine side-effects narrative | “My friend took the shot and got sick for weeks.” “Vaccines have dangerous chemicals that doctors hide.” A single, consistent misinformation frame. | 0.463 |
| Mpox (Low) | Broad outbreak speculation | “This virus comes from labs in Europe.” “It is fake news made up by the media.” Mixed conspiracy and denial themes. | 0.22 |
| Mpox (High) | Vaccine rollout critique | “People in rural areas still cannot access vaccines.” “Authorities promise doses but none arrive.” Focused grievance on distribution failure. | 0.33 |
| Model / Study | Dataset / Domain | Notes | |
|---|---|---|---|
| BERTopic-VP (this study) | COVID–19_FNIR misinformation tweets | 0.084 | Lowest coherence, reflecting narrative fragmentation. |
| BERTopic-VP (this study) | Mpox misinformation tweets | 0.286 | Higher cohesion, with fewer dominant frames. |
| BERTopic-VP (this study) | Constraint misinformation tweets | 0.463 | Highest cohesion in this study’s datasets. |
| BERTopic (reported) [ 50 ] | COVID–19 vaccine tweets | 0.52 | Higher coherence due to a more thematically uniform dataset. |
| BERTopic (reported) [ 51 ] | Facebook COVID–19 posts | 0.53 | Performs comparably to and, in some settings, modestly better than LDA on multilingual health data. |
| Model / Study | Dataset / Domain | Notes | |
|---|---|---|---|
| LDA [ 19 ] | Twitter vaccine discourse | Lower robustness in microtext settings. | |
| LDA [ 52 ] | Multi-domain comments | 0.031 | Lowest performance on short-text corpora. |