Click it or Leave it: Detecting and Spoiling Clickbait with Informativeness Measures and Large Language Models
Organizations: Faculty of Mathematics and Information Science, Warsaw University of Technology
Abstract
Clickbait headlines degrade the quality of online information and undermine user trust. We present a hybrid approach to clickbait detection that combines transformer-based text embeddings with linguistically motivated informativeness features. Using natural language processing techniques, we evaluate classical vectorizers, word embedding baselines, and large language model embeddings paired with tree-based classifiers. Our best-performing model, XGBoost over embeddings augmented with 15 explicit features, achieves an F1-score of 91%, outperforming TF-IDF, Word2Vec, GloVe, LLM prompt based classification, and feature-only baselines. The proposed feature set enhances interpretability by highlighting salient linguistic cues such as second-person pronouns, superlatives, numerals, and attention-oriented punctuation, enabling transparent and well-calibrated clickbait predictions. We release code and trained models to support reproducible research.
Figures & tables
| Name | Instances | Task | Features | Collection |
|---|---|---|---|---|
| Kaggle-1 | 32,000 | Detection | Title, binary label | News websites (method undocumented) |
| Kaggle-2 | 21,029 | Detection | Title, body, binary label | Kaggle competition |
| Clickbait Challenge 2017 | 38,830 | Detection | Title ( postText ), score (0.0–1.0) | Twitter; crowdsourced labels |
| SemEval Spoiling | 4,000 | Spoiling | Title, body, extractive + abstractive spoilers | Social media spoiling accounts |
| Measure | Description |
|---|---|
| Word Count | Total number of words in the text |
| Common Words Ratio | Proportion of stop words to total words |
| Capital Letters Ratio | Share of uppercase characters |
| Superlatives Ratio | Frequency of superlative adjectives or adverbs |
| FRES | Flesch Reading-Ease score; higher indicates easier readability |
| Polarity | Sentiment polarity of the text |
| Representation | Classifier | Acc. | Prec. | Rec. | F1 | AUC |
|---|---|---|---|---|---|---|
| Traditional Baselines | ||||||
| TF-IDF (raw) | Centroid | 0.786 | 0.884 | 0.659 | 0.755 | – |
| TF-IDF (raw) | Random Forest | 0.832 | 0.842 | 0.816 | 0.829 | 0.906 |
| Word2Vec (Google) | XGBoost | 0.832 | 0.840 | 0.818 | 0.829 | 0.908 |
| GloVe (100d) | XGBoost | 0.832 | 0.840 | 0.818 | 0.829 | 0.908 |
| Transformer Baselines | ||||||
| Model inputs | Classifier | F1-score | Improvement ( ) vs. Emb-only |
|---|---|---|---|
| Features only | Random Forest | 0.821 | – |
| OpenAI emb (3072d) | XGBoost | 0.864 | – |
| OpenAI emb (1000d) + Features | XGBoost | 0.909 | +4.5 points |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| No. | Name | Description |
|---|---|---|
| 1 | Character count | Number of characters, including whitespaces |
| 2 | Word count | Number of words |
| 3 | Mean word length | Average length of words |
| 4 | Common words ratio | Percentage of stop words |
| 5 | Capital letters ratio | Percentage of uppercase characters |
| 6 | Capital words count | Number of fully capitalized words (excluding one-letter words) |