Balancing Domestic and Global Perspectives: Evaluating Dual-Calibration and LLM-Generated Nudges for Diverse News Recommendation
Organizations: Grouplens Research, University of Minnesota, Minneapolis, Minnesota, United States · Northwestern University, Evanston, Illinois, United States
Abstract
In this study, we applied the ``personalized diversity nudge framework'' with the goal of expanding user reading coverage in terms of news locality (i.e., domestic and world news). We designed a novel topic-locality dual calibration algorithmic nudge and a large language model-based news personalization presentation nudge, then launched a 5-week real-user study with 120 U.S. news readers on the news recommendation experiment platform POPROX. With user interaction logs and survey responses, we found that algorithmic nudges can successfully increase exposure and consumption diversity, while the impact of LLM-based presentation nudges varied. User-level topic interest is a strong predictor of user clicks, while highlighting the relevance of news articles to prior read articles outperforms generic topic-based and no personalization. We also demonstrate that longitudinal exposure to calibrated news may shift readers' reading habits to value a balanced news digest from both domestic and world articles. Our results provide direction for future work on nudging for diverse consumption in news recommendation systems.
Figures & tables
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| ID | Question Text | Response Options | Phase |
| Q1 | I feel that it is important that a newsletter provides a variety of international and domestic news topics to its readers. | 5-point Likert (Strongly Disagree [SD] – Strongly Agree [SA]) | Pre & Post |
| Q2 | I feel that it is important that people read a variety of international and domestic news topics. | 5-point Likert (SD – SA) | Pre & Post |
| Q3 | I think that personalizing the text of news headlines and descriptions to my interests improves my news reading experience. | 5-point Likert (SD – SA) | Pre & Post |
| Q4 | I think that using artificial intelligence (AI) to personalize news previews to my interests is accurate and trustworthy. | 5-point Likert (SD – SA) | Pre & Post |
| Q5 | In recent newsletters you received, reflect on the relevance of the articles to your interests. About how many articles were relevant to your interests? | None, 1–3, 4–6, 7–9, All of them | Pre & Post |
| Q6 | In recent newsletters you received, reflect on the balance of international and domestic articles. I noticed… | A lot more intl., A little more intl., No difference, A little more domestic, A lot more domestic, I’m not sure | Pre & Post |
Recommender Systems
Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.
See all 514 papers