Popularity Bias

Follow this topic Ask about this topic

Latest papers

14 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs
Aug 29, 2026Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani +2Popularity Bias Language Model Bias Evaluation
University of Innsbruck · Chungbuk National University
Soft Curriculum Learning for Optimizing Fresh and Generalized Recommendations
Aug 11, 2026Arnab Bhadury, Siyan Zheng, Anlan Yu +6Popularity Bias Long-Tail Recommendation
Google Vancouver, BC, Canada · Google Mountain View, CA, USA
Stay or Stray - A Dynamical Systems Viewpoint of Popularity Bias
Aug 11, 2026Sarvesh Shashidhar, Lankireddy Prabhat, Arpit Agarwal +3Popularity Bias Algorithmic Bias
Centre of Machine Intelligence & Data Science Indian Institute of Technology, Bombay Mumbai, Maharashtra, India · Electrical & Computer Engineering Dept. Georgia Institute of Technology Atlanta, GA, USA · Dept. of Computer Science & Engineering Indian Institute of Technology, Bombay Mumbai, Maharashtra, India +3
Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval
Jul 1, 2026Ivan Ji, Liuyi Hu, Harrison +6Popularity Bias Hard Negative Mining
Zihao · 1Meta · Xiangjun
PROBE-Web: An Interactive System for Probing Evaluation Landscapes of Knowledge Graph Completion Models
Jun 8, 2026Sooho Moon, Yunyong KoPopularity Bias KG Completion
Chung-Ang University Seoul, Korea
Generalized Rank-based Evaluation for Knowledge Graph Completion: Perspectives, Framework, and Analyses
Jun 8, 2026Sooho Moon, Jian Kang, Yunyong KoPopularity Bias KG Completion
Chung-Ang University, South Korea · Mohamed bin Zayed University of Artificial Intelligence, UAE
Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders
May 16, 2026Jun Yin, Bangguo Zhu, Peng Huo +5Recommender Systems Popularity Bias in Recommender Systems
School of Computer Science and Engineering, Central South University, Changsha, China · National Super Computing Center, Tianjin, China · Faculty of Data Science, City University of Macau, Macau, China +1
Pretraining Exposure Explains Popularity Judgments in Large Language Models
May 12, 2026Jamshid Mozafari, Bhawna Piryani, Adam JatowtLanguage Model Pretraining Language Model Bias Evaluation
University of Innsbruck Innsbruck, Austria
Debiasing Message Passing to Mitigate Popularity Bias in GNN-based Collaborative Filtering
May 11, 2026Md Aminul Islam, Ahmed Sayeed Faruk, Sourav Medya +1Graph Neural Networks Popularity Bias
University of Illinois Chicago
CRAB: Codebook Rebalancing for Bias Mitigation in Generative Recommendation
Apr 6, 2026Zezhong Fan, Ziheng Chen, Luyi Ma +5Popularity Bias in Recommender Systems Recommender Systems
Walmart Global Tech, Sunnyvale, USA · Stony Brook University, NY, USA, Stony Brook, USA
Aligning Recommendations with User Popularity Preferences
Apr 1, 2026Mona Schirmer, Anton Thielmann, Pola Schwöbel +4Popularity Bias Popularity Bias in Recommender Systems
University of Amsterdam, Amsterdam, Netherlands · Amazon Music, Berlin, Germany · Amazon Music, Seattle, USA
Hijacking online reviews: sparse manipulation and behavioral buffering in popularity-biased rating systems
Mar 16, 2026Itsuki Fujisaki, Kunhao YangPopularity Bias in Recommender Systems Recommender Systems
College of Knowledge and Library Science, School of Informatics, Tsukuba University, Ibaraki, Japan · College of Engineering, Shibaura Institute of Technology; Tokyo, Japan
The Diversity Paradox revisited: Systemic Effects of Feedback Loops in Recommender Systems
Feb 18, 2026Gabriele Barlacchi, Margherita Lalli, Emanuele Ferragina +3Popularity Bias Recommender Systems
Scuola Normale Superiore and Università di Pisa · Scuola Normale Superiore · Sciences Po +2
Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity
May 23, 2025Shiyu Ni, Keping Bi, Jiafeng Guo +1LLM Uncertainty Estimation LLM Hallucination Mitigation
CAS Key Lab of Network Data Science and Technology, ICT, CAS · State Key Laboratory of AI Safety · University of Chinese Academy of Sciences

Aug 29, 2026J/K move · Enter open · S save

Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs

Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Mahmoud Abdalla+1

University of Innsbruck · Chungbuk National University

Open paper AskPDF arXiv

Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound evaluation. Modern LLMs systematically prefer popular but incorrect options over less popular correct ones, a vulnerability we call popularity bias. This pattern aligns with confidence miscalibration: model confidence remains high even as accuracy collapses for popular options. To systematically isolate this phenomenon, we introduce PopMCQ, a benchmark with six controlled strategies that vary option popularity while keeping the correct answer fixed. In our most adversarial setting, where all distractors are more popular than the correct option, models choose popular but wrong answers 66% of the time. To mitigate this bias, we propose PopDebias, a lightweight inference-time correction that estimates and removes a popularity prior from model predictions. It requires no fine-tuning, is label-free at test time (using only a small calibration split for parameter fitting), and adds negligible computational cost. Experiments on 22 open-source LLMs (0.5B to 32B parameters) show consistent improvements, with accuracy gains up to 54.1 percentage points under strong popularity pressure. The code and data are available https://github.com/DataScienceUIBK/PopMCQ