Aligning Recommendations with User Popularity Preferences

Mona Schirmer; Anton Thielmann; Pola Schwöbel; Thomas Martynec; Giuseppe Di Benedetto; Ben London; Yannik Stein

cs.IRApr 1, 2026

Aligning Recommendations with User Popularity Preferences

Authors: Mona Schirmer, Anton Thielmann, Pola Schwöbel, Thomas Martynec, Giuseppe Di Benedetto, Ben London, Yannik Stein

Organizations: University of Amsterdam, Amsterdam, Netherlands · Amazon Music, Berlin, Germany · Amazon Music, Seattle, USA

Abstract

Popularity bias is a pervasive problem in recommender systems, where recommendations disproportionately favor popular items. This not only results in "rich-get-richer" dynamics and a homogenization of visible content, but can also lead to misalignment of recommendations with individual users' preferences for popular or niche content. This work studies popularity bias through the lens of user-recommender alignment. To this end, we introduce Popularity Quantile Calibration, a measurement framework that quantifies misalignment between a user's historical popularity preference and the popularity of their recommendations. Building on this notion of popularity alignment, we propose SPREE, an inference-time mitigation method for sequential recommenders based on activation steering. SPREE identifies a popularity direction in representation space and adaptively steers model activations based on an estimate of each user's personal popularity bias, allowing both the direction and magnitude of steering to vary across users. Unlike global debiasing approaches, SPREE explicitly targets alignment rather than uniformly reducing popularity. Experiments across multiple datasets show that SPREE consistently improves user-level popularity alignment while preserving recommendation quality.

Figures & tables

Systematized Concept	Measurement Instrument
(S1) Absolute popularity of the recommended set of items ${\mathcal{S}}$	Average Recommendation Popularity ( Abdollahpouri et al., 2019 ) $\displaystyle\text{ARP}({\mathcal{S}})=\frac{\sum_{i\in{\mathcal{S}}}s(i)}{\|{\mathcal{S}}\|}$
	Average Log Recommendation Popularity ( Lichtenberg et al., [n. d.] ) $\displaystyle\text{ALRP}({\mathcal{S}})=\frac{\sum_{i\in{\mathcal{S}}}\log s(i)}{\|{\mathcal{S}}\|}$
(S2) Concentration of recommendation frequency on some items in catalogue ${\mathcal{I}}$	Coverage or Aggregate Diversity ( Adomavicius and Kwon, 2011 ) $\displaystyle\text{Coverage}({\mathcal{S}},{\mathcal{I}})=\frac{\|{\mathcal{S}}\|}{\|{\mathcal{I}}\|}$
	Shannon Entropy ( Carnovalini et al., 2025 ) $\displaystyle\text{H}({\mathcal{S}},{\mathcal{I}})=-\sum_{i\in\mathcal{I}}\phi(i)\log\phi(i)$ , where $\phi(i)=\frac{\hat{s}(i)}{\sum_{j\in{\mathcal{I}}}\hat{s}(j)}$
	Herfindahl Index ( Deldjoo, 2024 ) $\displaystyle\text{HHI}({\mathcal{S}},{\mathcal{I}})=\sum_{i\in{\mathcal{I}}}\left(\frac{\hat{s}(i)}{\sum_{j\in{\mathcal{I}}}\hat{s}(j)}\right)^{2}$
	Gini Index ( Gini, 1921 ; Zeng et al., 2012 ; Braun et al., 2024 ) $\displaystyle\text{Gini}({\mathcal{S}},{\mathcal{I}})=\frac{\sum_{i\in{\mathcal{I}}}(2i-\|{\mathcal{I}}\|-1)\hat{s}(i)}{\|{\mathcal{I}}\|\sum_{i\in{\mathcal{I}}}\hat{s}(i)}$ , with $\hat{s}(i)$ ordered in ascending order

Table 1. Overview of systematized concepts and corresponding measurement instruments for the background concept of popularity bias. Popularity bias has been understood at the system level (S1), as a function of all recommendations

{\mathcal{S}}

; at the item level (S2), as a function of the item set

{\mathcal{I}}

and the recommendations

{\mathcal{S}}

; and at the user level (S3), as a function of the user

u

Figure 1. Problematic cases for the UPD metric regarding content validity: Left: Recommendations for users $u$ (—) and $u^{\prime}$ (- -) are positively and negatively biased, respectively. Nevertheless, UPD assigns the same popularity bias due to the symmetry of the underlying Jensen–Shannon divergence. Middle: UPD does not measure popularity bias within bins (red dashed lines). Right: Since UPD does not differentiate within bins, no popularity bias is reported despite different popularity variance in ${\mathcal{H}}_{u}$ and ${\mathcal{R}}_{u}$ .

Desideratum	PopLift ( Abdollahpouri et al., 2020 )	LogPopDiff ( Lichtenberg et al., [n. d.] )	UPD ( Abdollahpouri et al., 2021 )	PCE (ours)	Calib. Curve (ours)
(D1) Ordinality	✓	✓	$\times$	$\times$	✓
(D2) Cardinality	✓	✓	$\times$	✓	✓
(D3) Dispersion	$\times$	$\times$	$\times$	✓	✓
(D4) Scale-invariance	✓	$\times$	✓	✓	✓
(D5) Robustness to outliers	$\times$	$\times$	✓	✓	✓

Table 2. Comparison of popularity bias metrics that operationalize (S3), the popularity alignment between the user and the recommender, with respect to our desiderata.

Figure 2. Visualization of popularity quantile calibration . Left: In the toy example, the user’s popularity preference $p(s|u)$ is more widely spread than that of the recommender, $q(s|u)$ . Middle: Consequently, their CDFs diverge. PCE measures the difference between the user CDF $F_{p}(s)$ and the recommender CDF $F_{q}(s)$ at fixed quantiles $\tau\in\{0,0.2,0.4,0.6,0.8,1\}$ (orange line). Right: The resulting calibration curve deviates from the perfectly calibrated oracle diagonal indicating the extent of miscalibration.

Figure 3. Trade-off between performance (NDCG) and popularity alignment (PCE) across different datasets and mitigation baselines. Points represent different mitigation strengths. The plots show the vicinity of the base model (larger plots on the left) and the global trends (smaller plots on the right).

Dataset	#users	#items	#interact.	interact.	interact.
Dataset	#users	#items	#interact.	per user	per item
Foursquare	2,293	15,177	494,807	216	33
MovieLens-1m	6,040	3,416	999,611	166	292
MovieLens-20m	138,493	18,345	19,984,024	144	1089
RateBeer	12,221	53,674	2,631,157	215	49

Table 3. Dataset statistics

Figure 4. Average calibration curves across users. Except for fs-tky, where the base model is already well calibrated, the recommender tends to favor less popular items (black curve below the dashed gray identity line). SPREE (green) mitigates this bias by bringing the curve closer to the oracle identity. In contrast, baselines that reduce global popularity (PopSteer and IPR) increase this miscalibration, while Personalized Popularity (PP) may overcorrect by recommending only the most popular items in the user’s history.

Dataset	PCE@100 $(\downarrow)$			ALRP@100
	Base	SPREE	SPREE w/o $\hat{f}$	Base	SPREE	SPREE w/o $\hat{f}$
fs-tky	0.210	0.200 (-4%)	0.509 (+143%)	4.663	4.713 (+1%)	3.636 (-22%)
ml-1m	0.211	0.176 (-16%)	0.434 (+106%)	6.339	6.337 (-0%)	5.883 (-7%)
ml-20m	0.418	0.382 (-9%)	0.378 (-10%)	9.012	9.017 (+0%)	8.912 (-1%)
ratebeer	0.321	0.293 (-9%)	0.335 (+4%)	5.652	5.860 (+4%)	5.577 (-1%)

Table 4. Ablation of the steering strength based on each user’s estimated bias ( Equation 5 ): SPREE without an estimator ( Equation 3 ) reduces global popularity (S1) but may increase popularity alignment (S3). Using a bias estimator improves alignment while keeping global popularity roughly constant.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5. Linear probe accuracy across sequence positions $t$ and transformer blocks $l$ for ml-1m, ml-20m, and fs-tky. After the padded region, accuracy rapidly converges toward 100%, indicating that popularity is linearly separable in activation space. Popularity is most strongly encoded at the final sequence positions in the last block.

Dataset	MSE	$R^{2}$
fs-tky	$0.027\pm 0.002$	$0.182\pm 0.042$
ml-1m	$0.015\pm 0.003$	$0.425\pm 0.100$
ml-20m	$0.019\pm 0.002$	$0.638\pm 0.029$
ratebeer	$0.026\pm 0.000$	$0.223\pm 0.031$

Appendix

Table 5. Predictive performance of the bias estimator

\hat{f}

on a hold-out set of user activations.

Popularity Bias

See all 14 papers

Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders
May 16, 2026Jun Yin, Bangguo Zhu, Peng Huo +5Recommender Systems Popularity Bias in Recommender Systems
School of Computer Science and Engineering, Central South University, Changsha, China · National Super Computing Center, Tianjin, China · Faculty of Data Science, City University of Macau, Macau, China +1
Stay or Stray - A Dynamical Systems Viewpoint of Popularity Bias
Aug 11, 2026Sarvesh Shashidhar, Lankireddy Prabhat, Arpit Agarwal +3Popularity Bias Algorithmic Bias
Centre of Machine Intelligence & Data Science Indian Institute of Technology, Bombay Mumbai, Maharashtra, India · Electrical & Computer Engineering Dept. Georgia Institute of Technology Atlanta, GA, USA · Dept. of Computer Science & Engineering Indian Institute of Technology, Bombay Mumbai, Maharashtra, India +3
The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders
Jun 20, 2026Weiqin Yang, Yue Pan, Chongming Gao +4Spectral Regularization Sequential Recommendation
Zhejiang University · State Key Laboratory of Blockchain and Data Security, Zhejiang University · College of Computer Science and Technology, Zhejiang University +2

May 16, 2026J/K move · Enter open · S save

Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders

Jun Yin, Bangguo Zhu, Peng Huo, Ruochen Liu+4

School of Computer Science and Engineering, Central South University, Changsha, China · National Super Computing Center, Tianjin, China · Faculty of Data Science, City University of Macau, Macau, China+1

Recommender Systems Popularity Bias in Recommender Systems Generative Recommendation Popularity Bias

Open paper AskPDF arXiv

Recently, Generative Recommenders (GRs), characterized by a unified end-to-end framework, have exhibited astonishing potential in transforming the recommendation paradigm. Despite their effectiveness, we recognize that GRs are still susceptible to the long-standing issue of popularity bias that has pervaded the recommendation community. Although a few studies have attempted to extend traditional debiasing methods to GRs, their effectiveness is marginal, and the fundamental reason why GRs suffer from popularity bias remains under-explored. To bridge this gap, this study focuses on two core aspects in GRs: the optimization of generative framework and the item tokenization based on semantic index. Based on theoretical analyses, we identify that the severe popularity bias emerges from the confluence of a token-level optimization flaw and the undifferentiated property of item tokenization. Accordingly, this study develops a novel generative recommender system, called Ghost, by designing the asymmetric unlikelihood optimization and the skeleton-founded tokenization. Extensive empirical evaluations across three datasets, alongside multiple SOTA baselines, reveal that Ghost substantially alleviates popularity bias and promotes fairer recommendations, while incurring slight degradation to the overall recommendation utility.

Abstract

Popularity Bias

Explore similar work

Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders

Stay or Stray - A Dynamical Systems Viewpoint of Popularity Bias

The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders