Recommendation systems have become central gatekeepers of online information, shaping user behaviour across a wide range of activities. In response, users increasingly organize and coordinate to steer algorithmic outcomes toward diverse goals, such as promoting relevant content or limiting harmful material, relying on platform affordances -- such as likes, reviews, or ratings. While these mechanisms can serve beneficial purposes, they can also be leveraged for adversarial manipulation, particularly in systems where such feedback directly informs safety guarantees. In this paper, we study this vulnerability in recently proposed risk-controlling recommender systems, which use binary user feedback (e.g., "Not Interested") to provably limit exposure to unwanted content via conformal risk control. We empirically demonstrate that their reliance on aggregate feedback signals makes them inherently susceptible to coordinated adversarial user behaviour. Using data from a large-scale online video-sharing platform, we show that a small coordinated group (comprising only 1% of the user population) can induce up to a 20% degradation in nDCG for non-adversarial users by exploiting the affordances provided by risk-controlling recommender systems. We evaluate simple, realistic attack strategies that require little to no knowledge of the underlying recommendation algorithm and find that, while coordinated users can significantly harm overall recommendation quality, they cannot selectively suppress specific content groups through reporting alone. Finally, we propose a mitigation strategy that shifts guarantees from the group level to the user level, showing empirically how it can reduce the impact of adversarial coordinated behaviour while ensuring personalized safety for individuals.
Figures & tables
Figure 1: Expected empirical risk of adversarial users at calibration time as a function of the filtering threshold λ∈[0,1] , for reporting rates γ∈{0.001,0.01,0.1} and different reporting strategies. The collective size is fixed to β=0.01 . Shaded areas indicate one standard deviation over 10 runs.
Figure 2: Expected performance reduction as a function of the fraction of adversarial users in the calibration set, β∈[0.001,0.1] , for different reporting rates γ∈{0.001,0.01,0.1} and reporting strategies. A reduction between 0 and 1 indicates an effect proportional to the size of the collective, whereas values greater than 1 indicate a disproportionate impact. Shaded areas denote confidence intervals obtained via bootstrapping over 10 runs.
Figure 3: Relationship between reporting intensity and risk reduction. A small fraction of carefully selected reported items in the calibration set ( Fig. 3(a) ) can induce a sharp reduction in expected unwanted content for non-adversarial users relative to the baseline ( None in Fig. 3(b) ). Across all panels, the collective size is fixed to β=0.01 and the reporting rate to γ=0.1 . We report the standard deviation over 10 runs as a shaded area or error bars.
Figure 4: Adversarial strategies substantially increase content repetition in recommendations, leading to degraded performance – up to 80% repeated items under the LowRisk strategy. The collective size is fixed to β=0.01 and the reporting rate to γ=0.1 . We report the standard deviation over 10 runs as error bars.
Figure 5: Difference in top- k exposure ( k=20 ) for group g=34 in Kuaishou . Comparing Random (γ) and Tag (g=34) , both strategies flag approximately 10% of items yet induce similar exposure reductions for non-adversarial users. Results are shown for collective sizes β∈{0.001,0.005,0.01} , with confidence intervals obtained via bootstrapping over 10 runs.
Figure 6: Difference ( Δ ) in performance between user-level and population-level thresholds in risk-controlling recommender systems at test time, under the LowRisk strategy. For nDCG @ 20 and Recall @ 20, a higher difference is better. For the previously seen items, a lower difference is better. Results are shown for reporting rates γ∈{0.001,0.01,0.1} , with confidence intervals obtained via bootstrapping over 10 runs. The collective size is fixed to β=0.01 .
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Exposure reduction for two item groups ( g∈{54,23} ) in Kuaishou . Although the two groups exhibit similar expected harmfulness, they differ in their risk score distributions: E[r(U,I)∣G=54]=0.604 and E[r(U,I)∣G=23]=0.576 ( Fig. 7(a) ). As a result, risk-control induces disparate exposure across the two groups ( Fig. 7(b) ), even in the absence of adversarial users .
College of Knowledge and Library Science, School of Informatics, Tsukuba University, Ibaraki, Japan · College of Engineering, Shibaura Institute of Technology; Tokyo, Japan