Recommendation systems have become central gatekeepers of online information, shaping user behaviour across a wide range of activities. In response, users increasingly organize and coordinate to steer algorithmic outcomes toward diverse goals, such as promoting relevant content or limiting harmful material, relying on platform affordances -- such as likes, reviews, or ratings. While these mechanisms can serve beneficial purposes, they can also be leveraged for adversarial manipulation, particularly in systems where such feedback directly informs safety guarantees. In this paper, we study this vulnerability in recently proposed risk-controlling recommender systems, which use binary user feedback (e.g., "Not Interested") to provably limit exposure to unwanted content via conformal risk control. We empirically demonstrate that their reliance on aggregate feedback signals makes them inherently susceptible to coordinated adversarial user behaviour. Using data from a large-scale online video-sharing platform, we show that a small coordinated group (comprising only 1% of the user population) can induce up to a 20% degradation in nDCG for non-adversarial users by exploiting the affordances provided by risk-controlling recommender systems. We evaluate simple, realistic attack strategies that require little to no knowledge of the underlying recommendation algorithm and find that, while coordinated users can significantly harm overall recommendation quality, they cannot selectively suppress specific content groups through reporting alone. Finally, we propose a mitigation strategy that shifts guarantees from the group level to the user level, showing empirically how it can reduce the impact of adversarial coordinated behaviour while ensuring personalized safety for individuals.
Figures & tables
Figure 1: Expected empirical risk of adversarial users at calibration time as a function of the filtering threshold λ∈[0,1] , for reporting rates γ∈{0.001,0.01,0.1} and different reporting strategies. The collective size is fixed to β=0.01 . Shaded areas indicate one standard deviation over 10 runs.
Figure 2: Expected performance reduction as a function of the fraction of adversarial users in the calibration set, β∈[0.001,0.1] , for different reporting rates γ∈{0.001,0.01,0.1} and reporting strategies. A reduction between 0 and 1 indicates an effect proportional to the size of the collective, whereas values greater than 1 indicate a disproportionate impact. Shaded areas denote confidence intervals obtained via bootstrapping over 10 runs.
Figure 3: Relationship between reporting intensity and risk reduction. A small fraction of carefully selected reported items in the calibration set ( Fig. 3(a) ) can induce a sharp reduction in expected unwanted content for non-adversarial users relative to the baseline ( None in Fig. 3(b) ). Across all panels, the collective size is fixed to β=0.01 and the reporting rate to γ=0.1 . We report the standard deviation over 10 runs as a shaded area or error bars.
Figure 4: Adversarial strategies substantially increase content repetition in recommendations, leading to degraded performance – up to 80% repeated items under the LowRisk strategy. The collective size is fixed to β=0.01 and the reporting rate to γ=0.1 . We report the standard deviation over 10 runs as error bars.
Figure 5: Difference in top- k exposure ( k=20 ) for group g=34 in Kuaishou . Comparing Random (γ) and Tag (g=34) , both strategies flag approximately 10% of items yet induce similar exposure reductions for non-adversarial users. Results are shown for collective sizes β∈{0.001,0.005,0.01} , with confidence intervals obtained via bootstrapping over 10 runs.
Figure 6: Difference ( Δ ) in performance between user-level and population-level thresholds in risk-controlling recommender systems at test time, under the LowRisk strategy. For nDCG @ 20 and Recall @ 20, a higher difference is better. For the previously seen items, a lower difference is better. Results are shown for reporting rates γ∈{0.001,0.01,0.1} , with confidence intervals obtained via bootstrapping over 10 runs. The collective size is fixed to β=0.01 .
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Exposure reduction for two item groups ( g∈{54,23} ) in Kuaishou . Although the two groups exhibit similar expected harmfulness, they differ in their risk score distributions: E[r(U,I)∣G=54]=0.604 and E[r(U,I)∣G=23]=0.576 ( Fig. 7(a) ). As a result, risk-control induces disparate exposure across the two groups ( Fig. 7(b) ), even in the absence of adversarial users .
Recommender systems operate as Black-Boxes, leaving users and regulators unable to steer their outputs toward specific intentions or audit their behavior. This lack of controllability, defined as the system's ability to respond to explicit guidance, remains an unaddressed dimension in existing evaluation paradigms. To fill this gap, we propose CtrlBench-Rec, a collaborative multi-agent framework for systematic assessment of controllability. We formalize three fundamental tasks: target content discovery, interest profile shaping, and popularity bias mitigation, which together measure steerability from explicit commands to implicit representation steering and finally to overcoming algorithmic biases.Extensive experiments on real-world datasets and multiple recommendation models demonstrate that our framework effectively quantifies controllability and exposes critical system bottlenecks, most notably persistent resistance to guiding long tail content. CtrlBench-Rec provides the first standardized toolkit for controllable recommendation research, algorithmic auditing, and user empowerment. Our code is released on https://github.com/caskcsg/CtrlBenchRec.
Jiwen Zhou, Xiang Liu, Mingming Li +5
Institute of Information Engineering, Chinese Academy of Sciences, China · School of Cyber Security, University of Chinese Academy of Sciences, China
Recommender systems have become core infrastructure for modern online platforms, personalizing content at scale and strongly influencing what users see, click on, and purchase. However, this dependence on user interaction also exposes them to shilling attacks, where malicious actors can inject fake profiles to distort item rankings and control visibility. Existing attacks often rely on target-specific fine-tuning or fixed profile templates, making them either difficult to adapt to different victims or easier to detect. To overcome these limitations, we propose the Agentic Group Attack System (AGAS), a coordinated shilling framework where a central Coordinator directs a group of role-switching worker agents to adaptively promote a target item across different victim families. The Coordinator dynamically adjusts the strategy when progress stalls or suppression signals increase, while workers pursue a shared objective and switch between active and inactive roles to avoid repetitive patterns. Under the same attack budgets and evaluation protocols, AGAS consistently surpasses strong baselines in target promotion while better preserving benign recommendation quality, weakening representative detectors, and achieving higher efficiency than prior attacks. These findings also emphasize that defending recommender systems may require mechanisms that can handle adaptive shilling campaigns, not just isolated fake-profile injections. Our code is available at https://github.com/phkhanhtrinh23/AGAS.
Quoc Viet Nguyen, Trinh Pham, Viet Huynh +4
Griffith University, Australia · Edith Cowan University, Australia · The University of Queensland, Australia +1
Online reviews and recommendation systems help users navigate overwhelming choice, but they are vulnerable to self-reinforcing distortions. This paper examines how a single malicious reviewer can exploit popularity-biased rating dynamics and whether behavioral heterogeneity in user responses can reduce the damage. We develop a minimal agent-based model in which users choose what to rate partly on the basis of currently displayed averages. We compare broad attacks that perturb many items with sparse attacks that selectively boost low-quality items and suppress high-quality items. Additional analyses not shown here indicate that sparse attacks are substantially more harmful than broad attacks because they better exploit popularity-based exposure. The main text then focuses on sparse attacks and asks how their effects change as the fraction of contrarian users increases. Three results stand out. First, attack-induced damage is strongest when prior honest reviews are scarce, revealing a transition from a fragile low-information regime to a more robust high-information regime. Second, sparse attacks are especially effective at artificially promoting low-quality items. Third, moderate contrarian diversity partially buffers these distortions, primarily by suppressing the rise of low-quality items rather than fully restoring high-quality items to the top. The findings suggest that recommendation robustness depends not only on attack detection and predictive accuracy, but also on review density, popularity feedback, and user response heterogeneity.
Itsuki Fujisaki, Kunhao Yang
College of Knowledge and Library Science, School of Informatics, Tsukuba University, Ibaraki, Japan · College of Engineering, Shibaura Institute of Technology; Tokyo, Japan