Encoded but Exposed: Language Embeddings can Reveal More Than They Should in Recommender Systems

Large language models (LLMs) used as listwise rerankers in recommendation systems suffer from position bias when serializing candidate sets into prompts. We show this order sensitivity creates an exploitable attack surface: an attacker can promote a label-0 target into the top-$k$ solely by reordering candidates, without changing item content, labels, or model parameters. We introduce $\mathrm{promo}@k$ to quantify this vulnerability, measuring the fraction of label-0 targets that can be elevated into top-$k$ rankings via permutation. Evaluating across three domains (MovieLens, Amazon Books, and Amazon Fashion), $\mathrm{promo}@5$ reaches up to 0.57 at an attack budget of $R$ = 50 orderings. Furthermore, ordinary permutation stability predicts vulnerability without running the attack. While a bidirectional T5 encoder scorer reduces exposure, permutation-consistency regularization and architectural invariance effectively mitigate it. Pointwise scoring avoids the bias issue but degrades ranking quality. These results demonstrate that input candidate order in listwise LLM reranking is a security-relevant attack vector. Code and data are available at https://github.com/geoz-lab/position_bias_attack.

Language models (LMs) are increasingly used in recommender systems to generate recommendations or coordinate recommendation tools. However, does their strong performance imply that serving-time LM inference is necessary? We challenge this seemingly reasonable but unverified assumption. To investigate this question, we introduce recovery-based mechanism auditing, which tests whether independent alternatives can achieve the accuracy of LM-based recommender systems without the audited inference mechanisms under controlled information and evaluation conditions. For generative recommendation, we propose LIME-Rec, which combines sequential, collaborative, and offline semantic information without autoregressive LM decoding. On three Amazon Reviews datasets (Beauty, Toys, and Sports), LIME-Rec achieves the highest reported mean in all 12 dataset-metric combinations, with up to a 15.2% relative improvement in Recall@10 over GRAM, an LM-based generative recommender. For agentic recommendation, we propose DART-Rec, which replaces LM-based tool control with deterministic fusion and fixed expert selection. On 1,000 Beauty users, given the same candidate items and expert scores, deterministic fusion achieves a 16.7% relative improvement in Recall@10 over a fine-tuned LM controller. With at most two tool calls, fixed expert selection matches the LM controllers' Hit@10 outcomes for every user in 17 of 18 comparisons. These findings demonstrate that strong recommendation performance does not necessarily require serving-time LM inference: under the evaluated task and information conditions, alternatives without the specified inference mechanisms can match or exceed the accuracy of their LM-based counterparts. To support reproducibility, code is available at https://github.com/Double-wk/LIME-Rec.

Early ranking stages in recommendation systems precompute item embeddings and cache them in-model for scoring within strict latency constraints. Because this cache exists only at serving time, outside the training loop, training and serving use different item representations, a structural discrepancy that limits quality and adds operational fragility. We show that co-designing the training and serving paths removes this representation discrepancy at its source. We introduce the memory layer, an in-model key-value embedding cache co-trained with the model: the item tower writes embeddings during training and the model reads them at serving, one source of truth for item representations by construction. Always-on embeddings cover items not yet cached, so every item receives a prediction, and the design consolidates three separate trainer-to-predictor update paths into a single self-contained pipeline. Deployed in production on Instagram Reels, the memory layer raises prediction coverage from 96% to 100%, improves embedding freshness from $O(5\text{ min})$ to $O(20\text{ s})$, and narrows the training-serving Normalized Entropy (NE) gap by up to 86%, yielding over $2\times$ recall for the freshest content and a 5-6% cold start engagement lift. Because embeddings are produced during training, the system needs no separate bulk-evaluation or publish-time recomputation, cutting training-and-publish computational cost by 30% at neutral serving computational cost.