Modern music streaming platforms face a persistent tradeoff: exploiting familiar content versus driving the exploration of novel items. While users frequently desire discovery, they hesitate to select unknown artists over proven favorites. Providing transparent, natural language rationales that explain why an unexplored item is recommended lowers this barrier. However, while Large Language Models (LLMs) excel at this nuanced explainability, their real-time deployment is severely bottlenecked by prohibitive inference costs and computational overhead. In this paper, we present an industry case study of a decoupled recommendation architecture that successfully scales exploration without compromising latency. Our system isolates LLM inference asynchronously offline, pre-computing personalized candidate pools of undiscovered artists alongside tailored rationales. Large-scale online A/B experiments validate our design. We demonstrate that combining LLM-backed recommendations with these explanatory rationales significantly reduces the trust barrier for new content, yielding statistically significant improvements in both user exploration and overall engagement on the discovery surfaces.
Figures & tables
Figure 1. An example of the "Your daily discover" shelf in the YouTube Music app. The italicized text at the bottom provides a personalized rationale for the recommended track. A screenshot of the YouTube Music app showing a recommended song card in the "Your daily discover" shelf. The card features the song "Until I Found You" by Stephen Sanchez and Em Beihold. At the bottom of the card, an italicized personalized rationale reads: "Reinvents classic mid-century crooner vibes for the modern era like Laufey".
Figure 2. Overview of the decoupled two-phase recommendation architecture. An offline LLM asynchronously generates personalized discovery profiles containing novel candidates and rationales, which a low-latency online layer retrieves during live sessions. Diagram of the Decoupled LLM Architecture showing Asynchronous Offline Phase and Low-Latency Online Phase.
Figure 3. Prompt workflow for candidate and rationale generation. The LLM segments a user’s comprehensive listening history into cohesive taste clusters, utilizing these clusters as a foundation to nominate novel artists and generate personalized rationales. Diagram of Candidate Generation Prompt with User Input and LLM Processing Engine.
Figure 4. A multi-step LLM-as-a-Judge evaluation rubric detailing three phases: assessing taste cluster cohesion, scoring candidate semantic alignment, and a targeted feedback loop that triggers on low scores to provide alternatives and drive prompt tuning. Diagram of LLM-as-a-Judge processing.
Metric
Value
95% Confidence Interval
New Nominator Views
4.1%
N/A
New Nominator Uniqueness Rate
74.00%
N/A
YDD Shelf-Level Engagement
+22.43%
[16.93%, 27.93%]
YDD Shelf-Level Discovery Metric (Retention)
+8.07%
[0.90%, 15.24%]
Table 1. Impact of LLM Rationales on Key Discovery Metrics