Multi-Agent Video Recommenders: Evolution, Patterns, and Open Challenges
Organizations: Google LLC, Mountain View, USA · Google LLC, San Bruno, USA
Abstract
Video recommender systems are among the most popular and impactful applications of AI, shaping content consumption and influencing culture for billions of users. Traditional single-model recommenders, which optimize static engagement metrics, are increasingly limited in addressing the dynamic requirements of modern platforms. In response, multi-agent architectures are redefining how video recommender systems serve, learn, and adapt to both users and datasets. These agent-based systems coordinate specialized agents responsible for video understanding, reasoning, memory, and feedback, to provide precise, explainable recommendations. In this survey, we trace the evolution of multi-agent video recommendation systems (MAVRS). We combine ideas from multi-agent recommender systems, foundation models, and conversational AI, culminating in the emerging field of large language model (LLM)-powered MAVRS. We present a taxonomy of collaborative patterns and analyze coordination mechanisms across diverse video domains, ranging from short-form clips to educational platforms. We discuss representative frameworks, including early multi-agent reinforcement learning (MARL) systems such as MMRF and recent LLM-driven architectures like MACRec and Agent4Rec, to illustrate these patterns. We also outline open challenges in scalability, multimodal understanding, incentive alignment, and identify research directions such as hybrid reinforcement learning-LLM systems, lifelong personalization and self-improving recommender systems.
Figures & tables
| Pattern | Primary Evaluation Focus | Representative Metrics | Critical Failure Points & Risks |
| Hierarchical Orchestration (e.g., MMRF, MMAgentRec) | Orchestration Effectiveness: How well does the central agent integrate diverse sub-goals to optimize the primary system objective? | Main objective metric (e.g., WatchTime), contribution weights (from attentive mechanism), system-wide latency. | Coordinator Bottleneck: The central agent becomes a single point of failure. Conflicting Goals: Auxiliary agents may work at cross-purposes, harming the main objective. |
| Pipeline-based Modular (e.g., VRAgent-R1, MACRec) | End-to-End Task Quality: How well does the final output perform after passing through all sequential stages? | Quality of intermediate outputs, error propagation rate, end-to-end latency. | Compounding Errors and Brittleness: An error in an early agent (e.g., IP Agent) can degrade the entire chain. |
| User-Agent Collaboration (e.g., TKGPT) | User-Perceived Agency: Does the user feel in control and satisfied with the system’s response to their natural language commands? | User satisfaction (SUS scores), task success rate (from user studies), latency from command to feed update. | Misinterpretation: The system may misunderstand the user’s (often ambiguous) intent and make drastic, undesirable changes to recommendations. |
| User Simulation Ensemble (e.g., Agent4Rec) | Behavioral Fidelity: How accurately does the simulated agent population replicate the statistical properties of real human users? | KL divergence (or similar) between simulated and real interaction distributions; replication of known macro-effects (e.g., filter bubbles). | Lack of Generalization: Agents overfit to initialization data and fail to model novel behaviors. Prohibitive Cost: High computational overhead for running thousands of LLM agents. |
Recommender Systems
Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.
See all 514 papers