Recommender systems rely heavily on heterogeneous behavioral feedback to infer user preference. Although abundant, these signals are imperfect measurements: the same observed behavior can arise from different underlying states, such as genuine enjoyment, passive consumption, or inattention. The challenge is especially acute in short-form video, where watch-based signals are strongly affected by measurement confounders such as video duration - the same watch time can imply different levels of preference for videos of different lengths, while ratio-based metrics can systematically favor short videos. As a result, optimizing raw engagement can amplify measurement artifacts rather than improving user value. We propose a Factorized Latent Value Model (FLVM) for measuring user preference from heterogeneous behavioral feedback. The model treats observed behaviors as noisy measurements of a low-dimensional, factorized latent value state and uses structured output heads to model heterogeneous feedback signals. A restricted baseline path captures predictable variation from measurement-confounding features such as video duration, user propensity, and session context, while a routed latent path estimates preference-relevant value advantage. The resulting latent value score can be integrated into an existing recommender system as a ranking feature or ranking score. On YouTube Shorts, a major short-form video platform, this model improves offline metrics and lifts a primary viewer enjoyment metric by 2.67% in online A/B tests.
Figures & tables
Figure 1. Factorized Latent Value Model. The encoder maps full impression features x to a factorized latent value state z=[zp,za,zs] . For each feedback reconstruction task k , a task-specific hard sparse mask mk restricts the latent input to decoder hk , which produces a logit-scale advantage δk . In parallel, a restricted baseline bk uses only measurement-confounding features xb⊆x to produce ℓkb . The final logit is ℓk=sg(ℓkb)+δk , and predictions are y^k=ok(ℓk) . The exact mask configuration is abstracted; the latent value score is scalarized from z for serving-time ranking. Architecture diagram of the Factorized Latent Value Model. Full impression features feed a latent value encoder, which produces three latent factors for consumption, engagement, and valence. A hard sparse routing module sends the latent state to several task-specific feedback reconstruction heads, while a restricted set of measurement-confounding features feeds matching baseline heads. Each reconstruction row combines a routed latent advantage with the corresponding stopped baseline logit to produce a final logit and prediction. A separate latent value score is derived from the factorized latent state for serving-time ranking.
Figure 2. Empirical validation of the two main design choices. (a) The baseline path absorbs duration confounding: predicted item watch time increases with video duration, while the served consumption score softplus(zp) stays roughly flat across duration buckets. (b) The three latent factors separate semantically: liked and disliked impressions are most separated along the valence factor zs , less so along the active-engagement factor za , and approximately invariant on the consumption factor zp . Two side-by-side charts. (a) Line plot showing predicted item watch time increasing approximately linearly across six video length buckets from 0-15 seconds to over 120 seconds, while softplus(z_p) remains roughly flat at around 3.5 across the same buckets. (b) Bar chart of three latent factors (consumption z_p, engagement z_a, valence z_s), each with two bars showing the relative shift of liked versus disliked impressions from the average user. The valence factor z_s shows the largest separation (positive for liked, negative for disliked); engagement z_a shows moderate separation in the same direction; consumption z_p is approximately invariant.
Signal
Metric
Production
FLVM
Item watch time
RMSE ↓
24.47
23.35
Completion
PR-AUC ↑
0.6898
0.6722
Like
PR-AUC ↑
0.2522
0.3190
Dislike
PR-AUC ↑
0.0038
0.0054
Table 1. Offline predictive performance versus the production multi-task baseline. PR-AUC (higher is better) for binary signals; RMSE (lower is better) for item watch time.
Signal
Baseline
Latent
Full
Completion
0.3065
0.6617
0.6722
Like
0.1860
0.0715
0.3190
Skip
0.1840
0.5016
0.5100
Dislike
0.0035
0.0035
0.0054
Table 2. Decomposition: PR-AUC of each path in FLVM. Baseline-only =ok(ℓkb) ; Latent-only =ok(δk) ; Full =ok(ℓk) . The two paths together exceed either alone.
With the rapid development of the Internet, users have increasingly higher expectations for the recommendation accuracy of online content consumption platforms. However, short videos often contain diverse segments, and users may not hold the same attitude toward all of them. Traditional binary-classification recommendation models, which treat a video as a single holistic entity, face limitations in accurately capturing such nuanced preferences. Considering that user consumption is a temporal process, this paper demonstrates that the timing of user actions can represent diverse intentions through statistical analysis and examination of action patterns. Based on this insight, we propose a novel modeling paradigm: Action-Aware Generative Sequence Network (A2Gen), which refines user actions along the temporal dimension and chains them into sequences for unified processing and prediction. First, we introduce the Context-aware Attention Module (CAM) to model action sequences enriched with item-specific contextual features. Building upon this, we develop the Hierarchical Sequence Encoder (HSE) to learn temporal action patterns from users' historical actions. Finally, through leveraging CAM, we design a module for action sequence generation: the Action-seq Autoregressive Generator (AAG). Extensive offline experiments on the Kuaishou's dataset and the Tmall public dataset demonstrate the superiority of our proposed model. Furthermore, through large-scale online A/B testing deployed on Kuaishou's platform, our model achieves significant improvements over baseline methods in multi-task prediction by leveraging sequential information. Specifically, it yields increases of 0.34% in user watch time, 8.1% in interaction rate, and 0.162% in overall user retention (LifeTime-7), leading to successful deployment across all traffic, serving over 400 million users every day.
Capturing user interests across extensive watch histories is critical for short-form video recommendation, yet scaling sequence length is limited by two bottlenecks: the semantic sparsity of atomic Video IDs and the quadratic computational complexity of Transformers. Traditional orthogonal Video IDs fail to capture content relationships and demand large embedding tables, while the quadratic complexity of self-attention restricts the maximum sequence length under strict industrial latency and resource constraints. In this work, we present a production-deployed framework for modeling ultra-long user behavior sequences at a billion-user scale. We first address the representation bottleneck by adopting content-native Semantic IDs. By utilizing depth-truncated, coarse-grained Semantic IDs, we shrink the embedding table size from corpus cardinality. This compact representation naturally generalizes to cold-start content through shared semantic prefixes. Second, to overcome the sequence scaling barrier, we introduce a Global-Aware Compression Transformer that leverages non-parametric temporal folding and unified global query integration to effectively condense the sequence, alleviating both the memory and computational bottlenecks of standard self-attention. Offline profiling on our computing infrastructure demonstrates an order-of-magnitude reduction in peak memory footprint and a drastic decrease in computational overhead. This efficiency gain enables supporting longer sequence lengths at an affordable cost in production, yielding substantial online gains in satisfied user engagement and satisfied content consumption in large-scale online A/B tests.
The contemporary media landscape is characterized by sensational short videos. While prior research examines the effects of individual multimodal features, the collective impact of multimodal features on viewer engagement with short videos remains unknown. Grounded in the theoretical framework of Message Sensation Value (MSV), this study develops and tests a computational model of MSV with multimodal feature analysis and human evaluation of 1,200 short videos. This model that predicts sensory and behavioral engagement was further validated across two unseen datasets from three short video platforms (combined N = 14,492). While MSV is positively associated with sensory engagement, it shows an inverted U-shaped relationship with behavioral engagement: Higher MSV elicits stronger sensory stimulation, but moderate MSV optimizes behavioral engagement. This research advances the theoretical understanding of short video engagement and introduces a robust computational tool for short video research.
Haoning Xue, Jingwen Zhang, Xiaohui Wang +2
Department of Communication, University of Utah · Department of Communication, University of California, Davis · Department of Media and Communication, City University of Hong Kong +1