Content Depth Matters in Short-Video Recommendation: Rethinking the Attention Economy
Authors: Liwei Deng, Jing Jiang, Zhiwei Li, Yang Wang, Guodong Long
Organizations: Australian Artificial Intelligence Institute, University of Technology Sydney · Evidence and Research, Department of Health, Disability and Ageing
Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capture attention within seconds. These systems inherently favor shallow-content videos that are effective at attracting immediate attention. However, growing evidence suggests that prolonged exposure to such content may negatively affect users' cognitive engagement and mental well-being, raising concerns about the long-term societal impact of the short-video platform. To tackle this challenge, this paper introduces a new metric, the Content Depth Score (CDS), to quantify the content depth of short videos. CDS measures the extent to which a video is expected to stimulate higher-order cognitive processes, using a seven-level scale grounded in established theories of cognitive psychology and learning. As an initial step toward this vision, we present SCOPE-Bench, the first benchmark for content-depth evaluation in short-video recommendation. Built upon a large-scale open-source short-video dataset, SCOPE-Bench provides CDS annotations for 150K videos, enabling systematic evaluation of RSs from a cognitive-content perspective. Leveraging SCOPE-Bench, we evaluate 13 representative RSs and reveal a consistent preference for shallow-content videos. Moreover, we find that these algorithms recommending cognitively deep content are only marginally better than random selection, highlighting a previously overlooked limitation of existing recommendation objectives. Our code and datasets are available at https://liweidengdavid.github.io/SCOPE-Bench/.
Figures & tables
Figure 1: Evaluating 13 Recommender Systems (RSs) on both Engagement metric (X axis) and our Content-Depth metric (Y axis). The regions are split into three levels corresponding to content-depth metric: Low (blue), Medium (green), and High (red). The dashed line marks the content depth of a random selection baseline. Existing RSs perform competitively on the engagement metric while their content-depth metric is low and near to the random selection.
CDS Level
Rubric Label
Operational Criterion
Dual Process Theory
Bloom’s Taxonomy
SOLO Taxonomy
Level 0
Affect
Mainly evokes affect, humor, spectacle, or atmosphere.
System 1
–
–
Level 1
Point
Presents an isolated opinion, or label without explaining why or how.
System 2
Remember
Prestructural
Level 2
Concept
Defines or illustrates a single idea with simple background or explanation.
System 2
Understand
Unistructural
Level 3
Procedure
Shows how a method can be used in concrete cases, typically through multiple steps or illustrative examples.
System 2
Apply
Multistructural
Level 4
Mechanism
Explains mechanisms, variables, constraints, conditions, causal links, or system relationships.
System 2
Analyze
Relational
Level 5
Judgment
Weighs evidence or competing explanations, including limitations, uncertainty, or counterexamples.
System 2
Evaluate
Relational / Extended Abstract
Table 1: Seven-level scoring rubric for CDS and its approximate theoretical anchors. Each level is defined by an operational criterion for consistent annotation. A dash indicates that no direct theoretical correspondence is assigned. The rubric captures a progression from affective responses to increasingly complex reasoning and transferable model construction.
Figure 2: Overview of the scalable CDS annotation workflow, which applies the rubric grounded in theories of cognition and learning. A gold-standard subset is used to select the LLM evaluator best aligned with human judgments for large-scale labeling.
Dataset
#Users
#Items
#Inter.
Caption
Image
ASR
Sparsity
Full
10,000
153,561
1,007,746
100.00%
88.56%
88.56%
99.93%
Sampled
6,654
31,496
128,105
100.00%
99.26%
99.26%
99.94%
Table 2: Statistics and modality coverage of the dataset.
Figure 3: Overview of the content-depth-aware RS evaluation framework for jointly assessing engagement and content depth.
Figure 4: Human annotation reliability analysis, showing high agreement in their CDS annotations.
Model
Spearman ↑
Kendall ↑
Pearson ↑
Exact ↑
MAE ↓
GLM-5.1
0.6658
0.6410
0.7663
71.67%
0.3337
GPT-5.5
0.6414
0.6108
0.7312
67.35%
0.4418
Gemini3.1-Pro
0.7270
0.7020
0.7683
76.71%
0.3097
Kimi-K2.6
0.7003
0.6714
0.7610
71.43%
0.3505
MiMo-V2.5-Pro
0.6356
0.6027
0.6944
67.35%
0.4538
Qwen3.7-Max
0.7177
0.6960
0.7934
78.03%
0.2605
Table 3: Comparison of LLM evaluators against human CDS judgments. ↑ and ↓ indicate that higher and lower values are better, respectively, and bold denotes the best result.
Figure 5: Distribution of CDS across all videos. Most videos fall into the low-CDS group, with few in the high-CDS group.
Rank
Low
Medium
High
1
Dance
Health
Finance
2
Comedy
Law
Military
3
Beauty
Science
History
4
Music
History
Law
5
Short Dramas
Finance
Science
6
Casual Videos
Real Estate
Information
Table 4: Top seven categories in each CDS group, ranked by proportions. Low-CDS categories differ from the others, while medium- and high-CDS categories largely overlap.
Figure 6: Lexicon-based enrichment across CDS levels. Each cell reports the relative enrichment of signal words at CDS levels, with positive and negative values indicating values above and below the overall average, respectively. The smallest enrichment value in each row is shown in underlined .
Variable
Association
Explained variance
Caption length
Spearman ρ=0.0828∗
R2=0.57%
Log-ASR length a
Spearman ρ=0.2182∗
R2=6.09%
Category
Cramér V=0.2469∗
η2=25.34%
Table 5: Associations between valid CDS ( s=∅ ) and surface-level input attributes. CDS is more strongly associated with ASR length and category than with caption length.
Perturbation
N
P( Δs=1 )
P( Δs=2 )
P( Δs>2 )
Low → Medium
50
0.00%
0.00%
0.00%
Low → High
50
0.00%
0.00%
0.00%
Medium → Low
60
6.67%
3.33%
0.00%
High → Low
46
13.04%
13.04%
0.00%
Table 6: Category counterfactual perturbation results measured by CDS shift magnitude. Source→Target indicates perturbing videos from the Source CDS group to Target-associated categories. Category perturbations leave low-CDS videos unchanged and cause only limited one- or two-level shifts for medium- and high-CDS videos.
Perturbation
Group
N
δ∣a∣
P( Δs=1 )
P( Δs=2 )
P( Δs>2 )
ASR length ↑
Low
50
+52.05%
2.00%
0.00%
0.00%
Medium
60
+51.39%
5.00%
0.00%
0.00%
ASR length ↓
Medium
60
-24.49%
8.33%
0.00%
0.00%
High
46
-27.35%
23.91%
15.22%
0.00%
Table 7: ASR counterfactual perturbation results measured by CDS shifts. Δsi and δ∣ai∣ denote the absolute CDS change and relative ASR length change, respectively. CDS remains largely stable under lengthening and changes modestly under shortening, with no shifts exceeding two levels.
Method
ShortVideoSampled
ShortVideoFull
R@20 ↑
A@20 ↑
E@20 ↑
R@20 ↑
A@20 ↑
E@20 ↑
ID-based Recommendation
BPR
3.30
6.90
6.85
2.22
7.06
7.10
NCF
3.01
6.49
6.42
2.09
7.46
7.53
LightGCN
3.54
7.36
7.33
2.37
6.89
7.08
Multimodal Recommendation
Table 8: Engagement and content-depth performance of baselines on two datasets. All metrics are scaled by ×100 , where R denotes Recall, A/E denote A-LCDS/E-LCDS, and the best results are bolded . Baselines are competitive on engagement but remain in the low-LCDS range and close to random recommendation on content-depth metric.
Recommender systems rely heavily on heterogeneous behavioral feedback to infer user preference. Although abundant, these signals are imperfect measurements: the same observed behavior can arise from different underlying states, such as genuine enjoyment, passive consumption, or inattention. The challenge is especially acute in short-form video, where watch-based signals are strongly affected by measurement confounders such as video duration - the same watch time can imply different levels of preference for videos of different lengths, while ratio-based metrics can systematically favor short videos. As a result, optimizing raw engagement can amplify measurement artifacts rather than improving user value. We propose a Factorized Latent Value Model (FLVM) for measuring user preference from heterogeneous behavioral feedback. The model treats observed behaviors as noisy measurements of a low-dimensional, factorized latent value state and uses structured output heads to model heterogeneous feedback signals. A restricted baseline path captures predictable variation from measurement-confounding features such as video duration, user propensity, and session context, while a routed latent path estimates preference-relevant value advantage. The resulting latent value score can be integrated into an existing recommender system as a ranking feature or ranking score. On YouTube Shorts, a major short-form video platform, this model improves offline metrics and lifts a primary viewer enjoyment metric by 2.67% in online A/B tests.
Shuo Chang, Yueqi Wang, Zihuan Diao +7
Google Mountain View, CA, USA · Google New York City, NY, USA · YouTube Google Mountain View, CA, USA
Short-form video platforms use recommender systems to maximize engagement through highly efficient personalized recommendations. However, the impact of these recommendations on users with ADHD compared to users without ADHD remains underexplored. Through this study, we introduce and operationalize the Engagement Trap, illustrating how recommender systems, while successfully optimizing for engagement, disproportionately disadvantage users with ADHD. This stratified study of 302 participants, recruited via the online platform Prolific, compares experiences between participants with and without ADHD. Our results show that while recommendations are perceived as relevant across groups, participants with ADHD report significantly higher levels of time blindness, post-usage regret, and emotional distress when consuming recommendations. Moreover, we collect feedback for several proof-of- concept, theoretical design interventions for neuro-inclusive design principles. These findings provide quantitative evidence of systemic differences in engagement-optimized recommender systems and highlight the unbalanced negative effects and interactions these systems create for participants with ADHD. We argue for neurodiversity-aware, human-centered design approaches that mitigate such algorithmic harms and support more equitable experiences.
Capturing user interests across extensive watch histories is critical for short-form video recommendation, yet scaling sequence length is limited by two bottlenecks: the semantic sparsity of atomic Video IDs and the quadratic computational complexity of Transformers. Traditional orthogonal Video IDs fail to capture content relationships and demand large embedding tables, while the quadratic complexity of self-attention restricts the maximum sequence length under strict industrial latency and resource constraints. In this work, we present a production-deployed framework for modeling ultra-long user behavior sequences at a billion-user scale. We first address the representation bottleneck by adopting content-native Semantic IDs. By utilizing depth-truncated, coarse-grained Semantic IDs, we shrink the embedding table size from corpus cardinality. This compact representation naturally generalizes to cold-start content through shared semantic prefixes. Second, to overcome the sequence scaling barrier, we introduce a Global-Aware Compression Transformer that leverages non-parametric temporal folding and unified global query integration to effectively condense the sequence, alleviating both the memory and computational bottlenecks of standard self-attention. Offline profiling on our computing infrastructure demonstrates an order-of-magnitude reduction in peak memory footprint and a drastic decrease in computational overhead. This efficiency gain enables supporting longer sequence lengths at an affordable cost in production, yielding substantial online gains in satisfied user engagement and satisfied content consumption in large-scale online A/B tests.