Content Depth Matters in Short-Video Recommendation: Rethinking the Attention Economy

Liwei Deng; Jing Jiang; Zhiwei Li; Yang Wang; Guodong Long

cs.AIAug 14, 2026

Content Depth Matters in Short-Video Recommendation: Rethinking the Attention Economy

Authors: Liwei Deng, Jing Jiang, Zhiwei Li, Yang Wang, Guodong Long

Organizations: Australian Artificial Intelligence Institute, University of Technology Sydney · Evidence and Research, Department of Health, Disability and Ageing

Abstract

Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capture attention within seconds. These systems inherently favor shallow-content videos that are effective at attracting immediate attention. However, growing evidence suggests that prolonged exposure to such content may negatively affect users' cognitive engagement and mental well-being, raising concerns about the long-term societal impact of the short-video platform. To tackle this challenge, this paper introduces a new metric, the Content Depth Score (CDS), to quantify the content depth of short videos. CDS measures the extent to which a video is expected to stimulate higher-order cognitive processes, using a seven-level scale grounded in established theories of cognitive psychology and learning. As an initial step toward this vision, we present SCOPE-Bench, the first benchmark for content-depth evaluation in short-video recommendation. Built upon a large-scale open-source short-video dataset, SCOPE-Bench provides CDS annotations for 150K videos, enabling systematic evaluation of RSs from a cognitive-content perspective. Leveraging SCOPE-Bench, we evaluate 13 representative RSs and reveal a consistent preference for shallow-content videos. Moreover, we find that these algorithms recommending cognitively deep content are only marginally better than random selection, highlighting a previously overlooked limitation of existing recommendation objectives. Our code and datasets are available at https://liweidengdavid.github.io/SCOPE-Bench/.

Figures & tables

Figure 1: Evaluating 13 Recommender Systems (RSs) on both Engagement metric (X axis) and our Content-Depth metric (Y axis). The regions are split into three levels corresponding to content-depth metric: Low (blue), Medium (green), and High (red). The dashed line marks the content depth of a random selection baseline. Existing RSs perform competitively on the engagement metric while their content-depth metric is low and near to the random selection.

CDS Level	Rubric Label	Operational Criterion	Dual Process Theory	Bloom’s Taxonomy	SOLO Taxonomy
Level 0	Affect	Mainly evokes affect, humor, spectacle, or atmosphere.	System 1	–	–
Level 1	Point	Presents an isolated opinion, or label without explaining why or how.	System 2	Remember	Prestructural
Level 2	Concept	Defines or illustrates a single idea with simple background or explanation.	System 2	Understand	Unistructural
Level 3	Procedure	Shows how a method can be used in concrete cases, typically through multiple steps or illustrative examples.	System 2	Apply	Multistructural
Level 4	Mechanism	Explains mechanisms, variables, constraints, conditions, causal links, or system relationships.	System 2	Analyze	Relational
Level 5	Judgment	Weighs evidence or competing explanations, including limitations, uncertainty, or counterexamples.	System 2	Evaluate	Relational / Extended Abstract

Table 1: Seven-level scoring rubric for CDS and its approximate theoretical anchors. Each level is defined by an operational criterion for consistent annotation. A dash indicates that no direct theoretical correspondence is assigned. The rubric captures a progression from affective responses to increasingly complex reasoning and transferable model construction.

Figure 2: Overview of the scalable CDS annotation workflow, which applies the rubric grounded in theories of cognition and learning. A gold-standard subset is used to select the LLM evaluator best aligned with human judgments for large-scale labeling.

Dataset	#Users	#Items	#Inter.	Caption	Image	ASR	Sparsity
Full	10,000	153,561	1,007,746	100.00%	88.56%	88.56%	99.93%
Sampled	6,654	31,496	128,105	100.00%	99.26%	99.26%	99.94%

Table 2: Statistics and modality coverage of the dataset.

Figure 3: Overview of the content-depth-aware RS evaluation framework for jointly assessing engagement and content depth.

Panel of Figure 4 — Figure 4: Human annotation reliability analysis, showing high agreement in their CDS annotations.

Model	Spearman $\uparrow$	Kendall $\uparrow$	Pearson $\uparrow$	Exact $\uparrow$	MAE $\downarrow$
GLM-5.1	0.6658	0.6410	0.7663	71.67%	0.3337
GPT-5.5	0.6414	0.6108	0.7312	67.35%	0.4418
Gemini3.1-Pro	0.7270	0.7020	0.7683	76.71%	0.3097
Kimi-K2.6	0.7003	0.6714	0.7610	71.43%	0.3505
MiMo-V2.5-Pro	0.6356	0.6027	0.6944	67.35%	0.4538
Qwen3.7-Max	0.7177	0.6960	0.7934	78.03%	0.2605

Table 3: Comparison of LLM evaluators against human CDS judgments.

\uparrow

and

\downarrow

indicate that higher and lower values are better, respectively, and bold denotes the best result.

Figure 5: Distribution of CDS across all videos. Most videos fall into the low-CDS group, with few in the high-CDS group.

Rank	Low	Medium	High
1	Dance	Health	Finance
2	Comedy	Law	Military
3	Beauty	Science	History
4	Music	History	Law
5	Short Dramas	Finance	Science
6	Casual Videos	Real Estate	Information

Table 4: Top seven categories in each CDS group, ranked by proportions. Low-CDS categories differ from the others, while medium- and high-CDS categories largely overlap.

Figure 6: Lexicon-based enrichment across CDS levels. Each cell reports the relative enrichment of signal words at CDS levels, with positive and negative values indicating values above and below the overall average, respectively. The smallest enrichment value in each row is shown in underlined .

Variable	Association	Explained variance
Caption length	Spearman $\rho=0.0828^{*}$	$R^{2}=0.57\%$
Log-ASR length a	Spearman $\rho=0.2182^{*}$	$R^{2}=6.09\%$
Category	Cramér $V=0.2469^{*}$	$\eta^{2}=25.34\%$

Table 5: Associations between valid CDS (

s\neq\varnothing

) and surface-level input attributes. CDS is more strongly associated with ASR length and category than with caption length.

Perturbation	$N$	P( $\Delta s=1$ )	P( $\Delta s=2$ )	P( $\Delta s>2$ )
Low $\rightarrow$ Medium	50	0.00%	0.00%	0.00%
Low $\rightarrow$ High	50	0.00%	0.00%	0.00%
Medium $\rightarrow$ Low	60	6.67%	3.33%	0.00%
High $\rightarrow$ Low	46	13.04%	13.04%	0.00%

Table 6: Category counterfactual perturbation results measured by CDS shift magnitude.

\text{Source}\rightarrow\text{Target}

indicates perturbing videos from the Source CDS group to Target-associated categories. Category perturbations leave low-CDS videos unchanged and cause only limited one- or two-level shifts for medium- and high-CDS videos.

Perturbation	Group	$N$	$\delta\|a\|$	P( $\Delta s=1$ )	P( $\Delta s=2$ )	P( $\Delta s>2$ )
ASR length $\uparrow$	Low	50	+52.05%	2.00%	0.00%	0.00%
ASR length $\uparrow$	Medium	60	+51.39%	5.00%	0.00%	0.00%
ASR length $\downarrow$	Medium	60	-24.49%	8.33%	0.00%	0.00%
ASR length $\downarrow$	High	46	-27.35%	23.91%	15.22%	0.00%

Table 7: ASR counterfactual perturbation results measured by CDS shifts.

\Delta s_{i}

and

\delta|a_{i}|

denote the absolute CDS change and relative ASR length change, respectively. CDS remains largely stable under lengthening and changes modestly under shortening, with no shifts exceeding two levels.

Method	ShortVideoSampled			ShortVideoFull
Method	R@20 $\uparrow$	A@20 $\uparrow$	E@20 $\uparrow$	R@20 $\uparrow$	A@20 $\uparrow$	E@20 $\uparrow$
ID-based Recommendation
BPR	3.30	6.90	6.85	2.22	7.06	7.10
NCF	3.01	6.49	6.42	2.09	7.46	7.53
LightGCN	3.54	7.36	7.33	2.37	6.89	7.08
Multimodal Recommendation

Table 8: Engagement and content-depth performance of baselines on two datasets. All metrics are scaled by

\times 100

, where R denotes Recall, A/E denote A-LCDS/E-LCDS, and the best results are bolded . Baselines are competitive on engagement but remain in the low-LCDS range and close to random recommendation on content-depth metric.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 516 papers

Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation
Sep 26, 2026Shuo Chang, Yueqi Wang, Zihuan Diao +7Recommender Systems User Preference Modeling
Google Mountain View, CA, USA · Google New York City, NY, USA · YouTube Google Mountain View, CA, USA
Quantifying the Engagement Trap: Impact of Short-form Video Recommender Systems on Users with ADHD
Sep 7, 2026Vedad Misirlic, Gregor Mayr, Elisabeth LexMental Health Human-Centered AI
Graz University of Technology, Austria
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
May 4, 2026Ruixiao Sun, Diego Uribe Mora, Zhimeng Jiang +7Semantic ID Recommendation Sequential Recommendation
Google Mountain View, USA

Sep 26, 2026J/K move · Enter open · S save

Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation

Shuo Chang, Yueqi Wang, Zihuan Diao, Ali Montazer+6

Google Mountain View, CA, USA · Google New York City, NY, USA · YouTube Google Mountain View, CA, USA

Recommender Systems User Preference Modeling Latent Variable Models

Open paper AskPDF arXiv

Recommender systems rely heavily on heterogeneous behavioral feedback to infer user preference. Although abundant, these signals are imperfect measurements: the same observed behavior can arise from different underlying states, such as genuine enjoyment, passive consumption, or inattention. The challenge is especially acute in short-form video, where watch-based signals are strongly affected by measurement confounders such as video duration - the same watch time can imply different levels of preference for videos of different lengths, while ratio-based metrics can systematically favor short videos. As a result, optimizing raw engagement can amplify measurement artifacts rather than improving user value. We propose a Factorized Latent Value Model (FLVM) for measuring user preference from heterogeneous behavioral feedback. The model treats observed behaviors as noisy measurements of a low-dimensional, factorized latent value state and uses structured output heads to model heterogeneous feedback signals. A restricted baseline path captures predictable variation from measurement-confounding features such as video duration, user propensity, and session context, while a routed latent path estimates preference-relevant value advantage. The resulting latent value score can be integrated into an existing recommender system as a ranking feature or ranking score. On YouTube Shorts, a major short-form video platform, this model improves offline metrics and lifts a primary viewer enjoyment metric by 2.67% in online A/B tests.

Abstract

Recommender Systems

Explore similar work

Mend the Measurement Gap: Latent User Preference Modeling for Short-Form Video Recommendation

Quantifying the Engagement Trap: Impact of Short-form Video Recommender Systems on Users with ADHD

Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling