LLM-Augmented Digital Twin for Policy Evaluation in Short-Video Platforms

Haoting Zhang; Yunduan Lin; Jinghai He; Denglin Jiang; Zuo-Jun Shen; Zeyu Zheng

cs.AIMar 11, 2026

LLM-Augmented Digital Twin for Policy Evaluation in Short-Video Platforms

Authors: Haoting Zhang, Yunduan Lin, Jinghai He, Denglin Jiang, Zuo-Jun Shen, Zeyu Zheng

Organizations: University of California, Berkeley · The Chinese University of Hong Kong · New York University · The University of Hong Kong

Abstract

Short-video platforms are closed-loop, human-in-the-loop ecosystems where platform policy, creator incentives, and user behavior co-evolve. This feedback structure makes counterfactual policy evaluation difficult in production, especially for long-horizon and distributional outcomes. The challenge is amplified as platforms deploy AI tools that change what content enters the system, how agents adapt, and how the platform operates. We propose a large language model (LLM)-augmented digital twin for short-video platforms, with a modular four-twin architecture (User, Content, Interaction, Platform) and an event-driven execution layer that supports reproducible experimentation. Platform policies are implemented as pluggable components within the Platform Twin, and LLMs are integrated as optional, schema-constrained decision services (e.g., persona generation, content captioning, campaign planning, trend prediction) that are routed through a unified optimizer. This design enables scalable simulations that preserve closed-loop dynamics while allowing selective LLM adoption, enabling the study of platform policies, including AI-enabled policies, under realistic feedback and constraints.

Figures & tables

Figure 1: Illustration of Four-Twin Architecture

Figure 2: Interaction Twin behavioral simulation loop. The flowchart depicts the multi-phase microdynamics of a user-content encounter: a user processes a new video impression via (i) an immediate hook response, (ii) interest matching, and (iii) stochastic watch-time sampling. If engagement criteria are satisfied, engagement actions (Like, Share, Comment) are triggered, optionally invoking an LLM to generate natural-language content, before the agent swipes to continue the loop.

Strategy	Watch and View		Gifts		LLM Cost ($)
Strategy	Time (s)	Gini	Revenue	Gini	LLM Cost ($)
Heuristic (S0)	9.680 $\pm$ 0.114	0.953 $\pm$ 0.008	5491 $\pm$ 353	0.624 $\pm$ 0.058	0.00 $\pm$ 0.00
LLM Planner (S1)	9.668 $\pm$ 0.084	0.942 $\pm$ 0.016	5690 $\pm$ 484	0.584 $\pm$ 0.075	2.28 $\pm$ 1.83

Table 1: Creator-economy outcomes aggregated across all adoption rates and monetization regimes. Values are reported as mean

\pm

std.

Figure 3: Distribution of creator earnings. The simulation reproduces a heavy-tailed income distribution in which a small fraction of creators capture most revenue. The LLM planner (green) shifts density modestly toward the center relative to the heuristic baseline (grey), consistent with the reduction in Gift-Gini (0.62

\rightarrow

0.58).

Figure 4: Ecosystem effects of LLM Planner adoption. As the adoption rate of the strategic planner increases from 0% to 100%, we observe a clear decrease in income inequality (Gini coefficient, red line, left axis). Total ecosystem revenue shows a modest, non-monotonic increase (blue dashed line, right axis). This suggests that widespread access to optimization tools can democratize performance without harming the overall platform economy.

Governance	Watch (s)	Skip	H-Entropy (bits)	View-Gini	LLM ($)
None (S0)	9.656 $\pm$ 0.057	0.363 $\pm$ 0.005	4.469 $\pm$ 0.034	0.886 $\pm$ 0.023	2.30 $\pm$ 0.44
Rule-based (S1)	9.674 $\pm$ 0.079	0.363 $\pm$ 0.004	4.469 $\pm$ 0.034	0.893 $\pm$ 0.023	2.31 $\pm$ 0.45
LLM-assisted (S2)	9.785 $\pm$ 0.018	0.361 $\pm$ 0.001	4.469 $\pm$ 0.034	0.964 $\pm$ 0.003	2.80 $\pm$ 0.25

Table 2: Set 2 platform outcomes by governance strategy (27 runs). Values (mean

\pm

std) are averaged across 3 budget tiers per strategy.

Figure 5: Trend Lifecycle and Forecasting. The solid line tracks the trend score of a viral hashtag (e.g., #sustainable_living) over time, driven by user interaction volume (grey bars). The dashed line represents the LLM-generated forecast issued during the emergence phase, demonstrating the Platform Twin’s ability to anticipate viral peaks before they occur.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Vector Field	Dims	Function & Lifecycle
content_interests	50	Active Learner. Initialized uniformly ( $0.1$ ). This vector serves as the primary mutable representation of user taste, updated in real-time by the VideoFeedbackLearner based on watch history.
visual_pref audio_pref text_pref	512 128 768	Architectural Slots. Reserved for future multi-modal expansion. Currently initialized to None . The Interaction Twin employs a projection fallback mechanism (using content_interests ) when these specific modality vectors are empty.

Appendix

Table 3: User Agent Vector Specifications

Archetype	Duration (s)	Viral Potential	Statistical Signature
DANCE	$15\pm 5$	High ( $0.8$ )	High hook strength, music-synced execution.
COMEDY	$45\pm 15$	High ( $0.9$ )	Narrative setup required, high completion reward.
EDUCATIONAL	$55\pm 10$	Medium ( $0.5$ )	Information-dense, low re-watchability.
GAMING	$30\pm 10$	Medium ( $0.6$ )	High visual chaos, niche audience alignment.
LIFESTYLE	$25\pm 10$	Low ( $0.3$ )	Aesthetic focus, lower hook strength.
MUSIC	$20\pm 5$	High ( $0.7$ )	Audio-dominant, high share probability.

Appendix

Table 4: Content Twin Video Archetype Definitions

Condition	Watch (s)	Compl. Rate	Skip Rate	Sess. Len.
P0C0 (tmpl. pers., tmpl. capt.)	9.75 $\pm$ 0.11	0.34 $\pm$ 0.00	0.36 $\pm$ 0.00	21.00 $\pm$ 0.01
P0C1 (tmpl. pers., LLM capt.)	9.77 $\pm$ 0.09	0.34 $\pm$ 0.00	0.36 $\pm$ 0.00	21.01 $\pm$ 0.05
P1C0 (LLM pers., tmpl. capt.)	8.14 $\pm$ 0.09	0.28 $\pm$ 0.00	0.43 $\pm$ 0.00	20.99 $\pm$ 0.03
P1C1 (LLM pers., LLM capt.)	8.20 $\pm$ 0.05	0.28 $\pm$ 0.00	0.43 $\pm$ 0.00	21.00 $\pm$ 0.04

Appendix

Table 5: Illustrative ablation (200-step horizon): session-level metrics. Values are mean

\pm

std across 3 seeds.

Policy Evaluation

Policy evaluation estimates the outcomes, value, risks, and fairness of decision rules before or after deployment.

See all 55 papers

PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimization
Mar 20, 2026Renhong Huang, Ning Tang, Jiarong Xu +6Large Language Model-Based Social Simulation Policy Optimization
Zhejiang University, Hangzhou, China · Fudan University, Shanghai, China · HKUST, Hong Kong, China +3
LOLGORITHM: Funny Comment Generation Agent For Short Videos
Apr 9, 2026Xuan Ouyang, Bouzhou Wang, Senan Wang +3Text Generation Controllable Text Generation
University of New South Wales Sydney, Australia · University of Sydney Sydney, Australia · The University of Hong Kong Hong Kong, China +1
SYN-DIGITS: A Synthetic Control Framework for Calibrated Digital Twin Simulation
Apr 8, 2026Grace Jiarui Fan, Chengpiao Huang, Tianyi Peng +2Synthetic Control Language Model Calibration
Finance Division, Columbia Business School. · Department of IEOR, Columbia University · Decision, Risk, and Operations Division, Columbia Business School. +1

Mar 20, 2026J/K move · Enter open · S save

PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimization

Renhong Huang, Ning Tang, Jiarong Xu, Yuxuan Cao+5

Zhejiang University, Hangzhou, China · Fudan University, Shanghai, China · HKUST, Hong Kong, China+3

Large Language Model-Based Social Simulation Policy Optimization User Simulation Social Media Analysis

Open paper AskPDF arXiv

Social platforms serve as central hubs for information exchange, where user behaviors and platform interventions jointly shape opinions. However, intervention policies like recommendation and content filtering, can unintentionally amplify echo chambers and polarization, posing significant societal risks. Proactively evaluating the impact of such policies is therefore crucial. Existing approaches primarily rely on reactive online A/B testing, where risks are identified only after deployment, making risk identification delayed and costly. LLM-based social simulations offer a promising pre-deployment alternative, but current methods fall short in realistically modeling platform interventions and incorporating feedback from the platform. Bridging these gaps is essential for building actionable frameworks to assess and optimize platform policies. To this end, we propose PolicySim, an LLM-based social simulation sandbox for the proactive assessment and optimization of intervention policies. PolicySim models the bidirectional dynamics between user behavior and platform interventions through two key components: (1) a user agent module refined via supervised fine-tuning (SFT) and direct preference optimization (DPO) to achieve platform-specific behavioral realism; and (2) an adaptive intervention module that employs a contextual bandit with message passing to capture dynamic network structures. Experiments show that PolicySim can accurately simulate platform ecosystems at both micro and macro levels and support effective intervention policy.

Abstract

Policy Evaluation

Explore similar work

PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimization

LOLGORITHM: Funny Comment Generation Agent For Short Videos

SYN-DIGITS: A Synthetic Control Framework for Calibrated Digital Twin Simulation