AffectWorld: Benchmarking Emotion-Driven Future World Prediction

Changhao Song; Yacong Chen; Hui Gao; Yazhou Zhang; Peng Zhang

cs.CVOct 13, 2026

AffectWorld: Benchmarking Emotion-Driven Future World Prediction

Authors: Changhao Song, Yacong Chen, Hui Gao, Yazhou Zhang, Peng Zhang

Organizations: School of Computer Science and Technology, Tianjin University, Tianjin, China · International Joint Institute of Tianjin University, Fuzhou, China

Abstract

The world encompasses both physical environments and human actors. Predicting its evolution requires understanding both physical laws and how emotions shape human actions. However, existing world model benchmarks primarily assess physical plausibility and perceptual quality, leaving emotion-driven world evolution largely untested.We introduce AffectWorld, a benchmark that evaluates the chain Emotion-Action-World State through three tasks: World-State Consistency, Emotion-Action Consistency, and Causal Consistency. It comprises 1,498 everyday video samples annotated with plausible emotions, action tendencies, and atomic world-state constraints. A counterfactual subset of 233 samples tests whether changing an agent's emotion under fixed scene and event conditions yields corresponding changes in actions and consequences. We define eight metrics across the three tasks and an additional counterfactual metric, combining automatic visual assessment with human-calibrated VLM/LLM judgments. Evaluations of five SoTA world models (e.g., Cosmos3) required world-state change scores of 0.100-0.280 and counterfactual emotion sensitivity scores of 0.027-0.070 on a [0, 1] scale. These results reveal substantial gaps in generating the consequences of emotion-driven actions and adapting future predictions to emotional interventions. AffectWorld provides a testbed for advancing world models that capture how emotions shape future physical and social states.

Figures & tables

Figure 1: AffectWorld illustrated with scenes from Avengers: Infinity War : (a) Thanos and Doctor Strange discuss the snap, (b) Anger over a loved one’s death drives Star-Lord to attack Thanos, allowing him to escape, (c) Thanos snaps his fingers, erasing half of all life, (Optional) Without Star-Lord’s outburst, the heroes might restrain Thanos.

Figure 2: Overview of the AffectWorld generation and evaluation process. A world model generates a future video from a history video, which is evaluated by three progressive tasks: world-state consistency, emotion-action consistency, and causal consistency. An optional counterfactual subtask, applicable to models that accept emotion conditions, alters only the agent’s emotion to test whether the generated future changes accordingly.

Figure 3: Taxonomy and distribution of (a) scene, (b) emotions, and (c) action tendencies.

Task 1: World-State Consistency
Metric	Video-to-Video (V2V)			Image-to-Video (I2V)
Metric	Cosmos3	Wan2.1-VACE	LTX-Video	HunyuanVideo-1.5	CogVideoX-5B
$S_{id}$	0.8336	0.7324	0.9282	0.7939	0.9538
$S_{depth}$	0.3241	0.3160	0.4032	0.1202	0.5456
$S_{temp}$	0.7182	0.7426	0.8491	0.6547	0.8988
$\mathbf{Score}_{T1}$	0.5136	0.4639	0.6308	0.3191	0.7390

Table 1: Main results on AffectWorld under video-to-video (V2V) and image-to-video (I2V) input settings.

Figure 4: Model performance across scene modes (Solo, Dyadic, Group) on three evaluation tasks.

Metric	Video-to-Video (V2V)			Image-to-Video (I2V)
Metric	Cosmos3	Wan2.1-VACE	LTX-Video	HunyuanVideo-1.5	CogVideoX-5B
$S_{action}^{o}$	0.4664	0.5852	0.4678	0.5896	0.5084
$S_{action}^{c}$	0.3246	0.3077	0.2791	0.3458	0.1903
$S_{causal}^{o}$	0.4992	0.5251	0.4324	0.5611	0.5044
$S_{causal}^{c}$	0.5193	0.5392	0.4512	0.4319	0.3327
$\mathrm{Sel}_{avg}$	0.2442	0.2630	0.2946	0.2750	0.2428

Table 2: Counterfactual emotion sensitivity on AffectWorld.

Figure 5: A downcast girl: reference future (top) and Cosmos-generated future (bottom).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Modality	Task	Phys.	Emo. Causal.	Gen.
Physical World Benchmarks
VideoPhy Bansal et al. (2025)	T2V	Physical commonsense	✓	✗	✓
WorldBench Upadhyay et al. (2026)	I2V	Intuitive physics	✓	✗	✓
WorldScore Duan et al. (2025)	Multi-type	World generation	$\circ$	✗	✓
World-in-World Zhang et al. (2026b)	Closed-loop	Embodied task success	✓	✗	✓
WorldArena Shang et al. (2026)	Embodied	Perception + function	✓	✗	✓

Appendix

Table 3: Comparison of AffectWorld with physical world benchmarks and affective and social understanding benchmarks. Phys.: physical understanding, Emo. Causal.: emotional causality, Gen.: evaluation of generated content. ✓: supported,

\circ

: partially supported, ✗: not supported.

Taxonomy	Category	Sub-categories
Scene	Solo	Daily Life, Personal Hobby
	Dyadic	Romantic, Education, Travel, Workplace, Healthcare, Personal Service, Consultation
	Group	Family Life, Friend Social, Sports, Leisure, Public Transit, Community, Shopping
Emotion	Sad	Guilty, Despair, Lonely, Bored, Disappointed, Awful
	Fear	Scared, Anxious, Insecure, Rejected
	Anger	Mad, Hurt, Threatened, Distant

Appendix

Table 4: Taxonomies for scene categories, emotions, and action tendencies.

Figure 6: Representative video samples across 16 scene categories. For each scene, we display several key frames extracted from the state_0 (history) video segment. The samples showcase the diversity of real-world scenarios from short-video platforms, covering Solo, Dyadic, and Group interaction modes across various everyday contexts including social activities, professional settings, and leisure scenarios.

Figure 7: Duration distributions of (a) source videos and (b) state_0 clips in the benchmark dataset.

Parameter	Value
Shared Semantic Evaluation
Judge runs	3
Judge temperature	0.3
Task 1: World-State Consistency
Sampling rate	2 fps
Maximum sampled frames	9

Appendix

Table 5: Core parameters and score mappings.

Evaluation Dimension	$\alpha\uparrow$	$\rho\uparrow$	MAE $\downarrow$
Event-Emotion ( $M_{emo}$ )	0.78	0.83	0.09
Emotion-Action ( $S_{action}$ )	0.74	0.80	0.11
Causal Consistency ( $S_{causal}$ )	0.72	0.77	0.12

Appendix

Table 6: Agreement values for semantic evaluation.

\alpha

: interval Krippendorff’s agreement among human raters,

\rho

: Spearman correlation between automatic scores and mean human scores, MAE: mean absolute error.

World Model Evaluation

World Model Evaluation measures whether learned simulators predict consistent, physically plausible, action-responsive futures that help agents plan and act.

See all 212 papers

HappyWorld-Bench
Sep 21, 2026Zhiqi Bai, Junai Cai, Yixin Chen +33World Models World Model Evaluation
Alibaba Token Hub, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI · Tsinghua University +2
World Agent: Can Language Models Keep a World Running?
Sep 26, 2026Weixing Chen, Weipeng Zhang, Nan An +2World Model Evaluation World Models
Sun Yat-sen University · South China University of Technology · Guangdong Key Laboratory of Big Data Analysis and Processing +2
What-If World: A Causal Benchmark for General World Models in Embodied Scenarios
May 26, 2026Kunlin Cai, Rui Song, Jinghuai Zhang +7Physical Consistency in Video Generation Causal World Models
1UCLA · University of Tennessee · 3Amazon

Sep 21, 2026J/K move · Enter open · S save

HappyWorld-Bench

Zhiqi Bai, Junai Cai, Yixin Chen, Jingrun Du+32

Alibaba Token Hub, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI · Tsinghua University+2

World Models World Model Evaluation Interactive World Models Embodied World Models Video World Models

Open paper AskPDF arXiv

Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, interaction, and modification. We introduce HappyWorld-Bench, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them. Our design is built on a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling, instantiated across three independent evaluation tracks: video world models, spatial world models, and embodied world models. HappyWorld-Bench comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases. Across all three tracks, we build and operate HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, which complement newly designed automated metrics that capture behavioral correctness. We evaluate 14 video world models, 9 spatial systems, and 8 embodied candidates under this unified framework. Results reveal remaining reliability gaps across all three tracks: video models exhibit reduced consistency during extended rollouts and revisits, spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution, and embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules. These findings highlight the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.

Abstract

World Model Evaluation

Explore similar work

HappyWorld-Bench

World Agent: Can Language Models Keep a World Running?

What-If World: A Causal Benchmark for General World Models in Embodied Scenarios