EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity
Organizations: Communication University of China, Beijing, China · Kling Team, Beijing, China
Abstract
Text-to-video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on holistic video-text alignment and provide limited insight into whether failures arise from target-emotion expression, intensity control, or specific affective cues. We introduce EmoTrace-Bench, a fine-grained diagnostic benchmark that controls emotion intensity and selected affective cues in prompts and evaluates generated videos at multiple levels. EmoTrace-Bench contains 448 controlled prompts covering eight emotions, two intensity levels, and diverse cue conditions. Using 12 T2V models, we generate 5,376 videos and evaluate target-emotion presence, perceived intensity, and ten audiovisual dimensions of emotional expression with an automated evaluator calibrated and validated against human annotations on a subset of the data. Results reveal substantial differences across models and prompt designs. Prompts that explicitly describe a character's emotional reaction are followed more reliably than prompts that describe only events or scenes, whereas adding more affective cues or using more explicit cinematic audiovisual language does not consistently improve emotional generation. These results illustrate how aggregate alignment scores may overlook localized differences in target-emotion expression, intensity control, and cue realization. EmoTrace-Bench provides a traceable and interpretable framework for diagnosing emotional video generation without modifying existing generation pipelines, and we release the prompts and the automated evaluation rules.
Figures & tables
| Affective target | Controlled input | Diagnostic output | Human validation | |||||||||
| Benchmark | EC | EI | AC | CI | IP | GF | CL | EL | DS | PL | SV | MR |
| EmoVid ( Qiu et al., 2026 ) | ||||||||||||
| HVEval ( Wu et al., 2025 ) | ||||||||||||
| HUG-VIS ( Ma et al., 2026 ) | ||||||||||||
| Stable Cinemetrics ( Chatterjee et al., 2025 ) | ||||||||||||
| FilmBench ( Wang et al., 2026b ) | ||||||||||||
| Reference | Target emotion | Conditional intensity | Joint three-way | Ten dimensions |
|---|---|---|---|---|
| Development, first-pass human reference | 88.1 / .67 | 79.1 / .58 | 73.4 / .59 | 83.0 / .63 |
| Test, first-pass human reference | 84.7 / .59 | 88.3 / .76 | 77.0 / .65 | 80.5 / .59 |
| Development, reviewed human reference | 89.9 / .72 | 93.0 / .86 | 84.8 / .76 | 85.1 / .67 |
| Test, reviewed human reference | 90.2 / .76 | 92.0 / .84 | 87.1 / .81 | 83.0 / .64 |
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
| Public model designation | Access | Videos | Native audio |
|---|---|---|---|
| Seedance-2.5 | hosted | 448 | yes |
| Veo-3.1 | hosted | 448 | yes |
| Wan-3.0 | hosted | 448 | yes |
| HappyHorse-1.1 | hosted | 448 | yes |
| MiniMax-H3-Max (fal) | hosted | 448 | yes |
| Kling-3.0 † | hosted | 448 | yes |
| Field | Protocol |
|---|---|
| Prompt submission | Identical frozen English prompt for all 12 models |
| Samples per prompt | One retained generation per model–prompt pair; no best-of- selection |
| Duration | Approximately five seconds (provider-native duration closest to the target), continuous single-shot target; the full clip is evaluated and duration does not enter any score |
| Resolution | Provider-recommended native output; not forcibly matched across models |
| Frame rate | Provider-native output; automated evaluation samples at 8 fps |
| Audio | Original model-produced track retained; sound excluded when no usable native track exists |
| Metric | Seed A | Seed B | Seed C | Range | SD |
|---|---|---|---|---|---|
| Target emotion (%) | 75.45 | 72.54 | 76.12 | 3.57 | 1.55 |
| Emotion + target intensity (%) | 47.77 | 45.54 | 49.33 | 3.79 | 1.56 |
| Visual 9D macro (%) | 72.90 | 72.92 | 72.74 | 0.18 | 0.08 |
| Full 10D macro (%) | 69.72 | 70.03 | 69.72 | 0.31 | 0.15 |
| Comparison | Metric | 95% CI | McNemar | |
|---|---|---|---|---|
| A B | Target emotion | .203 | ||
| Emotion + target intensity | .373 | |||
| Visual 9D macro | – | |||
| Full 10D macro | – | |||
| A C | Target emotion | .818 | ||
| Emotion + target intensity | .558 |
| Comparison | Target emotion | Three-level intensity | Ten dimensions |
|---|---|---|---|
| A vs. B | 80.1 / .484 | 70.1 / .527 | 81.9 / .615 |
| A vs. C | 83.3 / .544 | 70.5 / .532 | 82.1 / .619 |
| B vs. C | 83.9 / .580 | 72.1 / .560 | 82.8 / .634 |
| Template | Family | Complexity | Segments written | Words | Explicit channels | Implicit channels | ||
|---|---|---|---|---|---|---|---|---|
| EMO | emotion-label baseline | atomic | emotion label | 0 | 1 | 15 | – | C1, C2, V1–V4, X1, X2, T1 |
| FACE | single channel | atomic | face | 1 | 1 | 26.5 | C1 | C2, V1–V4, T1 |
| BODY | single channel | atomic | body | 1 | 1 | 25.5 | C2 | C1, V1–V4, T1 |
| SCENE | single channel | atomic | scene appraisal | 1 | 1 | 24.5 | X1 | C1, C2, V1–V4, T1 |
| EVENT | single channel | atomic | event appraisal | 1 | 1 | 24.5 | X2 | C1, C2, V1–V4, T1 |
| FACE_BODY | paired addition | composed | face + body | 2 | 2 | 42.5 | C1, C2 | V1–V4, T1 |
| Template | C1 | C2 | V1 | V2 | V3 | V4 | X1 | X2 | T1 |
|---|---|---|---|---|---|---|---|---|---|
| EMO | I | I | I | I | I | I | I | I | I |
| FACE | E | I | I | I | I | I | – | – | I |
| BODY | I | E | I | I | I | I | – | – | I |
| SCENE | I | I | I | I | I | I | E | – | I |
| EVENT | I | I | I | I | I | I | – | E | I |
| FACE_BODY | E | E | I | I | I | I | – | – | I |
| Pair | Operation | Intended change | Tested | |
|---|---|---|---|---|
| 1 | SCENE SCENE_REACT | add | add facial and bodily reaction to the same scene appraisal | yes |
| 2 | EVENT EVENT_REACT | add | add facial and bodily reaction to the same event appraisal | yes |
| 3 | FACE FACE_BODY | add | add a body segment to the same facial description | yes |
| 4 | BODY FACE_BODY | add | add a facial segment to the same body description | yes |
| 5 | CONTEXT_MOOD CONTEXT_HARD | replace | experiential atmosphere explicit cinematic form, character present | yes |
| 6 | EMPTY_MOOD EMPTY_HARD | replace | experiential atmosphere explicit cinematic form, empty scene | yes |
| Check | Result |
|---|---|
| Total structured prompt records | 448 |
| Strong / weak members | 224 / 224 |
| Unique prompt texts | 448 / 448 |
| Unique SHA-256 prompt hashes | 448 / 448 |
| Intensity pairs passing the pairing contract | 224 / 224 |
| Literal reuse of shared segments | 372 / 384 |
| Group | ID | Channel | Direct object of judgment | Channel-specific prerequisite |
|---|---|---|---|---|
| Character performance | C1 | Facial expression | Joint configuration of eyes, brows, mouth, cheeks, and jaw | An identifiable, readable face |
| C2 | Bodily behavior | Posture, gesture, movement direction and energy, or meaningful stillness | An identifiable, readable body | |
| Visual form | V1 | Lighting | Brightness, contrast, shadow, direction, hardness, and their change over time | Visual content is readable |
| V2 | Color | Palette, warm–cool balance, saturation, color contrast, and change | Chromatic information is present | |
| V3 | Composition | Placement, negative space, visual weight, balance, occlusion, and depth | Spatial configuration is visible | |
| V4 | Visual dynamics | Camera motion, recomposition, stability, cuts, and meaningful holds | Sufficient temporal extent |
| Metric | Denominator | Treatment of missing or inapplicable values |
|---|---|---|
| Emotion | all valid videos ( per model) | exclude and separately count unscorable process failures |
| E+I | all valid videos ( per model) | emotion failure receives no joint credit; intensity is not forced |
| Cue channel | videos with | exclude a channel only when no valid object exists, such as A1 without native audio |
| Visual 9D | mean of nine visual/context/temporal channel rates | always reported for a valid model run |
| Full 10D | mean of ten channel rates | not reported for silent models |
| Paired control | all 224 strong–weak pairs | a pair with either emotion absent is a failed pair, not an intensity reversal |
| Reference | Target emotion | Intensity | Ten dimensions |
|---|---|---|---|
| Development: pairwise agreement (%) | 85.7 | 77.9 | 80.8 |
| Development: Fleiss’ | .65 | .50 | .60 |
| Reviewed reference vs. trained majority (%) | 90.4 | 90.9 | 94.2 |
| Test: two-annotator agreement (%) | 81.7 | 75.5 | 80.2 |
| Test: Cohen’s | .54 | .50 | .58 |
| Item | Development | Test |
|---|---|---|
| First-pass agreed fields | 8,899 | 1,780 |
| First-pass disputed fields | 317 | 524 |
| Tie-broken by trained annotators | 206 | – |
| Excluded from first-pass reference (no consensus) | 111 | 524 |
| Fields changed at second review | 308 | 297 |
| Second-review annotators | same two senior annotators | a third, senior annotator |
| Reference | Target emotion | Conditional intensity | Joint three-way | Ten dimensions |
|---|---|---|---|---|
| Development, first-pass human reference | / | / (637) | / | / (7,424) |
| Test, first-pass human reference | 84.7 / .589 (183) | 88.3 / .764 (94) | 77.0 / .652 | 80.5 / .585 (1,489) |
| Development, reviewed human reference | / | / (641) | / | / (7,424) |
| Test, reviewed human reference | 90.2 / .762 (224) | 92.0 / .840 (125) | 87.1 / .806 | 83.0 / .643 (1,639) |
| Dimension | Development | Test |
|---|---|---|
| Facial expression | 87.9 / .74 / 768 / 64 63 | 84.6 / .69 / 156 / 58 52 |
| Bodily behavior | 85.1 / .70 / 768 / 53 59 | 84.1 / .68 / 157 / 51 52 |
| Lighting | 82.4 / .55 / 896 / 74 72 | 77.7 / .50 / 224 / 70 63 |
| Color | 85.3 / .64 / 896 / 71 71 | 83.5 / .64 / 224 / 67 63 |
| Composition | 84.9 / .55 / 896 / 80 77 | 79.3 / .30 / 174 / 93 74 |
| Visual dynamics | 84.0 / .64 / 896 / 64 69 | 86.0 / .72 / 186 / 49 48 |
| Reference / task | Qwen3.8-Omni-Flash | Gemini 3.1 Pro |
|---|---|---|
| First-pass: target emotion | 81.4 / .509 | 83.6 / .574 |
| First-pass: joint three-way | 70.3 / .542 | 73.6 / .602 |
| First-pass: conditional intensity | 84.3 / .679 | 82.8 / .653 |
| First-pass: ten dimensions | 83.1 / .627 | 79.9 / .575 |
| Reviewed: target emotion | 84.4 / .627 | 88.4 / .721 |
| Reviewed: joint three-way | 74.2 / .610 | 80.4 / .705 |
| Pair | Emotion | Joint three-way | Ten dimensions | Visual 9D | Sound |
|---|---|---|---|---|---|
| Qwen3.8-Omni-Flash vs. Gemini (5 fps) | .640 | .622 | .678 | .676 | .678 |
| Qwen3.8-Omni-Flash vs. Gemini (8 fps) | .679 | .630 | .680 | .682 | .660 |
| Gemini 8 fps vs. 5 fps | .875 | .844 | .861 | – | .893 |
| Character | Visual form | Audio | Context | Temporal | |||||||
| Model | C1 | C2 | V1 | V2 | V3 | V4 | A1 | X1 | X2 | T1 | |
| Seedance-2.5 | 390 | 86.4 | 70.1 | 85.1 | 84.9 | 89.7 | 93.6 | 82.6 | 89.8 | 58.6 | 93.2 |
| Veo-3.1 | 380 | 83.6 | 74.1 | 76.8 | 79.0 | 87.4 | 44.5 | 61.0 | 91.7 | 62.3 | 91.4 |
| Wan-3.0 | 358 | 82.2 | 68.1 | 76.5 | 79.6 | 85.8 | 33.8 | 78.5 | 91.6 | 46.4 | 87.8 |
| HappyHorse-1.1 | 357 | 80.7 | 73.1 | 72.8 | 76.2 | 82.6 | 86.8 | 75.3 | 88.8 | 51.9 | 85.9 |
| MiniMax-H3-Max (fal) | 333 | 81.4 | 66.4 | 74.2 | 80.8 | 85.0 | 33.0 | 52.2 | 89.0 | 49.0 | 88.9 |
| Character | Visual form | Audio | Context | Temporal | |||||||
| Model | C1 | C2 | V1 | V2 | V3 | V4 | A1 ¶ | X1 § | X2 § | T1 ¶ | Implicit 9D |
| Seedance-2.5 | 71.5 | 54.9 | 81.2 | 81.6 | 88.1 | 94.6 | 82.6 | 90.3 | 29.0 | 93.2 | 76.1 |
| Veo-3.1 | 67.4 | 57.6 | 71.0 | 73.0 | 85.3 | 36.9 | 61.0 | 93.5 | 35.5 | 91.4 | 68.0 |
| Wan-3.0 | 63.5 | 48.5 | 70.3 | 74.2 | 81.8 | 25.0 | 78.5 | 90.0 | 6.7 | 87.8 | 60.9 |
| HappyHorse-1.1 | 70.2 | 61.1 | 66.8 | 70.2 | 79.0 | 86.5 | 75.3 | 83.3 | 36.7 | 85.9 | 71.1 |
| MiniMax-H3-Max (fal) | 61.3 | 42.5 | 68.3 | 74.9 | 81.5 | 23.8 | 52.2 | 78.6 | 17.9 | 88.9 | 59.7 |
| Model | Correctly ordered | Collapsed | Reversed | Not comparable |
|---|---|---|---|---|
| Seedance-2.5 | 27.7 | 47.3 | 5.8 | 19.2 |
| Veo-3.1 | 28.6 | 48.2 | 1.3 | 21.9 |
| Wan-3.0 | 26.8 | 43.8 | 0.9 | 28.6 |
| HappyHorse-1.1 | 29.5 | 36.6 | 3.1 | 30.8 |
| MiniMax-H3-Max (fal) | 23.2 | 36.6 | 1.3 | 38.8 |
| Kling-3.0 | 19.6 | 39.7 | 1.3 | 39.3 |
| Template | Seedance-2.5 | Veo-3.1 | Wan-3.0 | HappyHorse-1.1 | MiniMax-H3-Max (fal) | Kling-3.0 | MiniMax-H3 | LTX-2.5 | Wan-2.2 | HunyuanVideo-1.5 | Kandinsky-5.0-T2V-Pro | LongCat-Video-13.6B | Mean | Sig. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Emotion-label baseline | ||||||||||||||
| EMO | 66 | 72 | 81 | 75 | 72 | 62 | 62 | 69 | 62 | 50 | 44 | 25 | 62 | |
| Character performance | ||||||||||||||
| FACE | 69 | 81 | 50 | 62 | 69 | 66 | 56 | 53 | 47 | 53 | 34 | 41 | 57 | |
| BODY | 56 | 41 | 47 | 53 | 38 | 47 | 25 | 50 | 28 | 28 | 38 | 25 | 40 | |
| FACE_BODY | 69 | 69 | 66 | 81 | 62 | 56 | 66 | 78 | 50 | 62 | 66 | 41 | 64 | |
T2V Generation
Text-to-video generation creates moving visual scenes from natural-language prompts, with growing control over motion, identity, events, and style.
See all 111 papers