EmoScene: A Dual-space Dataset for Controllable Affective Image Generation

Li He; Longtai Zhang; Wenqiang Zhang; Yan Wang; Lizhe Qi

cs.CVApr 1, 2026

EmoScene: A Dual-space Dataset for Controllable Affective Image Generation

Authors: Li He, Longtai Zhang, Wenqiang Zhang, Yan Wang, Lizhe Qi

Organizations: Fudan University · East China Normal University

Abstract

Text-to-image diffusion models achieve high visual fidelity, yet fine-grained affective control remains difficult because textual emotion cues often fail to specify the visual perceptual factors underlying affective expression. Existing visual-affect datasets are likewise often limited to discrete labels, specific domains, or limited supervision of perceptual attributes. We introduce EmoScene, a large-scale dual-space dataset for controllable affective image generation, containing 1.2M images across more than 300 scene categories. Its affective space jointly represents discrete emotions and continuous valence--arousal--dominance (VAD), its perceptual space records measurable appearance attributes, and contextual descriptions ground both in scene semantics. EmoScene combines multi-model annotation with human-in-the-loop quality control. A random audit of 30,519 images yields 91.22% agreement on discrete emotion labels, while an independent multi-rater evaluation yields Fleiss' $κ=0.85$ . After controlling for source and scene composition, affective dimensions and perceptual attributes exhibit stable associations across data sources, reflecting statistical tendencies rather than deterministic visual rules. To demonstrate the dataset's utility, we further develop AffectCtrl, which learns residuals in the conditioning space of frozen diffusion models to support categorical emotion generation and continuous control over VAD, brightness, and saturation. AffectCtrl achieves 85.75% categorical emotion accuracy, outperforms EmotiCrafter in valence and arousal control under a shared evaluation protocol, and obtains Pearson correlations of 0.673--0.765 across all five continuous axes. These results demonstrate that EmoScene provides a scalable data foundation for analyzing and controlling affective expression in visual generation.

Figures & tables

Figure 2: Motivation. EmoScene provides explicit VAD and brightness/saturation controls beyond underspecified emotion prompts.

General Info			Affective Space		Perceptual Space	Contextual Semantics
Dataset	Year	# Images	Emotion Type	VAD	Perceptual Attributes	Subject-level Anno.	Scene Anno.	Text/Lang. Anno.
ArtPhoto ( Machajdik and Hanbury 2010a )	2010	0.8k	8	-	-	-	-	-
Emotion6 ( Peng et al. 2015 )	2015	1.9k	6+1	V/A	-	-	-	-
FI ( You et al. 2016 )	2016	23k	8	-	-	-	-	-
IESN ( Zhao et al. 2016 )	2016	1000K	8	V/A/D	-	-	✓	✓
Emotic ( Kosti et al. 2017 )	2017	18k	26	V/A/D	-	✓ (human)	✓	-

Table 1: Comparison of representative emotion-related datasets and our proposed EmoScene. Columns distinguish its affective and perceptual representation spaces from the contextual annotations that ground them, while highlighting the generation-oriented design of the dataset. Abbreviations: #Images = number of labeled images; Perceptual Attributes include color, luminance, curvature, and complexity; Subject-level Anno. refers to annotations for specific entities(e.g., human or object); Text/Lang. Anno. indicates the presence of textual or language-level descriptions (e.g.,prompts).

Figure 3: Dual-Space Annotation Pipeline. EmoScene proceeds through image curation, contextual-semantic parsing, affective and perceptual annotation, and human-in-the-loop refinement. Full implementation details are provided in the supplementary material.

Figure 4: Representative examples of EmoScene’s dual-space annotations.

Panel of Figure 5 — Figure 5: Valence–Arousal structure of EmoScene. Each emotion is represented by its class-normalized centroid and covariance ellipse in the VA plane. The overlap between ellipses reflects shared affective regions and label ambiguity, while the global layout reveals a circumplex-like organization across valence and arousal.

Subset	Discrete (Acc./ $\kappa$ )	VAD (MSE / $r$ )
Random (30,519)	91.22%(Acc.)	0.070 / 0.050 / 0.033(MSE)
Bias-test (297)	85.00%(Acc.)	0.181 / 0.102 / 0.084(MSE)
Multi-rater	0.85 ( $\kappa$ )	0.960 / 0.984 / 0.901( $r$ )

Table 2: Human verification of MLLM labels (HITL Audit). We report the agreement between MLLM-generated labels and human ground truth across three subsets. Our labels achieve “almost perfect” agreement (

\kappa=0.85

) and extremely low VAD error.

Figure 7: Overview of AffectCtrl. The categorical branch maps an EmoSpace embedding to an affective token and adds a brightness–saturation residual. The continuous branch adds axis-wise VAD/brightness/saturation residuals to neutral SDXL embeddings. Both generators remain frozen.

Method	Emo-A $\uparrow$	Conf. $\uparrow$	Sem-C $\uparrow$	ACS $\uparrow$
SDXL direct prompt	71.00	0.655	0.602	0.656
FLUX.1 direct prompt	73.25	0.672	0.577	0.661
GPT-5.5 + SDXL	71.75	0.657	0.677	0.684
GPT-5.5 + FLUX.1	74.00	0.680	0.670	0.697
CoEmoGen-style ( Yuan et al. 2026 ) †	79.25	0.722	0.669	0.728
EmoGen ( Yang, Feng, and Huang 2024 )	80.25	0.704	0.603	0.703

Table 3: Categorical generation under emotion-only control. † Paper-level CoEmoGen-style reproduction. Emo-A is reported as a percentage.

Method	Valence $\downarrow$	Arousal $\downarrow$	Dominance $\downarrow$	Brightness $\downarrow$	Saturation $\downarrow$
SDXL direct prompt	1.905 $\pm$ 1.398	1.849 $\pm$ 1.453	2.041 $\pm$ 1.396	1.623 $\pm$ 0.923	1.360 $\pm$ 0.908
FLUX.1 direct prompt	1.649 $\pm$ 1.260	1.940 $\pm$ 1.450	2.022 $\pm$ 1.386	1.403 $\pm$ 1.140	1.566 $\pm$ 1.015
GPT-5.5 + SDXL	1.826 $\pm$ 1.421	1.689 $\pm$ 1.403	1.958 $\pm$ 1.407	1.566 $\pm$ 0.886	1.569 $\pm$ 1.075
GPT-5.5 + FLUX.1	1.649 $\pm$ 1.314	1.734 $\pm$ 1.387	1.903 $\pm$ 1.340	1.327 $\pm$ 1.008	1.515 $\pm$ 1.019
EmotiCrafter ( Dang et al. 2025 )	1.221 $\pm$ 0.912	1.897 $\pm$ 1.291	N/A	N/A	N/A
AttriCtrl (FLUX.1) ( Chen et al. 2025 )	N/A	N/A	N/A	1.914 $\pm$ 1.317	N/A

Table 4: Continuous control on 132 neutral prompts. EmotiCrafter supports VA only, AttriCtrl brightness only, and AffectCtrl the full VAD+perceptual interface. Errors use the normalized

[-3,3]

scale.

Study	Protocol	Vote/Accept $\uparrow$	Pair/Majority $\uparrow$
Ours vs. EmoGen	160 pairs, 3 raters	74.7%	82.5%
Ours vs. EmotiCrafter	80 pairs, 4 raters	80.6%	77.5%
Dual-space accept	100 images, 4 raters	67.8%	63.0%

Table 5: Human evaluation of categorical, VA, and full dual-space controllability.

Metric	Full	w/o $L_{\mathrm{pool}}$	w/o $L_{\mathrm{axis}}$	w/o $L_{0}$
Avg. Corr. $\uparrow$	0.757	0.649	0.661	0.726
Avg. MAE $\downarrow$	1.221	1.383	1.339	1.272

Table 6: Matched continuous-objective ablation averaged over V/A/D/B/S. Full is retrained separately from the checkpoint in Table 4 ; MAE uses the normalized

[-3,3]

scale.

Figure 8: Categorical comparison with EmoGen. Under matched seeds and sampling, AffectCtrl produces more target-specific cues for awe and disgust.

Figure 9: Five-axis control with AffectCtrl. Each row fixes the prompt and seed, fixes the other requested input controls at neutral values, and sweeps one axis from low to high, producing progressive changes while preserving recognizable content.

Figure S2: Human-in-the-loop verification UI. Left: the current image. Right: machine annotations to be verified: two discrete-emotion suggestions (e.g., from different annotators/models) and three V/A/D scores. The reviewer only selects Yes/No for each field. Selecting No requires a brief rationale in the comment box and moves the sample to the re-check queue; Yes accepts the value as final.

Figure S3: Pairing example. In the beach folder, the image ( .jpg ) and its annotation ( .json ) share the same <stem> ; the bottom block shows a truncated JSON corresponding to the image.

Analysis cohort	Images	Scenes	Source $\times$ scene cells	Selection role
Profile cohort	18,085	342	1,292	At most 2,500 per emotion; proportional stratified hash sample
Association cohort	27,252	344	1,374	At most 20 per eligible source $\times$ scene cell

Table S1: Frozen cohorts used by the scene-aware dataset analysis. Both cohorts are selected a priori by fixed hash from records with complete, protocol-compatible affective and perceptual annotations. Source

\times

scene cells with fewer than five records are excluded from the association cohort.

Emotion	Brightness	Saturation	Hue warmth
Contentment	$+.125\,[+.093,+.159]$ ; .002	$+.152\,[+.116,+.185]$ ; .002	$-.003\,[-.034,+.029]$ ; .883
Amusement	$+.215\,[+.159,+.286]$ ; .002	$+.120\,[+.067,+.169]$ ; .004	$+.029\,[+.002,+.064]$ ; .060
Excitement	$-.007\,[-.051,+.042]$ ; .878	$+.168\,[+.108,+.246]$ ; .002	$+.048\,[+.004,+.094]$ ; .057
Awe	$0.000\,[-.045,+.046]$ ; .993	$+.253\,[+.199,+.309]$ ; .002	$-.064\,[-.108,-.018]$ ; .017
Anger	$-.136\,[-.500,-.060]$ ; .019	$-.135\,[-.364,+.121]$ ; .206	$-.059\,[-.447,+.033]$ ; .183
Fear	$-.796\,[-.971,-.631]$ ; .002	$-.004\,[-.120,+.112]$ ; .973	$-.270\,[-.398,-.148]$ ; .002

Table S2: Exact scene-adjusted category profiles for the three color-related perceptual features. Entries are residual z-score

[95\%\ \mathrm{CI}]

; BH-adjusted

q

Panel of Table 18 — Table S3: Exact scene-adjusted category profiles for the three structural perceptual features. Entries are residual z-score $[95\%\ \mathrm{CI}]$ ; BH-adjusted $q$ .

Figure S6: Robustness to source and scene composition. Pearson correlations are computed on the same frozen 27,252-image association cohort. (a) Raw correlations; (b) correlations after centering both variables by scene; (c) correlations after centering within each source

\times

scene cell. Panel (c) reproduces the estimates in main-paper Fig. 6(b) to provide a direct comparison across adjustment protocols. Stable signs and similar magnitudes indicate that the reported coupling is not an artifact of a few dominant sources or scene categories.

Figure S7: Cross-source robustness of scene-adjusted associations. The pooled source

\times

scene-adjusted correlations are followed by estimates computed independently within Flickr, Pexels, Pixabay, and Unsplash after scene centering. The four source panels are diagnostic replications rather than additional significance tests; formal scene-cluster intervals for the pooled analysis are reported in Table S4 .

Figure S8: Perceptual shifts between high and low VAD quartiles. The three panels show valence, arousal, and dominance from left to right. Bars report the upper-minus-lower VAD-quartile difference in source

\times

scene-adjusted perceptual feature z-score; error bars are 95% scene-cluster bootstrap intervals from 2,000 resamples. Asterisks require an interval excluding zero and BH-adjusted

q<.05

across all 18 comparisons. This difference-in-means view complements the continuous correlations summarized in the main paper.

Figure S9: Counterexamples to shallow visual heuristics. Eight challenging samples pair VAD scores with annotation rationales. The top row contains low-luminance scenes with positive or high-arousal affect; the bottom row contains bright or visually static scenes whose semantics convey negative affect or high arousal. The gallery illustrates that the scene-aware statistical associations are non-deterministic and should not be interpreted as rules such as “low luminance implies negative valence.”

Method	V Corr. $\uparrow$	A Corr. $\uparrow$	D Corr. $\uparrow$	B Corr. $\uparrow$	S Corr. $\uparrow$
EmotiCrafter (SDXL) ( Dang et al. 2025 )	0.701	0.138	–	–	–
AttriCtrl (FLUX.1) ( Chen et al. 2025 )	–	–	–	0.657	–
Ours	0.765	0.673	0.756	0.762	0.718

Table S5: Supplementary Pearson correlation analysis for continuous control. EmotiCrafter exposes VA only; AttriCtrl (FLUX.1) exposes brightness control only.

Figure S10: Five-axis cross-axis response analysis. Rows denote the manipulated requested control and columns denote the measured VAD/HSV response; black outlines mark target-axis entries. The pooled matrix aggregates all prompts directly. The within-prompt matrix removes each prompt’s mean response across its five control levels, isolating fixed-content trajectories. Strong diagonal responses coexist with structured affect–perception coupling, so the result supports separately addressable target directions rather than complete disentanglement.

Panel A: Pearson Correlation ( $\uparrow$ )
Variant	V	A	D	B	S	Avg.
Full (matched)	0.791	0.660	0.780	0.773	0.784	0.757
w/o $L_{\mathrm{pool}}$	0.660	0.396	0.772	0.729	0.690	0.649
w/o $L_{\mathrm{axis}}$	0.733	0.411	0.634	0.785	0.744	0.661
w/o $L_{0}$	0.818	0.550	0.764	0.823	0.677	0.726
Panel B: Normalized MAE ( $\downarrow$ )

Table S6: Matched ablation of the continuous objective. Panel A reports Pearson correlation between requested levels and evaluator predictions; Panel B reports MAE on the normalized

[-3,3]

scale. Averages are computed over V/A/D/B/S. Bold denotes the best value in each column.

Panel A: Emotion Accuracy and Target-Emotion Confidence
	EmoGen		CoEmoGen-style †		EmoGen + EmoScene		Ours
Emotion	Emo-A $\uparrow$	Conf. $\uparrow$	Emo-A $\uparrow$	Conf. $\uparrow$	Emo-A $\uparrow$	Conf. $\uparrow$	Emo-A $\uparrow$	Conf. $\uparrow$
Amusement	96.00	0.879	96.00	0.835	100.00	0.924	90.00	0.850
Awe	90.00	0.810	98.00	0.956	92.00	0.818	98.00	0.867
Contentment	78.00	0.588	90.00	0.765	86.00	0.757	96.00	0.862
Excitement	92.00	0.816	90.00	0.845	86.00	0.725	82.00	0.798

Table S7: Per-emotion categorical comparison of learned methods under the common protocol of 50 images per emotion. Panel A reports eight-class emotion accuracy (Emo-A, in percentage) and target-emotion confidence (Conf.); Panel B reports semantic confidence (Sem-C) and ACS, the mean of Emo-A/100, Conf., and Sem-C. Bold indicates the best displayed value for each emotion and metric. † Our paper-level CoEmoGen-style reproduction, not an official result; the official authors did not release runnable training code or pretrained weights at the time of our experiments.

Emotion	Emo-A $\uparrow$	Conf. $\uparrow$	Sem-C $\uparrow$	ACS $\uparrow$
Amusement	22.00	0.225	0.593	0.346
Awe	52.00	0.430	0.527	0.492
Contentment	44.00	0.349	0.544	0.445
Excitement	56.00	0.458	0.603	0.540
Anger	22.00	0.196	0.605	0.340
Disgust	30.00	0.219	0.560	0.359

Table S8: Local reproduction of EmoCtrl under its native content-conditioned interface. The same 50 emotion-neutral content conditions are reused across all eight emotions, and all 400 images are scored with our common evaluators. Emo-A is reported in percentage. This result is kept separate from the emotion-only comparison because the input interfaces are not identical.

Emotion	# Pairs	Valid Votes	Ours Votes	EmoGen Votes	Ours Vote Pref. $\uparrow$	Ours Pair Win $\uparrow$
Overall	160	478	357	121	74.7%	82.5%
Amusement	20	60	42	18	70.0%	85.0%
Anger	20	59	51	8	86.4%	100.0%
Awe	20	60	50	10	83.3%	100.0%
Contentment	20	60	53	7	88.3%	95.0%
Disgust	20	60	52	8	86.7%	95.0%

Table S9: Human preference study for categorical emotion generation against EmoGen. Pair win is computed by majority vote over annotators for each image pair.

Figure S13: Discrete emotion generation across eight target categories. We show four representative AffectCtrl generations for each requested emotion: awe, amusement, contentment, excitement, disgust, anger, fear, and sadness. All images are generated by the full categorical branch, which combines an EmoScene-trained affective token with the perceptual residual adapter, using the common $512\times 512$ protocol (50 denoising steps and guidance scale 7.5). Semantic prompts and random seeds vary across samples to expose within-category scene diversity; the figure is a qualitative gallery rather than a paired baseline comparison.

Figure S14: Five-axis continuous dual-space control. Each sequence sweeps one requested axis from low to high (left to right) over the nominal normalized levels $[-3,-1.5,0,1.5,3]$ , while the remaining four requested inputs are fixed at their neutral settings. Prompts and seeds are fixed within each row. AffectCtrl produces progressive target-axis trajectories over valence, arousal, dominance, brightness, and saturation without implying that the measured responses are fully independent.

Panel of Figure 32 — Figure S15: Extended EmoScene Dataset Gallery (Part 1/3). The three gallery figures cover eight target emotions plus Neutral, with VAD scores, scene labels, and captions that expose intra-class semantic and perceptual diversity. This first part shows Amusement , Anger , and Awe across social interactions, urban conflicts, and natural landscapes. Dominance varies with semantic context, including low dominance in many awe scenes and a wider range for anger.

Panel of Figure 33 — Figure S16: EmoScene Dataset Gallery (Part 2/3). Intra-class diversity for Contentment , Disgust , and Excitement . These variations highlight the dataset’s ability to ground distinct activation levels and valences across entirely different scenes—from the absolute tranquility and low arousal of Contentment to the peak dynamic energy of Excitement and the multifaceted environmental revulsion of Disgust .

Panel of Figure 34 — Figure S17: EmoScene Dataset Gallery (Part 3/3). Intra-class diversity for Fear , Sadness , and Neutral . While Fear and Sadness capture intensely negative affective states with varying arousal, the Neutral category serves as a crucial baseline. Characterized by mid-range valence, arousal, and dominance (VAD $\approx$ 5/5/5), these neutral samples ground the dataset in everyday, purely functional, and emotionally detached semantics (e.g., mundane urban routines or utilitarian spaces).

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 461 papers

Sep 21, 2026

Beyond Emotion Prompts: Fine-Grained Text-to-Image Generation Driven by Valence-Arousal-Dominance

Although text-to-image models can accurately depict subjects and scenes, creators still struggle to specify the fine-grained emotions an image should convey without rewriting its content description. Natural language can suggest emotions, but it offers no control scale with stable meanings and ordered intensities. We propose EMOTRANS, which transforms psychologically grounded valence-arousal-dominance (VAD) coordinates into generation conditions that are independent of the content text and modulated across denoising stages, making emotional style a finely adjustable creative variable. To support this goal, we construct EMOVAD, an art-painting dataset that pairs objective content descriptions with separately collected emotional ratings from multiple annotators. We also coordinate emotional expression and content preservation through dual-branch training with a shared model. Objective and human evaluations show that the framework improves the accuracy of three-dimensional emotion control and produces perceptible, orderable continuous changes while maintaining competitive text alignment and image quality. This work provides a practical emotion-driven approach to image generation that extends objective content depiction to fine-grained emotional adjustment.

Minglang Li, Yueyue Fang, Xieping Gao

College of Information Science and Engineering Hunan Normal University, Changsha, China

Conditional Image Generation Controllable Image Generation T2I Diffusion Models

Ask PDF

Jul 11, 2026

EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation

Emotion-aware artistic image generation requires an image to match the input prompt, follow the specified artistic style, and convey the target emotion. In this challenge, the main difficulty is that the visual and affective attributes available in the training data are not explicitly provided at test time. Without these attributes, the generator has to decide not only what to depict, but also how the target emotion should be expressed through color, lighting, brushwork, composition, line, and layout. This creates a control gap between the available test prompt and the fine-grained conditions needed for emotion-aware artistic generation. To bridge this gap, we propose EmoStyle, a Z-Image-based framework that converts the input prompt into a structured generation state. An LLM reasoner first predicts affective cues (valence-arousal, dominant emotion, and therapeutic-effect labels) and an aspect-ratio decision. Instead of using these predictions only as additional prompt text, we encode the affective fields into an affective condition vector and inject it into the denoising blocks through AdaLN-style modulation. This allows the inferred control variables to directly guide the generation of intermediate features. Since emotional expression is also style-dependent, we further train a dedicated LoRA adapter for each artistic style bucket and select the corresponding expert during inference, enabling the same affective cues to be rendered with bucket-specific priors for color, texture, brushwork, and composition. Finally, a lightweight VLM-guided candidate selection step ranks the generated images based on prompt alignment, style consistency, emotional expression, and visual quality. In Track 1 of the AffectiveArt Challenge 2026, our USTC_PI_LAB_TEAM submission achieved first place.

Dexiang Hong, Yijie Guo, Weidong Chen +4

University of Science and Technology of China HeFei, China · Harbin Institute of Technology, WeiHai WeiHai, China

Conditional Image Generation T2I Generation Controllable Image Generation

Ask PDF

Aug 6, 2026

EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation

Emotion shapes how viewers interpret a scene, yet existing video generators entangle global atmosphere, affect-bearing semantic cues, and temporal progression within a single text condition. We present EmoWorld, a framework that decouples these factors within a frozen flow-matching video diffusion transformer (Video DiT). A one-time preparation stage extracts layer-specific affect directions and a reusable cue library from geometry-preserving neutral and emotion-edited panoramas. At inference, Visual Atmosphere Steering (VAS) injects atmosphere directions into hidden states, Semantic Affective Steering (SAS) isolates a separately scalable prompt residual for semantic cues, and Temporal Affective Steering (TAS) interpolates endpoint residual fields across denoising and video time. On Wan2.2, VAS improves target-emotion alignment by 19% while reducing a temporal-fluctuation proxy by 48%; SAS improves target-emotion alignment by 37% and increases detected affect-bearing cues by 36%; and TAS improves transition monotonicity by 15% over the strongest baseline. EmoWorld is evaluated across 27 emotion categories in text-to-video and image-to-video settings, demonstrates portability across multiple Video-DiT backbones, and supports camera-conditioned composition without updating generator parameters.

Bingyuan Wang, Baistan Zhyldyzbekov, Kunyu Feng +1

The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology

Image-to-Video Generation Video Diffusion Models T2V Generation Controllable Video Generation Temporal Consistency in Video Generation

Ask PDF

Abstract

Image Generation

Explore similar work

Beyond Emotion Prompts: Fine-Grained Text-to-Image Generation Driven by Valence-Arousal-Dominance

EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation

EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation