cs.CVApr 1, 2026
SaveEmoScene: A Dual-space Dataset for Controllable Affective Image Generation
Organizations: Fudan University · East China Normal University
Abstract
Text-to-image diffusion models achieve high visual fidelity, yet fine-grained affective control remains difficult because textual emotion cues often fail to specify the visual perceptual factors underlying affective expression. Existing visual-affect datasets are likewise often limited to discrete labels, specific domains, or limited supervision of perceptual attributes. We introduce EmoScene, a large-scale dual-space dataset for controllable affective image generation, containing 1.2M images across more than 300 scene categories. Its affective space jointly represents discrete emotions and continuous valence--arousal--dominance (VAD), its perceptual space records measurable appearance attributes, and contextual descriptions ground both in scene semantics. EmoScene combines multi-model annotation with human-in-the-loop quality control. A random audit of 30,519 images yields 91.22% agreement on discrete emotion labels, while an independent multi-rater evaluation yields Fleiss' . After controlling for source and scene composition, affective dimensions and perceptual attributes exhibit stable associations across data sources, reflecting statistical tendencies rather than deterministic visual rules. To demonstrate the dataset's utility, we further develop AffectCtrl, which learns residuals in the conditioning space of frozen diffusion models to support categorical emotion generation and continuous control over VAD, brightness, and saturation. AffectCtrl achieves 85.75% categorical emotion accuracy, outperforms EmotiCrafter in valence and arousal control under a shared evaluation protocol, and obtains Pearson correlations of 0.673--0.765 across all five continuous axes. These results demonstrate that EmoScene provides a scalable data foundation for analyzing and controlling affective expression in visual generation.
Figures & tables
Figure 2: Motivation. EmoScene provides explicit VAD and brightness/saturation controls beyond underspecified emotion prompts.
| General Info | Affective Space | Perceptual Space | Contextual Semantics | |||||
|---|---|---|---|---|---|---|---|---|
| Dataset | Year | # Images | Emotion Type | VAD | Perceptual Attributes | Subject-level Anno. | Scene Anno. | Text/Lang. Anno. |
| ArtPhoto ( Machajdik and Hanbury 2010a ) | 2010 | 0.8k | 8 | - | - | - | - | - |
| Emotion6 ( Peng et al. 2015 ) | 2015 | 1.9k | 6+1 | V/A | - | - | - | - |
| FI ( You et al. 2016 ) | 2016 | 23k | 8 | - | - | - | - | - |
| IESN ( Zhao et al. 2016 ) | 2016 | 1000K | 8 | V/A/D | - | - | ✓ | ✓ |
| Emotic ( Kosti et al. 2017 ) | 2017 | 18k | 26 | V/A/D | - | ✓ (human) | ✓ | - |
Table 1: Comparison of representative emotion-related datasets and our proposed EmoScene. Columns distinguish its affective and perceptual representation spaces from the contextual annotations that ground them, while highlighting the generation-oriented design of the dataset. Abbreviations: #Images = number of labeled images; Perceptual Attributes include color, luminance, curvature, and complexity; Subject-level Anno. refers to annotations for specific entities(e.g., human or object); Text/Lang. Anno. indicates the presence of textual or language-level descriptions (e.g.,prompts).
Figure 3: Dual-Space Annotation Pipeline. EmoScene proceeds through image curation, contextual-semantic parsing, affective and perceptual annotation, and human-in-the-loop refinement. Full implementation details are provided in the supplementary material.
Figure 4: Representative examples of EmoScene’s dual-space annotations.
Figure 5: Valence–Arousal structure of EmoScene. Each emotion is represented by its class-normalized centroid and covariance ellipse in the VA plane. The overlap between ellipses reflects shared affective regions and label ambiguity, while the global layout reveals a circumplex-like organization across valence and arousal.
| Subset | Discrete (Acc./ ) | VAD (MSE / ) |
|---|---|---|
| Random (30,519) | 91.22%(Acc.) | 0.070 / 0.050 / 0.033(MSE) |
| Bias-test (297) | 85.00%(Acc.) | 0.181 / 0.102 / 0.084(MSE) |
| Multi-rater | 0.85 ( ) | 0.960 / 0.984 / 0.901( ) |
Table 2: Human verification of MLLM labels (HITL Audit). We report the agreement between MLLM-generated labels and human ground truth across three subsets. Our labels achieve “almost perfect” agreement ( ) and extremely low VAD error.
Figure 7: Overview of AffectCtrl. The categorical branch maps an EmoSpace embedding to an affective token and adds a brightness–saturation residual. The continuous branch adds axis-wise VAD/brightness/saturation residuals to neutral SDXL embeddings. Both generators remain frozen.
| Method | Emo-A | Conf. | Sem-C | ACS |
|---|---|---|---|---|
| SDXL direct prompt | 71.00 | 0.655 | 0.602 | 0.656 |
| FLUX.1 direct prompt | 73.25 | 0.672 | 0.577 | 0.661 |
| GPT-5.5 + SDXL | 71.75 | 0.657 | 0.677 | 0.684 |
| GPT-5.5 + FLUX.1 | 74.00 | 0.680 | 0.670 | 0.697 |
| CoEmoGen-style ( Yuan et al. 2026 ) † | 79.25 | 0.722 | 0.669 | 0.728 |
| EmoGen ( Yang, Feng, and Huang 2024 ) | 80.25 | 0.704 | 0.603 | 0.703 |
Table 3: Categorical generation under emotion-only control. † Paper-level CoEmoGen-style reproduction. Emo-A is reported as a percentage.
| Method | Valence | Arousal | Dominance | Brightness | Saturation |
|---|---|---|---|---|---|
| SDXL direct prompt | 1.905 1.398 | 1.849 1.453 | 2.041 1.396 | 1.623 0.923 | 1.360 0.908 |
| FLUX.1 direct prompt | 1.649 1.260 | 1.940 1.450 | 2.022 1.386 | 1.403 1.140 | 1.566 1.015 |
| GPT-5.5 + SDXL | 1.826 1.421 | 1.689 1.403 | 1.958 1.407 | 1.566 0.886 | 1.569 1.075 |
| GPT-5.5 + FLUX.1 | 1.649 1.314 | 1.734 1.387 | 1.903 1.340 | 1.327 1.008 | 1.515 1.019 |
| EmotiCrafter ( Dang et al. 2025 ) | 1.221 0.912 | 1.897 1.291 | N/A | N/A | N/A |
| AttriCtrl (FLUX.1) ( Chen et al. 2025 ) | N/A | N/A | N/A | 1.914 1.317 | N/A |
Table 4: Continuous control on 132 neutral prompts. EmotiCrafter supports VA only, AttriCtrl brightness only, and AffectCtrl the full VAD+perceptual interface. Errors use the normalized scale.
| Study | Protocol | Vote/Accept | Pair/Majority |
|---|---|---|---|
| Ours vs. EmoGen | 160 pairs, 3 raters | 74.7% | 82.5% |
| Ours vs. EmotiCrafter | 80 pairs, 4 raters | 80.6% | 77.5% |
| Dual-space accept | 100 images, 4 raters | 67.8% | 63.0% |
Table 5: Human evaluation of categorical, VA, and full dual-space controllability.
| Metric | Full | w/o | w/o | w/o |
|---|---|---|---|---|
| Avg. Corr. | 0.757 | 0.649 | 0.661 | 0.726 |
| Avg. MAE | 1.221 | 1.383 | 1.339 | 1.272 |
Table 6: Matched continuous-objective ablation averaged over V/A/D/B/S. Full is retrained separately from the checkpoint in Table 4 ; MAE uses the normalized scale.
Figure 8: Categorical comparison with EmoGen. Under matched seeds and sampling, AffectCtrl produces more target-specific cues for awe and disgust.
Figure 9: Five-axis control with AffectCtrl. Each row fixes the prompt and seed, fixes the other requested input controls at neutral values, and sweeps one axis from low to high, producing progressive changes while preserving recognizable content.
Figure S2: Human-in-the-loop verification UI. Left: the current image. Right: machine annotations to be verified: two discrete-emotion suggestions (e.g., from different annotators/models) and three V/A/D scores. The reviewer only selects Yes/No for each field. Selecting No requires a brief rationale in the comment box and moves the sample to the re-check queue; Yes accepts the value as final.
Figure S3: Pairing example. In the beach folder, the image ( .jpg ) and its annotation ( .json ) share the same <stem> ; the bottom block shows a truncated JSON corresponding to the image.
| Analysis cohort | Images | Scenes | Source scene cells | Selection role |
|---|---|---|---|---|
| Profile cohort | 18,085 | 342 | 1,292 | At most 2,500 per emotion; proportional stratified hash sample |
| Association cohort | 27,252 | 344 | 1,374 | At most 20 per eligible source scene cell |
Table S1: Frozen cohorts used by the scene-aware dataset analysis. Both cohorts are selected a priori by fixed hash from records with complete, protocol-compatible affective and perceptual annotations. Source scene cells with fewer than five records are excluded from the association cohort.
| Emotion | Brightness | Saturation | Hue warmth |
|---|---|---|---|
| Contentment | ; .002 | ; .002 | ; .883 |
| Amusement | ; .002 | ; .004 | ; .060 |
| Excitement | ; .878 | ; .002 | ; .057 |
| Awe | ; .993 | ; .002 | ; .017 |
| Anger | ; .019 | ; .206 | ; .183 |
| Fear | ; .002 | ; .973 | ; .002 |
Table S2: Exact scene-adjusted category profiles for the three color-related perceptual features. Entries are residual z-score ; BH-adjusted .
Table S3: Exact scene-adjusted category profiles for the three structural perceptual features. Entries are residual z-score ; BH-adjusted .
Figure S6: Robustness to source and scene composition. Pearson correlations are computed on the same frozen 27,252-image association cohort. (a) Raw correlations; (b) correlations after centering both variables by scene; (c) correlations after centering within each source scene cell. Panel (c) reproduces the estimates in main-paper Fig. 6(b) to provide a direct comparison across adjustment protocols. Stable signs and similar magnitudes indicate that the reported coupling is not an artifact of a few dominant sources or scene categories.
Figure S7: Cross-source robustness of scene-adjusted associations. The pooled source scene-adjusted correlations are followed by estimates computed independently within Flickr, Pexels, Pixabay, and Unsplash after scene centering. The four source panels are diagnostic replications rather than additional significance tests; formal scene-cluster intervals for the pooled analysis are reported in Table S4 .
Figure S8: Perceptual shifts between high and low VAD quartiles. The three panels show valence, arousal, and dominance from left to right. Bars report the upper-minus-lower VAD-quartile difference in source scene-adjusted perceptual feature z-score; error bars are 95% scene-cluster bootstrap intervals from 2,000 resamples. Asterisks require an interval excluding zero and BH-adjusted across all 18 comparisons. This difference-in-means view complements the continuous correlations summarized in the main paper.
Figure S9: Counterexamples to shallow visual heuristics. Eight challenging samples pair VAD scores with annotation rationales. The top row contains low-luminance scenes with positive or high-arousal affect; the bottom row contains bright or visually static scenes whose semantics convey negative affect or high arousal. The gallery illustrates that the scene-aware statistical associations are non-deterministic and should not be interpreted as rules such as “low luminance implies negative valence.”
| Method | V Corr. | A Corr. | D Corr. | B Corr. | S Corr. |
|---|---|---|---|---|---|
| EmotiCrafter (SDXL) ( Dang et al. 2025 ) | 0.701 | 0.138 | – | – | – |
| AttriCtrl (FLUX.1) ( Chen et al. 2025 ) | – | – | – | 0.657 | – |
| Ours | 0.765 | 0.673 | 0.756 | 0.762 | 0.718 |
Table S5: Supplementary Pearson correlation analysis for continuous control. EmotiCrafter exposes VA only; AttriCtrl (FLUX.1) exposes brightness control only.
Figure S10: Five-axis cross-axis response analysis. Rows denote the manipulated requested control and columns denote the measured VAD/HSV response; black outlines mark target-axis entries. The pooled matrix aggregates all prompts directly. The within-prompt matrix removes each prompt’s mean response across its five control levels, isolating fixed-content trajectories. Strong diagonal responses coexist with structured affect–perception coupling, so the result supports separately addressable target directions rather than complete disentanglement.
| Panel A: Pearson Correlation ( ) | ||||||
|---|---|---|---|---|---|---|
| Variant | V | A | D | B | S | Avg. |
| Full (matched) | 0.791 | 0.660 | 0.780 | 0.773 | 0.784 | 0.757 |
| w/o | 0.660 | 0.396 | 0.772 | 0.729 | 0.690 | 0.649 |
| w/o | 0.733 | 0.411 | 0.634 | 0.785 | 0.744 | 0.661 |
| w/o | 0.818 | 0.550 | 0.764 | 0.823 | 0.677 | 0.726 |
| Panel B: Normalized MAE ( ) | ||||||
Table S6: Matched ablation of the continuous objective. Panel A reports Pearson correlation between requested levels and evaluator predictions; Panel B reports MAE on the normalized scale. Averages are computed over V/A/D/B/S. Bold denotes the best value in each column.
| Panel A: Emotion Accuracy and Target-Emotion Confidence | ||||||||
|---|---|---|---|---|---|---|---|---|
| EmoGen | CoEmoGen-style † | EmoGen + EmoScene | Ours | |||||
| Emotion | Emo-A | Conf. | Emo-A | Conf. | Emo-A | Conf. | Emo-A | Conf. |
| Amusement | 96.00 | 0.879 | 96.00 | 0.835 | 100.00 | 0.924 | 90.00 | 0.850 |
| Awe | 90.00 | 0.810 | 98.00 | 0.956 | 92.00 | 0.818 | 98.00 | 0.867 |
| Contentment | 78.00 | 0.588 | 90.00 | 0.765 | 86.00 | 0.757 | 96.00 | 0.862 |
| Excitement | 92.00 | 0.816 | 90.00 | 0.845 | 86.00 | 0.725 | 82.00 | 0.798 |
Table S7: Per-emotion categorical comparison of learned methods under the common protocol of 50 images per emotion. Panel A reports eight-class emotion accuracy (Emo-A, in percentage) and target-emotion confidence (Conf.); Panel B reports semantic confidence (Sem-C) and ACS, the mean of Emo-A/100, Conf., and Sem-C. Bold indicates the best displayed value for each emotion and metric. † Our paper-level CoEmoGen-style reproduction, not an official result; the official authors did not release runnable training code or pretrained weights at the time of our experiments.
| Emotion | Emo-A | Conf. | Sem-C | ACS |
|---|---|---|---|---|
| Amusement | 22.00 | 0.225 | 0.593 | 0.346 |
| Awe | 52.00 | 0.430 | 0.527 | 0.492 |
| Contentment | 44.00 | 0.349 | 0.544 | 0.445 |
| Excitement | 56.00 | 0.458 | 0.603 | 0.540 |
| Anger | 22.00 | 0.196 | 0.605 | 0.340 |
| Disgust | 30.00 | 0.219 | 0.560 | 0.359 |
Table S8: Local reproduction of EmoCtrl under its native content-conditioned interface. The same 50 emotion-neutral content conditions are reused across all eight emotions, and all 400 images are scored with our common evaluators. Emo-A is reported in percentage. This result is kept separate from the emotion-only comparison because the input interfaces are not identical.
| Emotion | # Pairs | Valid Votes | Ours Votes | EmoGen Votes | Ours Vote Pref. | Ours Pair Win |
|---|---|---|---|---|---|---|
| Overall | 160 | 478 | 357 | 121 | 74.7% | 82.5% |
| Amusement | 20 | 60 | 42 | 18 | 70.0% | 85.0% |
| Anger | 20 | 59 | 51 | 8 | 86.4% | 100.0% |
| Awe | 20 | 60 | 50 | 10 | 83.3% | 100.0% |
| Contentment | 20 | 60 | 53 | 7 | 88.3% | 95.0% |
| Disgust | 20 | 60 | 52 | 8 | 86.7% | 95.0% |
Table S9: Human preference study for categorical emotion generation against EmoGen. Pair win is computed by majority vote over annotators for each image pair.
Figure S13: Discrete emotion generation across eight target categories. We show four representative AffectCtrl generations for each requested emotion: awe, amusement, contentment, excitement, disgust, anger, fear, and sadness. All images are generated by the full categorical branch, which combines an EmoScene-trained affective token with the perceptual residual adapter, using the common protocol (50 denoising steps and guidance scale 7.5). Semantic prompts and random seeds vary across samples to expose within-category scene diversity; the figure is a qualitative gallery rather than a paired baseline comparison.
Figure S14: Five-axis continuous dual-space control. Each sequence sweeps one requested axis from low to high (left to right) over the nominal normalized levels , while the remaining four requested inputs are fixed at their neutral settings. Prompts and seeds are fixed within each row. AffectCtrl produces progressive target-axis trajectories over valence, arousal, dominance, brightness, and saturation without implying that the measured responses are fully independent.
Figure S15: Extended EmoScene Dataset Gallery (Part 1/3). The three gallery figures cover eight target emotions plus Neutral, with VAD scores, scene labels, and captions that expose intra-class semantic and perceptual diversity. This first part shows Amusement , Anger , and Awe across social interactions, urban conflicts, and natural landscapes. Dominance varies with semantic context, including low dominance in many awe scenes and a wider range for anger.
Figure S16: EmoScene Dataset Gallery (Part 2/3). Intra-class diversity for Contentment , Disgust , and Excitement . These variations highlight the dataset’s ability to ground distinct activation levels and valences across entirely different scenes—from the absolute tranquility and low arousal of Contentment to the peak dynamic energy of Excitement and the multifaceted environmental revulsion of Disgust .
Figure S17: EmoScene Dataset Gallery (Part 3/3). Intra-class diversity for Fear , Sadness , and Neutral . While Fear and Sadness capture intensely negative affective states with varying arousal, the Neutral category serves as a crucial baseline. Characterized by mid-range valence, arousal, and dominance (VAD 5/5/5), these neutral samples ground the dataset in everyday, purely functional, and emotionally detached semantics (e.g., mundane urban routines or utilitarian spaces).
More in this topic
Image Generation
Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.
See all 461 papers