What We Talk About When We Talk About Emotion Editing

Xingzu Zhan; Lin Gu; Ruogu Fang

cs.CVOct 9, 2026

What We Talk About When We Talk About Emotion Editing

Authors: Xingzu Zhan, Lin Gu, Ruogu Fang

Organizations: Department of Biomedical Engineering, Vanderbilt University, Nashville, Tennessee, USA · Research Institute of Electrical Communication, Tohoku University, Sendai, Japan

Abstract

Emotional image editing aims to change the emotion a scene conveys while preserving its essential content. Because the same scene can support different emotional interpretations, achieving an emotional goal requires visual changes grounded in the source context. Motivated by the Component Process Model, which links emotional responses to event appraisal, we investigate neural representations as a reference for these editing decisions. Using TRIBE v2, an fMRI encoding model, we analyze 400 source-target pairs across eight emotion categories. Hidden states from EmoEdit, EmoEditor, and AffectDelta correspond more strongly to predicted target-image responses in visual regions than in affect-related regions. This imbalance motivates ROI-ReFT, which combines source visual responses as a content reference with a predicted affective response change as an editing direction. Two interacting branches use both conditions to guide a frozen editor throughout generation. Predicted responses of the generated image supervise affective change and constrain visual drift. On a post hoc selected subset of 400 editing requests, ROI-ReFT achieves 91.75% target-emotion accuracy and 83.25% joint success, with the same output both matching the target emotion and reaching source-output DINO-I of at least 0.80. Our code and dataset will be publicly released.

Figures & tables

Figure 1: Emotional image editing requires connecting visual content to emotional meaning. (a) Conceptual illustration of cognitive reappraisal: women crying outside a church may suggest grief or joy under funeral or wedding interpretations ( Ochsner et al., 2002 ) . (b) Across 400 source–target pairs, hidden states from EmoEdit, EmoEditor, and AffectDelta show stronger correspondence with TRIBE-predicted target-image responses in both early and higher visual regions than in affect-related regions (Fisher $z$ scores). Appendix G provides the encoding protocol and controls for EmoEdit and EmoEditor.

Method	Emo-A $\uparrow$	$\Delta$ Target $\uparrow$	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$
LDAST ( Fu et al., 2022 )	30.500%	$-0.154$	0.508	0.672	0.462
SDEdit ( Meng et al., 2022 )	22.250%	$-0.229$	0.159	0.544	0.565
IP2P ( Brooks et al., 2023 )	53.500%	$+0.039$	0.948	0.739	0.270
AIF ( Weng et al., 2023 )	31.250%	$-0.141$	0.774	0.806	0.267
EmoEdit ( Yang et al., 2025 )	66.750%	$+0.169$	0.745	0.806	0.267
EmoEditor ( Lin et al., 2025 )	45.750%	$-0.032$	0.807	0.795	0.267

Table 1: Quantitative comparison of emotion editing and image preservation on the selected 400 editing requests. Arrows indicate whether higher or lower values are better. Best and second-best results in each column are bold and underlined, respectively, with ranks determined before rounding. Ties receive the same marking.

Figure 3: Joint Success Rate (JSR) across DINO-I thresholds on the same 400 selected editing requests. An output counts as successful only when its predicted emotion matches the target and its source–output DINO-I is at least $\tau$ . Higher curves indicate more outputs satisfying both requirements. ROI-ReFT achieves 83.250% JSR at $\tau=0.80$ (333 of 400 outputs).

Panel of Figure 4 — Figure 4: Qualitative comparison for eight target emotions. Each row shows the input image, the target emotion at left, and outputs from eight editing methods. Ours denotes ROI-ReFT with a ReFT write gain of 0.5, and AffectDelta is the base editor. IP2P denotes InstructPix2Pix.

Variant	Emo-A $\uparrow$	$\Delta$ Target $\uparrow$	DINO-I $\uparrow$	CLIP-I $\uparrow$	LPIPS $\downarrow$	JSR(0.80) $\uparrow$
ReFT w/o ROI	78.500%	$+0.231$	0.803	0.781	0.319	67.500%
Visual-only	83.000%	$+0.246$	0.888	0.845	0.215	76.000%
Affective-only	89.000%	$+0.289$	0.788	0.760	0.335	74.000%
Dual, independent	89.500%	$+0.292$	0.837	0.801	0.277	78.500%
Dual, interactive	91.750%	$\underline{+0.301}$	0.862	0.819	0.252	83.250%
Full w/o $\mathcal{L}_{a}$	87.000%	$+0.267$	0.858	0.817	0.257	77.250%

Table 2: Ablation results on the same selected subset of 400 editing requests. Full denotes dual ROI guidance with interactive reading. All percentages use a denominator of 400; JSR additionally requires DINO-I

\geq 0.80

for the same output. Best and second-best results in each column are bold and underlined, respectively, with ranks determined before rounding.

Configuration	Early visual $z\uparrow$	Higher visual $z\uparrow$	Affective ROI $z\uparrow$	$\Delta z_{\mathrm{aff}}\uparrow$
AffectDelta	0.427	0.447	0.258	—
Full + Random $q_{a}$	0.441	0.464	0.246	$-0.012$
Full + Shuffled $q_{a}$	0.448	0.460	0.262	$+0.004$
Random affective ROI	0.453	0.470	0.284	$+0.026$
Full w/o $\mathcal{L}_{a}$	0.468	0.482	0.352	$\underline{+0.094}$
ROI-ReFT Full	0.552	0.562	0.478	$\mathbf{+0.220}$

Table 3: Hidden-state correspondence with TRIBE-predicted target-image responses, reported as Fisher

z

scores. AffectDelta is the baseline in Figure What We Talk About When We Talk About Emotion Editing b. The last column is the affective score minus 0.258. Best and second-best results are bold and underlined, respectively.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

	EmoEditor			EmoEdit
ROI group	Actual	Shuffled	Random	Actual	Shuffled	Random
Early visual	0.391	0.016	$-0.015$	0.405	0.051	$-0.039$
V4/higher visual	0.460	0.032	$-0.039$	0.356	0.070	$-0.017$
Affective	0.240	0.007	$-0.035$	0.237	0.132	0.001
Auditory control	0.232	0.034	0.074	0.187	0.105	0.092
Sensorimotor control	0.206	$-0.014$	$-0.021$	0.191	0.106	$-0.001$

Appendix

Table 4: Hidden-state correspondence with TRIBE-predicted paired-target responses for existing editors. Scores are Fisher-transformed Pearson correlations. Shuffled hidden preserves target-emotion groups while disrupting pairings; Random features replaces hidden features with same-shaped Gaussian features.

Explore similar work

What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape
Jul 27, 2026Qing Li, Zeyu Dong, Yin Cui +2Text-Guided Image Editing Visual Reasoning
School of Artificial Intelligence, Shenzhen Technology University · School of Innovation Design, Shenzhen Technology University · Stanford University
AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control
Apr 12, 2026Shi Chen, Xuecheng Wu, Heli Sun +8Image Editing Evaluation Affective Computing
Xi’an Jiaotong University · Zhengzhou University · National University of Singapore +2
MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling
May 4, 2026Xinyi Yin, Yiduo Wang, Tingqi Hu +6Valence-Arousal Modeling Image Editing
School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China · School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China · School of Journalism and New Media, Xi’an Jiaotong University, Xi’an 710049, China +1

Jul 27, 2026J/K move · Enter open · S save

What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape

Qing Li, Zeyu Dong, Yin Cui, Chuan Yan+1

School of Artificial Intelligence, Shenzhen Technology University · School of Innovation Design, Shenzhen Technology University · Stanford University

Text-Guided Image Editing Visual Reasoning Image Editing

Open paper AskPDF arXiv

Emotional image editing requires more than applying affective filters or modifying predefined visual factors: an effective edit must identify what a particular image can afford for a target emotion. Existing affective image manipulation methods, including recent agentic variants, largely operate within bounded strategy spaces based on predefined factor taxonomies, knowledge libraries, or conventional editing templates, and therefore often miss image-specific, context-grounded strategies. We introduce EmoScope, a multi-agent framework that reframes the task from "how should I edit?" to "what can I edit?" EmoScope first discovers an image-specific editable space through emotion-conditioned affordance reasoning, then uses a semantic hierarchy of anchors, variables, and context to balance content consistency and emotional expressiveness before executing and verifying the edit. Because its plans are expressed as image-specific affordances rather than retrieved templates, EmoScope also exposes the editing strategy as an interactive surface for user refinement at the plan level. In a large-scale human evaluation covering all eight Mikels emotion categories, with 4,693 valid responses across 1,824 pairwise questions, participants preferred EmoScope over two competitive baselines by 88.1% on average. Attribution analysis further shows that EmoScope selects target-emotion-adaptive strategies rather than applying a uniform template. The same affordance-level plan also supports lightweight user refinement in an interactive pilot. Finally, we show that classifier-based metrics exhibit emotion-conditional blind spots toward non-stereotypical, context-grounded edits, and present a relative content-emotion preference-affinity landscape showing that EmoScope's advantage varies systematically across image-emotion combinations.

Abstract

Explore similar work

What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape

AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control

MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling