Towards LLM-centric Affective Visual Customization via Efficient and Precise Emotion Manipulating

Jiamin Luo; Xuqian Gu; Jingjing Wang; Jiahong Lu

cs.CVFeb 20, 2026

Towards LLM-centric Affective Visual Customization via Efficient and Precise Emotion Manipulating

Authors: Jiamin Luo, Xuqian Gu, Jingjing Wang, Jiahong Lu

Organizations: School of Computer Science and Technology, Soochow University, Suzhou, China

Abstract

Previous studies on visual customization primarily rely on the objective alignment between various control signals (e.g., language, layout and canny) and the edited images, which largely ignore the subjective emotional contents, and more importantly lack general-purpose foundation models for affective visual customization. With this in mind, this paper proposes an LLM-centric Affective Visual Customization (L-AVC) task, which focuses on generating images within modifying their subjective emotions via Multimodal LLM. Further, this paper contends that how to make the model efficiently align emotion conversion in semantics (named inter-emotion semantic conversion) and how to precisely retain emotion-agnostic contents (named exter-emotion semantic retaining) are rather important and challenging in this L-AVC task. To this end, this paper proposes an Efficient and Precise Emotion Manipulating approach for editing subjective emotions in images. Specifically, an Efficient Inter-emotion Converting (EIC) module is tailored to make the LLM efficiently align emotion conversion in semantics before and after editing, followed by a Precise Exter-emotion Retaining (PER) module to precisely retain the emotion-agnostic contents. Comprehensive experimental evaluations on our constructed L-AVC dataset demonstrate the great advantage of the proposed EPEM approach to the L-AVC task over several state-of-the-art baselines. This justifies the importance of emotion information for L-AVC and the effectiveness of EPEM in efficiently and precisely manipulating such information.

Figures & tables

Figure 1. Three samples to illustrate the L-AVC task. The red boxes indicate differences from the original images.

Figure 2. The overall architecture of our proposed EPEM approach (a). Wherein (b) shows the process of model editing in the EIC module (see Section 3.2 ), while (c) and (d) show the process of interaction between MLLM and diffusion in the PER module (see Section 3.3 ), and FC represents the fully-connected layer.

Figure 3. A histogram (a) to illustrate statistics of our L-AVC dataset before and after editing, and two fan charts to show the emotional distribution of different visual elements, with (b) and (c) representing pre-editing and post-editing.

Approach	Consistency				Fidelity	Accuracy			Efficiency
Approach	FID $\downarrow$	LPIPS $\downarrow$	SSIM	CLIP-I	CLIP-T	M-Eval	G-Eval	H-Eval	Time
ControlNet	0.140	0.596	29.38	69.24	20.64	38.61	42.4	58.4	8.9
Prompt-to-Prompt	0.131	0.558	45.64	73.27	21.57	45.22	53.4	65.0	7.6
InstructPix2Pix	0.125	0.508	42.71	72.76	22.56	47.42	56.6	66.2	7.2
SDEdit	0.115	0.441	47.67	71.08	22.45	51.08	60.2	68.8	7.1
DiffEdit	0.118	0.426	49.68	72.35	21.19	52.45	63.6	69.0	7.3

Table 1. Comparison of several visual customization models and EPEM on L-AVC dataset. The

\downarrow

beside FID and LPIPS indicates the lower metric means better performance. Bold and underlined indicate the highest and second-highest performance.

Approach	Action			Facial			Object			Scene			Col&Bri
Approach	FID $\downarrow$	CLIP-I	H-Eval	FID $\downarrow$	CLIP-I	H-Eval	FID $\downarrow$	CLIP-I	H-Eval	FID $\downarrow$	CLIP-I	H-Eval	FID $\downarrow$	CLIP-I	H-Eval
ControlNet	0.163	63.59	39.4	0.162	63.19	52.4	0.129	71.93	63.2	0.124	70.91	65.4	0.124	76.58	71.6
Prompt-to-Prompt	0.140	67.45	45.2	0.157	68.66	57.8	0.123	75.13	71.6	0.114	75.06	73.6	0.122	80.05	76.8
InstructPix2Pix	0.134	68.78	46.8	0.139	67.41	56.4	0.124	73.80	72.6	0.113	74.63	76.8	0.116	79.16	78.4
SDEdit	0.121	65.37	54.0	0.128	65.11	61.6	0.113	73.65	73.2	0.104	72.42	76.0	0.109	78.83	79.2
DiffEdit	0.126	66.97	53.2	0.134	67.04	61.2	0.113	74.33	74.4	0.111	74.87	75.6	0.106	78.54	80.6

Table 2. Comparison of several visual customization models and our EPEM approach on our L-AVC dataset to evaluate the emotion editing performance of each visual element, where Col&Bri is short for the visual element color and brightness.

Figure 4. Two samples to illustrate the precise inter-emotion conversion manipulating ((a), (b), (c)) and exter-emotion contents retraining ((d), (e), (f)) two challenges.

Figure 5. Qualitative comparison of our EPEM approach and several advanced visual customization models. The boxes and texts in blue and red indicate the visual elements before and after editing via our EPEM approach.

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 315 papers

Controllable Affective Generation via Latent Vector Steering
Aug 26, 2026Xixian Yong, Siyuan Chang, Yingying Zhang +2Controllable Text Generation Activation Steering
Gaoling School of Artificial Intelligence, Renmin University of China · Tencent Jarvis Lab · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +1
AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control
Apr 12, 2026Shi Chen, Xuecheng Wu, Heli Sun +8Image Editing Evaluation Affective Computing
Xi’an Jiaotong University · Zhengzhou University · National University of Singapore +2
AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis
Apr 7, 2026Dong She, Xianrong Yao, Liqun Chen +3VLM Evaluation VLM Calibration
School of Future Technology,South China University of Technology,Guangzhou

Aug 26, 2026J/K move · Enter open · S save

Controllable Affective Generation via Latent Vector Steering

Xixian Yong, Siyuan Chang, Yingying Zhang, Xian Wu+1

Gaoling School of Artificial Intelligence, Renmin University of China · Tencent Jarvis Lab · Beijing Key Laboratory of Research on Large Models and Intelligent Governance+1

Controllable Text Generation Activation Steering Emotion Representation in Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable affective generation via latent vector steering. EmoVec extracts emotion-specific directions from paired neutral and emotion-conditioned responses using contrastive activation addition, and further refines them through task-specific debiasing and principal subspace removal. During inference, these vectors are injected into the final residual stream with static or scenario-adaptive scaling, enabling continuous control over emotional intensity without updating model weights. Experiments across three LLMs and eight emotions show that EmoVec consistently improves emotional salience while largely preserving semantic content, fluency, and coherence. Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, establishing EmoVec as a practical inference-time method for affective control in deployed LLMs.

Abstract

Image Editing

Explore similar work

Controllable Affective Generation via Latent Vector Steering

AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control

AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis