cs.CVNov 28, 2025

Robust Image Emotion Classification via Multi-Level Affective Captioning

Authors: Zibo Zhou, Zhengjun Zhai, Huimin Chen

Organizations: School of Computer Science, Northwestern Polytechnical University, No.127 Youyi Xilu, Xi’an, 710072, Shaanxi, China · School of Cybersecurity, Northwestern Polytechnical University, No.127 Youyi Xilu, Xi’an, 710072, Shaanxi, China

Abstract

Image emotion classification (IEC) has been studied for many years and has recently attracted increasing attention with the rapid development of deep learning. Although recent studies have benefited from knowledge embedded in pre-trained visual models, the performance is still limited by the "affective gap" between low-level visual features and high-level affective semantics. Psychological studies suggest that language can convey fine-grained and emotion-rich information. Inspired by this observation, we attempt to narrow the "affective gap" by introducing affective text. Specifically, we design a multi-level contrastive loss to detect emotional concepts from images and introduce an emotional-attribute-guided large language model approach to generate affective sentences. Then, a pre-trained language model is employed to combine emotional concepts with affective sentences for IEC. In addition, we introduce a contrastive loss with a semantic-similarity-based sampling strategy to address large intra-class differences and limited inter-class differences in affective datasets. Moreover, unlike many previous studies, we explicitly consider images containing embedded text. Extensive experiments on multiple benchmarks show that the proposed method helps narrow the "affective gap" and achieves competitive performance.

Figures & tables

Explore similar work

CardsList
  1. AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis

    Apr 7, 2026Dong She, Xianrong Yao, Liqun Chen +3VLM EvaluationVLM Calibration

  2. IECLA: Interpretable Emotion Contribution Learning for Affective Image Editing

    Oct 18, 2026Yijie Guo, Dexiang Hong, Weidong Chen +2Image EditingFeature Attribution

  3. Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models

    Jan 27, 2026Filip Nowicki, Hubert Marciniak, Jakub Łączkowski +6Affective ComputingEmotion Recognition