cs.CVApr 7, 2026
SaveAICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis
Organizations: School of Future Technology,South China University of Technology,Guangzhou
Abstract
Vision-Language Models (VLMs) have demonstrated strong capabilities in perception, yet holistic Affective Image Content Analysis (AICA), which integrates perception, reasoning, and generation into a unified framework, remains underexplored. To address this gap, we introduce AICA-Bench, a comprehensive benchmark with three core tasks: Emotion Understanding (EU), Emotion Reasoning (ER), and Emotion-Guided Content Generation (EGCG). We evaluate 23 VLMs and identify two major limitations: weak intensity calibration and shallow open-ended descriptions. To address these issues, we propose Grounded Affective Tree (GAT) Prompting, a training-free framework that combines visual scaffolding with hierarchical reasoning. Experiments show that GAT reduces intensity errors and improves descriptive depth, providing a strong baseline for future research on affective multimodal understanding and generation.
Figures & tables
Figure 1: Illustration of the three affective tasks in the AICA-Bench benchmark.
| Benchmark | Model | Tasks | #Datasets | #AICA Datasets | #Instr. | #Models | Prompt |
| EVE Bhattacharyya and Wang (2025) | VLM | EU | 5 | 5 | 8,009 | 7 | B+CoT |
| AffectGPT Lian et al. (2025) | MLLM | EU | 9 | 3 | - | 17 | B |
| EEmo-Bench Gao et al. (2025) | MLLM | EU | 1 | 1 | 6,773 | 19 | B |
| EmoBench-M Hu et al. (2025) | MLLM | EU | 13 | - | 6,226 | 20 | B |
| MOSABench Song et al. (2024) | MLLM | EU | 1 | 1 | 1,000 | 8 | B |
| AICA-Bench (Ours) | VLM | EU, ER, EGCG | 9 | 9 | 18,124 | 23 | B+CoT |
Table 1: Comparison of recent emotion benchmarks for VLMs and MLLMs. EU : Emotion Understanding, ER : Emotion Reasoning, EGCG : Emotion-guided Content Generation. #Instr. = total evaluation instructions; Prompt = B : Basic prompting, CoT : Chain-of-Thought prompting.
Figure 2: The instruction curation pipeline of the AICA-Bench benchmark. It consists of two stages: (1) image sample selection and filtering from multiple affective datasets, and (2) automatic instruction generation across three tasks—emotion understanding (EU), emotion reasoning (ER), and emotion-guided content generation (EGCG).
| Model | ER | EGCG | ||||
| MSE | MAE | Pearson | MSE | MAE | Pearson | |
| Qwen2.5VL-7B | 1.220 | 0.775 | 0.472 | 0.845 | 0.645 | 0.528 |
| Gemini2.5-Pro | 1.940 | 0.940 | 0.441 | 0.855 | 0.620 | 0.735 |
| ChatGPT-4o | 1.360 | 0.790 | 0.502 | 0.580 | 0.470 | 0.775 |
| Ours | 0.400 | 0.295 | 0.880 | 0.260 | 0.240 | 0.900 |
Table 2: Performance comparison on ER and EGCG tasks against human annotations.
| Model | EU Basic | EU CoT | EU Avg. | ER Avg. | EG Avg. | Overall Avg. (%) |
| Closed-source Models | ||||||
| Gemini-2.5-pro | 66.97 | 67.57 | 67.27 | 79.08 | 74.13 | 73.49 |
| Qwen-VL-max | 64.07 | 65.98 | 65.02 | 77.75 | 75.93 | 72.90 |
| ChatGPT-4o | 64.44 | 65.42 | 64.93 | 77.81 | 75.73 | 72.82 |
| Gemini-2.5-flash | 68.05 | 69.32 | 68.68 | 76.55 | 68.19 | 71.14 |
| ChatGPT-4o-mini | 60.15 | 63.68 | 61.91 | 76.45 | 74.09 | 70.81 |
Table 3: Main results on the AICA-Bench benchmark. EU : Emotion Understanding, ER : Emotion Reasoning, EGCG : Emotion-guided Content Generation.The complete leaderboard is provided in Appendix Table 9 .
Figure 3: (a) Performance gains from model scaling diminish beyond 7B parameters. (b) Models consistently struggle with abstract art compared to realistic photos. (c) Masking facial cues causes an 11.1% performance drop, revealing the models’ heavy reliance on visual shortcuts (faces) over holistic context.
Figure 4: (a) EU: The dominance of intensity errors (Blue) over valence errors (Red) reveals an arousal bottleneck. (b)-(c) ER & EGCG: Across both tasks, models achieve high emotion alignment but consistently lack descriptive depth.
Figure 5: Representative failure cases of EU, ER, and EGCG tasks.
Figure 6: The GAT Prompting Framework.
| Model | EU | ER | EGCG | |||
| Basic | GAT | Basic | GAT | Basic | GAT | |
| Gemini-2.5-Pro | 66.97 | 71.15 | 79.08 | 82.45 | 74.13 | 78.25 |
| Gemini-2.5-Flash | 68.05 | 70.41 | 76.55 | 79.02 | 68.19 | 72.07 |
| ChatGPT-4o | 64.44 | 69.82 | 77.81 | 83.10 | 75.73 | 79.40 |
| GPT-4o-mini | 60.15 | 66.02 | 76.45 | 77.92 | 74.09 | 77.54 |
| Qwen-VL-max | 64.07 | 69.34 | 77.75 | 79.18 | 75.93 | 79.66 |
Table 4: Comprehensive performance comparison across 15 models. We compare Baseline and GAT for EU, ER, and EGCG, where green values indicate improvements of GAT over the Baseline.
Figure 7: GAT corrects major intensity-confusion errors in EU (left) and improves descriptive depth for ER and EGCG (right). Abbreviations: Amu = Amusement, Con = Contentment, Exc = Excitement, Awe = Awe, Ang = Anger, Fear = Fear.
| Method | Gemini- 2.5-Pro | ChatGPT- 4o | Qwen2.5- VL-7B | Qwen2- VL-7B |
| EU | ||||
| Baseline | 66.97 | 64.44 | 56.43 | 53.52 |
| w/ only VS | 68.40 | 66.55 | 58.50 | 55.60 |
| w/ only AT | 69.95 | 68.15 | 60.10 | 57.40 |
| w/ VS+AT (GAT) | 71.15 | 69.82 | 61.94 | 59.08 |
| ER | ||||
Table 5: Component-wise ablation of GAT on four representative models. VS denotes visual scaffolding. AT denotes the AffectToT reasoning module for EU. Clue denotes the grounded clue construction module for ER and EGCG. Green values indicate the best result within each task block.
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Year | Original Image Count | Images Used in AICA | Proportion in AICA | Type | Emotion Label Model |
| EMOTIC | 2019 | 23,571 | 1,000 | 12.37% | social | Ekman |
| EmoSet-118K | 2023 | 118,102 | 1,000 | 12.37% | social | Mikels |
| FindingEmo | 2024 | 25,000 | 1,000 | 12.37% | social | Plutchik |
| FI | 2016 | 23,308 | 1,000 | 12.37% | social | Mikels |
| Emotion6 | 2015 | 1,980 | 1,000 | 12.37% | social | Ekman |
| FlickrLDL | 2017 | 10,700 | 1,000 | 12.37% | social | Mikels |
Table 6: Datasets Used in the AICA Benchmark
Figure 8: Example of an Evoked Emotion Prediction instruction in the EU Basic and CoT setting.
Figure 9: Example of an Expressed Emotion Recognition instruction in the EU Basic and CoT setting.
Figure 10: Examples of an ER and EGCG.
| Task | Evaluation Metrics |
| Emotion Understanding (EU) | Accuracy, Macro F1, Weighted F1 |
| Emotion Reasoning (ER) | Emotion Alignment, Descriptiveness, Causal Soundness |
| Emotion-guided Content Generation (EGCG) | Emotion Alignment, Descriptiveness |
Table 7: Evaluation metrics used for each task in AICA-Bench
Figure 11: Human Annotation Interface
Figure 12: Examples of the Instruction-Tuning Dataset Format
| Parameter | Value |
| Base Model | Qwen2.5-VL-7B |
| Fine-tuning Framework | LLaMA-Factory |
| Fine-tuning Method | LoRA |
| GPU | 1 NVIDIA A100 |
| Effective Batch Size | 8 (per-device: 2, gradient accumulation: 4) |
| Learning Rate | 1.0e-4 |
Table 8: Scoring model training configuration
| Model | EU Basic | EU CoT | EU Avg. | ER Avg. | EG Avg. | Overall Avg. (%) |
| Closed-source Models | ||||||
| Gemini-2.5-pro | 66.97 | 67.57 | 67.27 | 79.08 | 74.13 | 73.49 |
| Qwen-VL-max | 64.07 | 65.98 | 65.02 | 77.75 | 75.93 | 72.90 |
| ChatGPT-4o | 64.44 | 65.42 | 64.93 | 77.81 | 75.73 | 72.82 |
| Gemini-2.5-flash | 68.05 | 69.32 | 68.68 | 76.55 | 68.19 | 71.14 |
| ChatGPT-4o-mini | 60.15 | 63.68 | 61.91 | 76.45 | 74.09 | 70.81 |
Table 9: Performance comparison on AICA-Bench tasks. EU: Emotion Understanding, ER: Emotion Reasoning, EGCG: Emotion-guided Content Generation. Scores are shown as percentages.
Figure 13: Radar charts comparing performance across tasks for eight major VLM model series (both open-source and closed-source)
Figure 14: Performance comparison across subtasks (EU, ER, EGCG) and overall average scores for open-source models. Each line represents a model series, with the best-performing model in each subtask marked by a star.
| Model | ER | EGCG | ||||||||||||
| Align | Desc | Causal | Overall | Align | Desc | Overall | ||||||||
| Base | GAT | Base | GAT | Base | GAT | Base | GAT | Base | GAT | Base | GAT | Base | GAT | |
| GPT-4o | 4.12 | 4.36 | 3.05 | 3.61 | 3.18 | 3.74 | 3.45 | 3.90 | 4.20 | 4.48 | 3.22 | 3.79 | 3.71 | 4.02 |
| Qwen2.5-VL-7B | 3.88 | 4.14 | 2.94 | 3.47 | 3.02 | 3.56 | 3.28 | 3.72 | 3.95 | 4.21 | 3.10 | 3.62 | 3.53 | 3.86 |
Table 10: Human evaluation on ER and EGCG (200 images, 10 annotators). Values are mean ratings (no ).
Figure 15: Visual Scaffolding examples used in GAT Prompting. The red contours highlight the segmented regions that serve as visual anchors for guiding the model’s grounded reasoning.
Figure 16: A sample correct case of EU-Basic.
Figure 17: A sample correct case of EU-Basic.
Figure 18: A sample error case of EU-Basic.
Figure 19: A sample error case of EU-Basic.
Figure 20: A sample error case of EU-Basic.
Figure 21: A sample error case of EU-Basic.
Figure 22: A sample correct case of EU-CoT.
Figure 23: A sample correct case of EU-CoT.
Figure 24: A sample correct case of EU-CoT.
Figure 25: A sample error case of EU-CoT.
Figure 26: A sample error case of EU-CoT.
Figure 27: A sample error case of EU-CoT.
Figure 28: A sample error case of EU-CoT.
Figure 29: Basic Prompting vs. CoT Prompting Case Study
Figure 30: A sample correct case of EGCG
Figure 31: A sample correct case of EGCG
Figure 32: A sample correct case of EGCG
Figure 33: A sample correct case of EGCG
Figure 34: A sample correct case of EGCG
Figure 35: A sample correct case of EGCG
Figure 36: A sample error case of EGCG
Figure 37: A sample error case of EGCG