Multimodal Sentiment Analysis

Multimodal sentiment analysis (MSA) aims to understand human sentiment by integrating information from various sources like text, audio, and video. Current research emphasizes robust methods for handling incomplete data, improving the fusion of heterogeneous modalities (often prioritizing text), and mitigating biases inherent in datasets. This involves exploring advanced architectures like transformers and employing techniques such as contrastive learning, knowledge distillation, and attention mechanisms to enhance accuracy and interpretability. MSA's impact spans various fields, including human-computer interaction, social media analysis, and mental health monitoring, by enabling more nuanced and accurate understanding of emotional expression.

Papers

February 16, 2024

Emoji Driven Crypto Assets Market Reactions
Xiaorui Zuo, Yao-Tsung Chen, Wolfgang Karl Härdle
Multimodal Sentiment Analysis Investor Sentiment Cryptocurrency Market Sentiment Annotation

January 12, 2024

WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
Wenbin Wang, Liang Ding, Li Shen, Yong Luo, Han Hu, Dacheng Tao
Sentiment Analysis Large Vision Language Model Multimodal Sentiment Analysis Contextual Knowledge Deriving Wisdom Context Fusion

December 28, 2023

Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach
Weide Liu, Huijing Zhan, Hao Chen, Fengmao Lv
Knowledge Transfer Cross Modal Attention Multimodal Sentiment Analysis Missing Modality Sentiment Prediction Multimodal Supervision

December 19, 2023

PowMix: A Versatile Regularizer for Multimodal Sentiment Analysis
Efthymios Georgiou, Yannis Avrithis, Alexandros Potamianos
Multimodal Sentiment Analysis Multimodal Architecture Intra Modality

December 12, 2023

Multimodal Sentiment Analysis: Perceived vs Induced Sentiments
Aditi Aggarwal, Deepika Varshney, Saurabh Patel
User Sentiment Multimodal Sentiment Analysis Facial Emotion Recognition Multimodal Framework Social Medium Sentiment Image Sentiment

December 4, 2023

Improving Multimodal Sentiment Analysis: Supervised Angular Margin-based Contrastive Learning for Enhanced Fusion Representation
Cong-Duy Nguyen, Thong Nguyen, Duc Anh Vu, Luu Anh Tuan
Multimodal Representation Multimodal Sentiment Analysis Representation Fusion Unimodal Representation Angular Margin Contrastive

October 9, 2023

Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis
Haoyu Zhang, Yu Wang, Guanghao Yin, Kejun Liu, Yuanyuan Liu, Tianshu Yu
LeArning Abstract Multimodal Transformer Multimodal Sentiment Analysis Multimodal Sentiment Representation Editing

September 4, 2023

UniSA: Unified Generative Framework for Sentiment Analysis
Zaijing Li, Ting-En Lin, Yuchuan Wu, Meng Liu, Fengxiao Tang, Ming Zhao, Yongbin Li
Sentiment Analysis Aspect Based Sentiment Analysis Multimodal Sentiment Analysis Multimodal Sentiment Unified Generative

August 25, 2023

Exploiting Diverse Feature for Multimodal Sentiment Analysis
Jia Li, Wei Qian, Kun Li, Qi Li, Dan Guo, Meng Wang
Physiological Signal Multimodal Sentiment Analysis Personalization Performance Feature Diversity Sentiment Prediction

July 20, 2023

General Debiasing for Multimodal Sentiment Analysis
Teng Sun, Juntong Ni, Wenjie Wang, Liqiang Jing, Yinwei Wei, Liqiang Nie
Multimodal Sentiment Analysis Multimodal Feature Debiasing Framework Biased Feature Sentiment Information

June 27, 2023

ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis
Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu
Multimodal Sentiment Analysis Multimodal Sentiment Sentiment Analysis Benchmark

June 10, 2023

June 2, 2023

Syntax-aware Hybrid prompt model for Few-shot multi-modal sentiment analysis
Zikai Zhou, Haisong Feng, Baiyou Qiao, Gang Wu, Donghong Han
Text Modality Sentiment Analysis Cross Modality Multimodal Sentiment Analysis SYNTAX Score

May 24, 2023

Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion
Shaoxiang Wu, Damai Dai, Ziwei Qin, Tianyu Liu, Binghuai Lin, Yunbo Cao, Zhifang Sui
Fine Grained Multimodal Fusion Multimodal Sentiment Analysis Mutual Information Maximization Multimodal Signal Multimodal Sequence

May 15, 2023

Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning
Songning Lai, Jiakang Li, Guinan Guo, Xifeng Hu, Yulong Li, Yuan Tan, Zichen Song, Yutong Liu, Zhaoxia Ren, Chun Wan, Danmin Miao, Zhi Liu
Self Supervised Cross Modal Hierarchical Representation Multimodal Sentiment Analysis Sharing Matter Private Machine Learning Multimodal Sentiment

May 12, 2023

Multimodal Sentiment Analysis: A Survey
Songning Lai, Xifeng Hu, Haoxuan Xu, Zhaoxia Ren, Zhi Liu
Deep Learning Artificial Intelligence Timely Survey Multimodal Sentiment Analysis Research Direction

May 7, 2023

Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models
Sixia Li, Shogo Okada
Inherent Interpretability Text Modality Sentiment Analysis Large Scale Language Model Interpretable Way Multimodal Sentiment Analysis Polarity Prediction

May 5, 2023

The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisation
Lukas Christ, Shahin Amiriparian, Alice Baird, Alexander Kathan, Niklas Müller, Steffen Klug, Chris Gagne, Panagiotis Tzirakis, Eva-Maria Meßner, Andreas König, Alan Cowen, Erik Cambria, Björn W. Schuller
Target Emotion Multimodal Sentiment Analysis Personalization Performance MUSE Humor Sub Challenge Multimodal Dual Emotion Sub Challenge

March 27, 2023

TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models
Md Kamrul Hasan, Md Saiful Islam, Sangwu Lee, Wasifur Rahman, Iftekhar Naim, Mohammed Ibrahim Khan, Ehsan Hoque
Pre Trained Language Model Multimodal Sentiment Analysis Sarcasm Detection Humor Detection Non Verbal Cue