Multimodal Sentiment

Multimodal sentiment analysis aims to understand human emotions by integrating information from various sources like text, images, and audio, surpassing the limitations of text-only sentiment analysis. Current research focuses on improving the fusion of these modalities, exploring techniques like transformers, contrastive learning, and Bayesian methods to address challenges such as weak inter-modality correlations, missing data, and dataset biases. This field is significant for its potential to enhance applications ranging from social media monitoring and advertising effectiveness to mental health assessment and human-computer interaction, offering more nuanced and accurate emotion detection than unimodal approaches.

Papers

June 23, 2022

The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress
Lukas Christ, Shahin Amiriparian, Alice Baird, Panagiotis Tzirakis, Alexander Kathan, Niklas Müller, Lukas Stappen, Eva-Maria Meßner, Andreas König, Alan Cowen, Erik Cambria, Björn W. Schuller
Emotion Recognition Multimodal Sentiment Analysis Continuous Chronic Stress Human Humor Multimodal Sentiment MUSE Humor Sub Challenge Affective Response Sub Challenge

April 28, 2022

Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities
Jiandian Zeng, Tianyi Liu, Jiantao Zhou
Transformer Based Multimodal Sentiment Analysis Transformer Encoder Multimodal Sentiment Multimodal Fusion Model

April 12, 2022

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection
Zhen Li, Bing Xu, Conghui Zhu, Tiejun Zhao
Contrastive Learning Multimodal Data Multimodal Feature Multimodal Sentiment Token Fusion Layer Fusion

March 1, 2022

Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors
Yang Wu, Yanyan Zhao, Hao Yang, Song Chen, Bing Qin, Xiaohuan Cao, Wenting Zhao
Sentiment Analysis Multimodal Sentiment Analysis Multimodal Sentiment Automatic Speech Recognition Error

January 24, 2022

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis
Georgios Paraskevopoulos, Efthymios Georgiou, Alexandros Potamianos
Multimodal Fusion Multimodal Sentiment Analysis Cross Modal Interaction Level Representation Modality Invariant Multi Level Fusion Multimodal Sentiment

Multimodal Sentiment

Papers

The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress

Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection

Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis