Multimodal Emotion Recognition

Follow this topic Ask about this topic

Latest papers 141

19 papers in the last four weeks, down 10% on the four weeks before. 0.1% of all new papers.

All topics

Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation
Mar 12, 2026Byeongjin Jung, Chanyeong Park, Sejoon LimValence-Arousal Modeling Multimodal Emotion Recognition
Kookmin University, Republic Of Korea
Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modeling
Mar 12, 2026Junhyeong Byeon, Jeongyeol Kim, Sejoon LimMultimodal Emotion Recognition Facial Expression Recognition
Kookmin University, Republic of Korea
Nuanced Emotion Recognition Based on a Segment-based MLLM Framework Leveraging Qwen3-Omni for AH Detection
Mar 12, 2026Liang Tang, Hongda Li, Jiayu Zhang +5Ambivalence and Hesitancy Recognition Emotion Recognition
Lenovo, Beijing, China
Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimation
Mar 12, 2026Yubeen Lee, Sangeun Lee, Junyeop Cha +1Valence-Arousal Modeling Multimodal Emotion Recognition
Electronics and Telecommunications Research Institute
Multimodal Self-Attention Network with Temporal Alignment for Audio-Visual Emotion Recognition
Mar 11, 2026Inyong Koo, yeeun Seong, Minseok Son +2Multimodal Emotion Recognition Audio-Visual Alignment
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea · Graduate School of Green Growth and Sustainability, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
MissBench: Benchmarking Multimodal Affective Analysis under Imbalanced Missing Modalities
Mar 10, 2026Tien Anh Pham, Phuong-Anh Nguyen, Duc-Trong Le +1Multimodal Model Evaluation Missing-Modality Learning
VNU University of Engineering and Technology, Hanoi, Vietnam
Solution to the 10th ABAW Expression Recognition Challenge: A Robust Multimodal Framework with Safe Cross-Attention and Modality Dropout
Mar 9, 2026Jun Yu, Naixiang Zheng, Guoyuan Wang +5Multimodal Emotion Recognition Facial Expression Recognition
University of Science and Technology of China · Unisound AI Technology Co., Ltd.
AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognition
Mar 7, 2026Yunsheng Wang, Yuntao Shou, Yilong Tan +3Emotion Recognition in Conversations Multimodal Emotion Recognition
College of Computer and Mathematics, Central South University of Forestry and Technology · Department of Computer Science, State University of New York
Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognition
Mar 7, 2026Zhexian Huang, Bo Zhao, Hui Ma +5Multimodal Emotion Recognition Multimodal Learning
Great Bay University · The Chinese University of Hong Kong, Shenzhen · City University of Hong Kong (Dongguan) +1
Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy
Mar 2, 2026Jiahao Huang, Fengyan Lin, Xuechao Yang +4Affective Computing Multimodal Large Language Models
Fujian Normal University · RMIT University · Independent Researcher +1
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
Feb 27, 2026Yiyang Fang, Wenke Huang, Pei Fu +5Multimodal Large Language Models Multimodal Reasoning
School of Computer Science, Wuhan University. · MiLM Plus, Xiaomi Inc.
A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations
Feb 26, 2026Soumya Dutta, Smruthi Balaji, Sriram GanapathyEmotion Recognition in Conversations Multimodal Emotion Recognition
LEAP Lab, Department of Electrical Engineering, IISc Bangalore, Bangalore, 560012, Karnataka, India · Microsoft, Bangalore, Karnataka, India
EmoOmni: Bridging Emotional Understanding and Expression in Omni-Modal LLMs
Feb 25, 2026Wenjie Tian, Zhixian Zhao, Jingbin Hu +4Multimodal Large Language Models Emotion-Aware Dialogue Generation
Northwestern Polytechnical University, Xi’an, China · University of Surrey, Guildford, United Kingdom
iMiGUE-Speech: A Spontaneous Speech Dataset for Affective Analysis
Feb 25, 2026Sofoklis Kakouros, Fang Kang, Haoyu ChenAffective Computing Multimodal Emotion Recognition
Center for Machine Vision and Signal Analysis, University of Oulu, Finland
Memory-guided Prototypical Co-occurrence Learning for Mixed Emotion Recognition
Feb 24, 2026Ming Li, Yong-Jin Liu, Fang Liu +6Emotion Recognition Multimodal Emotion Recognition
MOE-Key Laboratory of Pervasive Computing, Department of Computer Science and Technology, Tsinghua University, Beijing, China. · the Key State Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China · the School of Computer Science and Technology, and Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China +1
Multi-Channel Speech Enhancement for Cocktail Party Speech Emotion Recognition
Feb 21, 2026Youjun Chen, Guinan Li, Mengzhe Geng +9Speech Emotion Recognition Multimodal Emotion Recognition
The Chinese University of Hong Kong, Hong Kong SAR, China · National Research Council Canada, Canada · Institute of Software, Chinese Academy of Sciences, China
Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection
Feb 18, 2026Rong Fu, Ziming Wang, Shuo Yin +3Multimodal Emotion Recognition Multimodal Sentiment Analysis
Independent Researcher
CREMD: Crowd-Sourced Emotional Multimodal Dogs Dataset
Feb 17, 2026Jinho Baek, Houwei Cao, Kate BlackwellEmotion Recognition Multimodal Emotion Recognition
dept. of Computer Science, New York Institute of Technology, New York, USA
EEmo-Logic: A Unified Dataset and Multi-Stage Framework for Comprehensive Image-Evoked Emotion Assessment
Feb 1, 2026Lancheng Gao, Ziheng Jia, Zixuan Xing +4Multimodal Large Language Models Multimodal QA
Institute of Image Communication and Network Engineering, Shanghai Key Laboratory of Digital Media Processing and Transmissions, Shanghai Jiao Tong University, Shanghai · School of Communication & Electronic Engineering, East China Normal University, Shanghai
ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion
Jul 8, 2025Yuhan Li, Wei Zhang, Juan Chen +3Cross-Modal Learning Missing-Modality Learning
National University of Defense Technology · The Fifth Electronic Research Institute of MIIT · University of Chinese Academy of Sciences +2
Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts
May 30, 2025Christopher Bagdon, Aidan Combs, Carina Silberer +1Social Media Analysis Multimodal Emotion Recognition
Fundamentals of Natural Language Processing, University of Bamberg, Germany · Department of Sociology, The Ohio State University, USA · Institut für Maschinelle Sprachverarbeitung, University of Stuttgart, Germany

Mar 12, 2026J/K move · Enter open · S save

Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation

Byeongjin Jung, Chanyeong Park, Sejoon Lim

Kookmin University, Republic Of Korea

Valence-Arousal Modeling Multimodal Emotion Recognition Prompt Learning

Open paper AskPDF arXiv

Valence-arousal (VA) estimation is crucial for capturing the nuanced nature of human emotions in naturalistic environments. While pre-trained vision-language models such as CLIP have demonstrated remarkable semantic alignment capabilities, their application to continuous regression tasks is often limited by the discrete nature of text prompts. In this paper, we propose a novel multimodal framework for VA estimation that introduces Distance-aware Soft Prompt Guidance to bridge the gap between semantic representations and continuous affective dimensions. Specifically, we partition the VA space into multiple discrete regions, each associated with distinct textual descriptions. Rather than relying on hard categorization, we employ a Gaussian kernel to compute soft labels based on the Euclidean distance between the ground-truth coordinates and the region centers, allowing the model to learn fine-grained emotional transitions. For multimodal integration, our architecture utilizes a CLIP image encoder and an Audio Spectrogram Transformer to extract robust visual and acoustic features. These features are temporally modeled using Gated Recurrent Units and integrated through a hierarchical fusion scheme that sequentially combines cross-modal attention for alignment and gated fusion for adaptive refinement. Experimental results on the Aff-Wild2 dataset show that the proposed semantic-guided approach outperforms the official baseline and demonstrates robust performance on in-the-wild data.