Multimodal Emotion Recognition

Follow this topic Ask about this topic

Latest papers 141

19 papers in the last four weeks, down 10% on the four weeks before. 0.1% of all new papers.

All topics

GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition
Oct 13, 2026Ting Zhu, Huizhi Liang, Boguslaw Obara +2Multimodal Emotion Recognition Multimodal Fusion
Newcastle University, Newcastle upon Tyne, United Kingdom · Taiyuan University of Technology, Shanxi, China
EmoGenie: Benchmarking and Evaluating Expressed and Evoked Emotion in T2I Generation with MLLMs
Oct 13, 2026Zixuan Xing, Lancheng Gao, Shengyan Li +4T2I Generation Evaluation Multimodal Emotion Recognition
Shanghai Jiao Tong University
Evidential Multimodal Transformers with Dempster-Shafer Fusion for Emotion Recognition
Oct 9, 2026Muhammad Usama, Yeoreum JungMultimodal Emotion Recognition Uncertainty-Aware Multimodal Fusion
Braindeck Inc., Seoul, Republic of Korea · School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding
Oct 8, 2026Jia Li, Yichao He, Yangchen Yu +6Multimodal Emotion Recognition Emotion Recognition
Hefei University of Technology · Nanyang Technological University
Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes
Oct 8, 2026Hao Li, Jinye Zhang, Bobo Li +5Affective Computing Multimodal Emotion Recognition
School of Computer Science, National Engineering Research Center for Multimedia Software, Wuhan University, Wuhan 430072, China
From Expression to Reaction: Role-aware Visual Transfer and Stimulus-guided Reasoning for Interlocutor Emotion Recognition
Oct 2, 2026Wei Wang, Zhaowu Li, Jianjie Luo +3Emotion Recognition in Conversations Multimodal Emotion Recognition
Guangdong University of Technology Guangzhou, China · Hong Kong Metropolitan University Hong Kong, China
Modeling Shared and Individual Structure for Cross-Subject Continuous Affect Regression from EEG-fNIRS
Oct 2, 2026Xuan Wang, Bing Wang, Shuai Chang +3Valence-Arousal Modeling Affective Computing
LongEmo: Towards Emotion Understanding and Reasoning in Long Videos
Sep 30, 2026Shuo Zhang, Yifan Zhou, Han Wang +15Memory-Augmented VLMs Multimodal Emotion Recognition
BUPT · SJTU · THU +6
Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue
Sep 30, 2026Yutong Hu, Jinho ChoiEmotion Recognition in Conversations Multimodal Emotion Recognition
Emory University Atlanta, GA
VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict
Sep 29, 2026Jiale Dai, Liuxian Ma, Xiaoke Niu +5Multimodal Emotion Recognition Multimodal Fusion
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Artificial Intelligence, Tsinghua University · China United Network Communications Group Co., Ltd.
Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning
Sep 29, 2026Cheng Ye, Weidong Chen, Zhaobo Qi +2Multimodal Large Language Models Multimodal Emotion Recognition
University of Science and Technology of China, Hefei · Harbin Institute of Technology, Weihai
Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention
Sep 29, 2026Cheng Ye, Weidong Chen, Peipei Song +1Video Captioning Multimodal Emotion Recognition
University of Science and Technology of China, Hefei
AffectReveal: Event-Grounded Emotion Recognition Beyond Visual Appearances
Sep 29, 2026Yihao Qian, Runhao Zeng, Sicheng Zhao +3Evidence-Grounded Reasoning Multimodal Emotion Recognition
South China University of Technology, Guangzhou, China · Shenzhen MSU-BIT University, Shenzhen, China · Tsinghua University, Beijing, China
T-SNN: Temporal Simplicial Neural Network for EEG Decoding
Sep 27, 2026Nikita Malik, Shubhajit Roy, Mohit Kataria +4Cross-Subject EEG Decoding EEG Decoding
Bharti School of Telecommunication Technology and Management Indian Institute of Technology, Delhi · Department of Computer Science & Engineering Indian Institute of Technology, Gandhinagar · Yardi School of Artificial Intelligence Indian Institute of Technology, Delhi +4
Differential Attention Unlocks Complementary EEG and Speech Fusion for Emotion Recognition
Sep 25, 2026Philip H. Lee, Shreeram Suresh Chandra, John H. L. HansenMultimodal Emotion Recognition EEG Emotion Recognition
Independent Researcher, USA · Center for Language and Speech Processing (CLSP), Johns Hopkins University, Baltimore, MD, USA · Center for Robust Speech Systems (CRSS), University of Texas at Dallas, Richardson, TX, USA
Oracle Complementarity Is Not Realizable Complementarity in Frozen-Encoder Audio-Visual Emotion Recognition
Sep 24, 2026Benjamin HurtMultimodal Fusion Multimodal Emotion Recognition
University of California, Santa Barbara
BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport
Sep 23, 2026Yanbing Wang, Shenyue Wang, Chunyang YuEmotion Recognition in Conversations Multimodal Emotion Recognition
OPPO Research Institute
From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities
Sep 17, 2026Desta Haileselassie Hagos, Saurav Keshari Aryal, Legand L. BurgeEmotion Recognition Multimodal Emotion Recognition
Department of Electrical Engineering and Computer Science, Howard University, Washington DC, USA
Affective Shared Autonomy: Temporal Affect Dynamics and Subjective Evaluation in Bimanual Teleoperation Tasks
Sep 17, 2026Zhengji Liang, Guiyin Tian, Sijin Qu +2Robot Teleoperation Multimodal Emotion Recognition
Department of Data and Systems Engineering, The University of Hong Kong, Hong Kong SAR, China. · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China.
Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts
Sep 16, 2026Christopher Bagdon, Carina Silberer, Roman KlingerSocial Media Analysis Multimodal Emotion Recognition
Fundamentals of Natural Language Processing, University of Bamberg, Germany · Institut für Maschinelle Sprachverarbeitung, University of Stuttgart, Germany
Affect-Prototype Guided Fusion for Open-Vocabulary Incomplete Multi-modal Emotion Recognition
Sep 15, 2026Yichi Zhang, Shenyue Wang, Jing Luo +2Multimodal Emotion Recognition Multimodal Fusion
School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China · OPPO Research Institute, Shanghai, China
ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation
Sep 12, 2026Guan-Hua Wen, Hou-Chiang Tseng, Kuan-Yu ChenEmotion Recognition in Conversations Multimodal Emotion Recognition
National Taiwan University of Science and Technology, Taipei, Taiwan
Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding
Sep 10, 2026Ziwei Wang, Xingyi He, Hongbin Wang +3Data Augmentation Diffusion Transformer
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, China
Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometry
Sep 9, 2026Oriol Marín, Roger Marí, Gloria Haro +1Emotion Recognition in Conversations Multimodal Emotion Recognition
Eurecat, Centre Tecnològic de Catalunya, Barcelona, Spain · Universitat Pompeu Fabra, Barcelona, Spain · Serra Húnter Fellow Programme, Universitat Pompeu Fabra, Barcelona, Spain
RAFM-SER++: A Lightweight Multimodal Emotion Recognition Framework for Real-Time Behavioral Monitoring in Surveillance Systems
Sep 7, 2026Ngo Truong Dinh, Tung-Lam Bui, Chi-Trung Duong +3Efficient Multimodal Inference Multimodal Emotion Recognition
Faculty of Information Technology University of Science, VNU-HCM Ho Chi Minh City, Vietnam
Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras
Sep 7, 2026Jiaqi Chen, Qinfu Xu, Hao Zhuang +1Multimodal Emotion Recognition Emotion Recognition
Beijing Institute of Technology Beijing, China
MVFA: A Multi-View Text-Guided Multimodal Fusion LLM Adapter for Sentiment Analysis and Emotion Recognition
Sep 5, 2026Pengfei Shao, Jisheng Dang, Jiawen Fang +7Multimodal Sentiment Analysis Multimodal Emotion Recognition
School of Information Science and Technology, Guangdong University of Foreign Studies, Guangdong, China · School of Information Science and Engineering, Lanzhou University, Lanzhou, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangdong, China +2
Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition
Sep 4, 2026Tingyi Lin, Wen-Ren Yang, Kuanwei ChenMultimodal Emotion Recognition Emotion Recognition in Conversations
Department of Electrical Engineering, National Changhua University of Education, Changhua, Taiwan · Department of Computer Science and Information Engineering, National Central University, Taoyuan, Taiwan
Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion
Sep 4, 2026Xu Lin, Ke Wang, Hui Kang +1Multimodal Emotion Recognition Multimodal Fusion
College of Computer Science and Technology, Jilin University, Changchun 130012, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China
ArtSociety: Multi-Agent Multimodal Collaboration for Art Emotion Understanding
Sep 3, 2026Jian Li, Fanfan Ji, Jinxiang Lai +5Multimodal Emotion Recognition Affective Computing
Nanjing University Suzhou, China · CSE The Hong Kong University of Science and Technology Hong Kong, China · Tencent YoutuLab Shanghai, China

Oct 13, 2026J/K move · Enter open · S save

GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition

Ting Zhu, Huizhi Liang, Boguslaw Obara, Shufei Duan+1

Newcastle University, Newcastle upon Tyne, United Kingdom · Taiyuan University of Technology, Shanxi, China

Multimodal Emotion Recognition Multimodal Fusion Geometric Representation Learning Hierarchical Representation Learning Hyperbolic Representation Learning

Open paper AskPDF arXiv

Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as speech and text. Psychological studies suggest that emotions exhibit latent hierarchical structured relationships in their semantic representations. However, most MER approaches rely on Euclidean representations, which capture fine-grained cross-modal interactions but do not explicitly provide a geometric prior for organizing hierarchical emotional structures. To mitigate this gap, we propose GeoMER, a joint Euclidean-Hyperbolic modeling framework that introduces hyperbolic emotion-core routing to capture higher-level, abstract emotional representations for MER. The proposed framework jointly models token-level cross-modal semantic interactions and latent emotion representations in two complementary geometric spaces. Specifically, GeoMER consists of a Euclidean Multimodal Fusion Module (EuclFusion) for multimodal representation fusion and a Hyperbolic Geometric Core Routing (HypGeoRoute) module that performs geometry-aware core assignment and aggregation in a hyperbolic manifold to capture more abstract feature representations. The HypGeoRoute module complements the Euclidean token-level modeling. We experiment on the IEMOCAP multimodal MER benchmark and demonstrate that the proposed method achieves the best performance, with 83.54% weighted accuracy and 83.35% unweighted accuracy, outperforming Euclidean baselines, including MemoCMT.