Emotion Recognition in Conversations

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 38

All topics

From Speech to Editable Concepts: Probing Emotion Recognition with Concept Bottleneck Models
Sep 30, 2026Hezhao Zhang, Thomas HainConcept Bottleneck Models Emotion Recognition in Conversations
School of Computer Science, University of Sheffield Sheffield, United Kingdom
Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue
Sep 30, 2026Yutong Hu, Jinho ChoiEmotion Recognition in Conversations Multimodal Emotion Recognition
Emory University Atlanta, GA
NSV-Shift: A Contrastive Benchmark for Non-Speech Vocalization Understanding and Response Adaptation in Speech-to-Speech Models
Sep 27, 2026Ziwei ChenSpeech Generation Evaluation Audio Understanding
University of California San Diego
Empath: Tracing Multi-Level Emotion Dynamics in Crisis Counseling Dialogues
Sep 24, 2026Ziwei Gong, Yuchen Huang, Wen Liang +4Mental Health Counseling Emotional Support Conversation
Columbia University, USA · Barnard College, USA
BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport
Sep 23, 2026Yanbing Wang, Shenyue Wang, Chunyang YuEmotion Recognition in Conversations Multimodal Emotion Recognition
OPPO Research Institute
Enriching Speech Emotion Representations with Conversational Context
Sep 22, 2026Arthur Peuvot, Romaric Besançon, Gaël de Chalendar +2Representation Learning Emotion Recognition in Conversations
Université Paris-Saclay, CEA, List, France · LISN, CNRS, Université Paris-Saclay, France
When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
Sep 16, 2026Sai Babu Udayagiri, Arjun Chouhan, Ravisekhar Kanagala +1LLM Inference Efficiency Selective Prediction
Five9
Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometry
Sep 9, 2026Oriol Marín, Roger Marí, Gloria Haro +1Emotion Recognition in Conversations Multimodal Emotion Recognition
Eurecat, Centre Tecnològic de Catalunya, Barcelona, Spain · Universitat Pompeu Fabra, Barcelona, Spain · Serra Húnter Fellow Programme, Universitat Pompeu Fabra, Barcelona, Spain
Shifting Relational Paradigms for Affective Computing: Affective Resonance, Vitality Affects, and Vocal Interaction Fields
Sep 9, 2026Cy Gorman, Yihang YaoEmotion Recognition in Conversations Affective Computing
Nurobodi, Australia
Surfacing the Unsaid: CUE-Bench for Affective Stance in Chinese Discourse
Aug 11, 2026Zhenyan Zheng, Yunyao Zhang, Junxi Sheng +2Stance Detection Intent Classification
Huazhong University of Science and Technology
Rationale-Guided Learning for Multimodal Emotion Recognition
Aug 11, 2026Sujung Oh, Jung Uk Kim, Sangmin LeeMultimodal Reasoning Emotion Recognition in Conversations
Pixel Lab, Sungkyunkwan University, South Korea · Visual AI Lab, Kyung Hee University, South Korea · Pixel Lab, Korea University, South Korea
Generation or Judgement? A Paradigm Perspective on LLM-Based Emotion-Cause Pair Extraction in Conversation
Jul 29, 2026Weijie Feng, Hongchuang Wang, Binbin Liu +1Relation Extraction Emotion Recognition in Conversations
Hefei University of Technology
AtmosERC: Modeling Dialogue-Level Affective Atmosphere for Emotion Recognition in Conversation
Jul 29, 2026Weijie Feng, Tongwei Zhang, Binbin Liu +1Emotion Recognition in Conversations
Hefei University of Technology
SpEmoC: A Balanced Speaker-Segment Multimodal Emotion Benchmark
Jul 20, 2026Sania Bano, Shahzad Ahmad, Santosh Kumar Vipparthi +2Emotion Recognition in Conversations Multimodal Emotion Recognition
CVPR Lab, Indian Institute of Technology Ropar, India · Østfold University College, Norway · CVPR Lab, SCSS, Trinity College Dublin, Ireland
EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation
Jul 19, 2026Zilong Huang, Kong Aik Lee, Chong-Xin Gan +3Emotion Recognition in Conversations Multimodal Emotion Recognition
Dept. of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China
EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation
Jul 19, 2026Zilong Huang, Kong Aik Lee, Junjie Li +2Emotion Recognition in Conversations Multimodal Emotion Recognition
Dept. of Electrical and Electronic Engineering, The Hong Kong Polytechnic University · Speech, Language, and Cognition Laboratory, The University of Hong Kong
TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech
Jun 29, 2026Sathvik Manikantan Napa Ugandhar, Hao Zhang, Alison Gunzler +5Emotion Recognition in Conversations
Department of Computer Science, Johns Hopkins University, USA · Department of Electrical and Computer Engineering, Johns Hopkins University, USA · Amazon Research, UK +1
EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis
Jun 22, 2026Wai Laam Mak, Isibor Kennedy Ihianle, Pedro MachadoSpeech Processing Emotion Recognition in Conversations
School of Science and Technology, Nottingham Trent University, Nottingham, UK
Learning Robust Pair Confidence for Multimodal Emotion-Cause Pair Extraction
Jun 17, 2026Zhuangzhuang Pan, Ning Dong, Yingna Su +1Pairwise Preference Learning Multimodal Robustness
Institute for Advanced Studies Universiti Malaya Kuala Lumpur 50603, Malaysia · School of Information Engineering Suqian University Suqian 223800, China · Digitization Department Suzhou University of Technology Suzhou 215500, China
Explainable and Trustworthy Speech Emotion Recognition Using Confidence Score and Reinforcement Learning Rectified Speech Emotion Descriptors
Jun 12, 2026Youjun Chen, Xurong Xie, Mengzhe Geng +9Explainable Artificial Intelligence Reinforcement Learning
The Chinese University of Hong Kong, Hong Kong SAR, China · Institute of Software, Chinese Academy of Sciences, China · National Research Council Canada, Canada +1
LLM-Based Synthetic Ground Truth Generation for Audio-Based Emotion Classification via In-Context Learning
Jun 10, 2026Qing Huang, Pooja Pol, Jianing ZhangLLM-Assisted Annotation Emotion Recognition in Conversations
School of Business, Technical University of Applied Sciences Augsburg, Friedberger Str. 4, 86161 Augsburg, Germany · Data Science und Autonome Systeme Technologietransferzentrum (TTZ), An der Schmiede 19, 86899 Landsberg am Lech, Germany
Titans-as-a-Layer: Test-Time Memory for Conversational Speech Emotion Recognition
Jun 7, 2026Daniel Chen, Qicong Hu, Yang Xiao +2Memory-Augmented Language Models Audio Understanding
Department of Computer Science, University of Auckland, Auckland, New Zealand · School of Computer and Information Technology, Melbourne, Australia
Bayesian Spectral Emotion Transition Discovery from Multi-Annotator Disagreement
Jun 1, 2026Keito Inoshita, Takato UenoAnnotator Disagreement Emotion Recognition in Conversations
Faculty of Business and Commerce, Kansai University, Suita, Japan · Data Science and AI Innovation Promotion Center, Shiga University, Hikone, Japan · Graduate School of Data Science, Shiga University, Hikone, Japan
Evaluation of Conversational Agents: Understanding Culture, Context and Environment in Emotion Detection
May 28, 2026Martha Teiko Teye, Yaw Marfo Missah, Emmanuel Ahene +2Sentiment Analysis Emotion Recognition in Conversations
Cluster of Excellence, University of Stuttgart, 70174 Stuttgart, German · Department of Computer Science, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana · Institute for Ethics in Artificial Intelligence, Technical University of Munich, 80333 Münich, Germany
Emotion Recognition in Sign Language Conversation
May 22, 2026Yusong Wang, Keyu Mao, Takao Obi +2Emotion Recognition in Conversations Sign Language Recognition
Institute of Science Tokyo, Japan · New York University, New York, 11201, USA
AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligence
May 20, 2026Kate M. Lubrano, Faisal Sayed, Ankita Rathod +4LLM Evaluation Emotion Recognition in Conversations
1Pareto, California, US · 2Thoughtful, California, US · University of Pennsylvania, Pennsylvania, US
Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition
May 20, 2026Phuong-Anh Nguyen, The-Son Le, Duc-Trong Le +1Modality Imbalance Emotion Recognition in Conversations
Faculty of Information Technology, VNU University of Engineering and Technology, 100000, Hanoi, Vietnam.
Modeling Emotional Dynamics in Agent-to-Agent Interactions on Moltbook
May 19, 2026Syed Mhamudul Hasan, Abdur R. ShahidMulti-Agent LLM Systems LLM Agent Evaluation
Southern Illinois University Carbondale, IL, USA
VISAFF: Speaker-Centered Visual Affective Feature Learning for Emotion Recognition in Conversation
May 18, 2026Linan ZHU, Zihao Zhai, Xiao Han +4VLM Adaptation Emotion Recognition in Conversations
1Zhejiang University of Technology · 2ETH Zurich, Switzerland
EGI: A Multimodal Emotional AI Framework for Enhancing Scrum Master Real-time Self-Awareness
May 17, 2026Jingni Huang, Peter BloodsworthSoftware Engineering Emotion Recognition in Conversations
Department of Computer Science University of Oxford Oxford, United Kingdom

Sep 30, 2026, cs.SDJ/K move · Enter open · S save

From Speech to Editable Concepts: Probing Emotion Recognition with Concept Bottleneck Models

Hezhao Zhang, Thomas Hain

School of Computer Science, University of Sheffield Sheffield, United Kingdom

Concept Bottleneck Models Emotion Recognition in Conversations Speech Emotion Recognition Speech Language Models

Open paper AskPDF arXiv

Speech emotion recognition (SER) is the task of assigning emotion labels to utterances. Early systems relied on acoustic features, whereas recent approaches combine multiple modalities, most commonly speech and text. Still, performance remains poor on many datasets. Large language models (LLMs) have therefore attracted interest for SER, as they can process diverse inputs jointly with instructions. However, direct audio input raises questions of explainability. To address similar questions in image classification, concept bottleneck models were introduced. This work adapts concept bottlenecks to SER to examine how individual predictions depend on transcripts, acoustic descriptions and speaker attributes. Experiments test three LLMs on CREMA-D, IEMOCAP and MELD, with concepts extracted by separate tools. On scripted corpora, LLMs are strongly biased towards the transcript in the zero-shot setting, which lowers Macro-F1 from 27.8 to 5.8 on CREMA-D. Fine-tuning removes this bias, and the transcript raises Macro-F1 from 41.8 to 45.1. Removing speech rate changes 48% of Neutral predictions to Disgust on CREMA-D; removing intensity level on MELD changes predictions despite little change in Macro-F1. These findings show that aggregate performance changes alone do not capture the effects of concept removal on individual predictions.

Momentum

Related topics

Latest papers 38