Disentangling Reasoning in Large Audio-Language Models for Ambiguous Emotion Prediction
Organizations: University of Auckland, New Zealand · The University of Melbourne, Australia · University of Birmingham, United Kingdom
Abstract
Speech emotion recognition plays an important role in various applications. However, most existing approaches predict a single emotion label, oversimplifying the inherently ambiguous nature of human emotional expression. Recent large audio-language models show promise in generating richer outputs, but their reasoning ability for ambiguous emotional understanding remains limited. In this work, we reformulate ambiguous emotion recognition as a distributional reasoning problem and present the first systematic study of ambiguity-aware reasoning in LALMs. Our framework comprises two complementary components: an ambiguity-aware objective that aligns predictions with human perceptual distributions, and a structured ambiguity-aware chain-of-thought supervision that guides reasoning over emotional cues. Experiments on IEMOCAP and CREMA-D demonstrate consistent improvements across SFT, DPO, and GRPO training strategies.
Figures & tables
| Dataset | Methods | JS | BC | Brier | |
|---|---|---|---|---|---|
| IEMOCAP | Base model | 0.40 | 0.64 | 0.51 | 0.15 |
| Audio-Reasoner | 0.36 | 0.67 | 0.52 | 0.15 | |
| SFT | 0.29 | 0.73 | 0.60 | 0.12 | |
| DPO | 0.23 | 0.79 | 0.65 | 0.09 | |
| GRPO | 0.36 | 0.67 | 0.55 | 0.13 | |
| GRPO z | 0.20 | 0.82 | 0.67 | 0.07 |
| Setting | CoT | KL | JS | BC | Brier | |
|---|---|---|---|---|---|---|
| In-domain | ✓ | ✓ | 0.21 | 0.82 | 0.57 | 0.04 |
| In-domain | ✗ | ✓ | 0.22 | 0.81 | 0.57 | 0.04 |
| Cross-domain | ✓ | ✓ | 0.38 | 0.65 | 0.50 | 0.15 |
| Cross-domain | ✗ | ✓ | 0.52 | 0.52 | 0.31 | 0.22 |