DiaRelay: Relaying Dialogue Context with a Constant-Size Memory for Emotion Recognition in Conversation
Authors: Zihao Zhou, Bin Yang, Jinghui Qin, Kebing Jin
Organizations: Guangdong University of Technology · Guizhou Provincial Laboratory of Big Data, State Key Laboratory of Public Big Data, Guizhou University
Emotion Recognition in Conversation (ERC) requires models to identify subtle emotional cues that are often distributed across distant dialogue turns. Existing methods typically incorporate dialogue history through a fixed context window. However, short windows discard potentially useful long-range evidence, while enlarging the window repeatedly re-encodes overlapping utterances, increases computational and memory costs, and may introduce irrelevant context. Moreover, commonly used parameter-efficient adaptation methods, such as LoRA, mainly introduce fixed low-rank transformations in the feature space and do not explicitly maintain a dialogue-level state or condition their transformations on the evolving conversational context. To address these limitations, we propose a lightweight adapter, DiaRelay, to enable LLMs to explicitly maintain a dialogue-level memory for accurate ERC. Based on LoRA, DiaRelay introduces two extra tightly collaborative components, Selective Relay Memory Transition and Dual-axis Relay Memory Read. Selective Relay Memory Transition progressively aggregates useful historical evidence into a bounded relay memory and propagates it across successive utterance predictions. This allows earlier emotional cues to influence later predictions after they leave the local context window, without re-encoding the complete dialogue history or expanding the backbone context length. Dual-axis Relay Memory Read uses the propagated memory to dynamically modulate low-rank feature transformations, enabling context-dependent representation adaptation without test-time gradient updates. Extensive experiments show that DiaRelay can achieve SOTA weighted F1 and accuracy on MELD while obtaining competitive results on IEMOCAP with only an extra 7.1M trainable parameters, indicating the effectiveness and generalizability of our DiaRelay in enhancing LLM-based emotional understanding.
Figures & tables
Figure 1: Overall architecture of DiaRelay.
Dataset
Partition
Utterances
Dialogues
IEMOCAP
train + valid
5,810
120
test
1,623
31
MELD
train + valid
11,098
1,152
test
2,610
280
Table 1: Statistics of the two datasets.
Method
Backbone
Mod
IEMOCAP
MELD
Acc.
W-F1
Acc.
W-F1
DialogueRNN †
CNN
T
63.40
62.75
59.54
57.03
DialogueGCN †
CNN
T
65.25
64.18
59.46
58.10
COSMIC †‡
RoBERTa
T
–
65.28
–
65.21
MMLA (SFT)
Llama-3.2-3B
T
50.00
49.24
64.41
63.40
MSE-Adapter †
ChatGLM3-6B
TAV
–
–
66.23
65.13
Table 2: Comparison with representative ERC methods on IEMOCAP and MELD. Bold values indicate the best overall results, while underlined values indicate the best results among methods using backbones no larger than 6B. † indicates the use of future or full-dialogue information. ‡ indicates the use of additional retrieved, generated, commonsense, biography, or speaker-related knowledge. ∗ denotes our causal reimplementation of PRC-Emo, where future utterances are excluded. Mod denotes modality, T denotes text modality, A denotes audio modality, and V denotes visual modality. For PRC-Emo (Causal), the two backbones correspond to IEMOCAP and MELD, respectively.
Region
Turn
Speaker
Utterance
Case 1: Dialogue 1264; Ground-truth emotion: fear
Outside-window clue
ut−6
Chandler
You kissed my best Ross! … Or something to that effect.
Local window
ut−4
Chandler
Really stupid.
ut−3
Mrs. Bing
Really stupid.
ut−2
Mrs. Bing
And I don’t even know how it happened.
ut−1
Mrs. Bing
I’m sorry, honey, I promise it will never happen again.
Table 3: Case studies on MELD. For each example, we show one representative historical clue outside the explicit four-utterance history window, the complete local window, and the predictions of different variants. Correct predictions are shown in bold.
Variant
Δq
Δo
W-F1
M-F1
Acc.
MELD, Qwen3-8B
LoRA-only
–
–
67.01
52.20
68.74
Window-local DiaRelay
✓
✓
68.95
55.75
70.42
w/o Error-Corrective Updating
✓
✓
69.37
54.67
70.70
w/o Output Correction
✓
–
69.10
54.18
70.65
w/o Query Correction
–
✓
69.41
55.42
70.84
Table 4: Ablation results of DiaRelay. Component-level ablations are conducted on MELD with Qwen3-8B, while the IEMOCAP results evaluate the overall improvement over the LoRA-only baseline. W-F1, M-F1, and Acc. denote weighted F1, macro F1, and accuracy, respectively. All values are percentages.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Future/full dialogue
External knowledge
Modality
Result source
DialogueRNN
Yes
No
T
Published result
DialogueGCN
Yes
No
T
Published result
COSMIC
Yes
Yes
T
Published result
MMLA (SFT)
No
No
T
Published result
MSE-Adapter
Yes
No
TAV
Published result
InstructERC
No
Yes
T
Published result
Appendix
Table S5: Information settings of the compared methods. T, A, and V denote text, audio, and visual modalities, respectively.
Item
Setting
Backbones
Qwen3-4B and Qwen3-8B
Input modality
Text only
Explicit context
At most four preceding utterances plus the current target
Backbone quantization
4-bit NF4 with double quantization
Computation precision
bfloat16; relay memories stored in FP32
LoRA rank / alpha / dropout
32 / 128 / 0.05
Appendix
Table S6: Model, optimization, and decoding configuration used in the main DiaRelay experiments.
Gradient span K
W-F1
M-F1
Acc.
1
68.82
54.01
69.50
2
70.06
57.04
71.15
3
68.30
54.32
69.69
4
69.07
54.75
70.61
Appendix
Table S7: Effect of the cross-utterance gradient span K on MELD with Qwen3-8B. All results are reported at the fixed third epoch. K controls the differentiable span during training rather than the forward memory horizon.
Figure S2: Layer-wise diagnostics of the query-side memory read in DaRMR. (a) Cosine similarity between the memory-derived address and the current key and query representations. The symbol St−1 shown in the panel denotes the same relay memory written as Rt−1 in the paper. (b) KL divergence between the relay-conditioned and base attention distributions across Transformer layers. The panels are used as mechanism diagnostics rather than standalone performance comparisons.
Rank r
Memory entries
W-F1
M-F1
Acc.
4
16
68.47
54.34
70.27
8
64
70.06
57.04
71.15
16
256
69.63
55.46
71.00
32
1024
69.84
55.96
70.84
Appendix
Table S8: Effect of the relay rank on MELD with Qwen3-8B. Memory entries denote the number of elements in the r×r relay memory maintained by each adapted layer.
Emotion Recognition in Conversations studies how dialogue context, speaker dynamics, and language, voice, and visual cues reveal changing emotional states.
Emotion Recognition in Conversation (ERC) plays a key role in developing emotionally responsive conversational systems, and contextual modeling is central to this task. Several studies have developed contextual models that capture dialogue history, speaker interactions, and emotion dynamics to support emotion prediction. However, the specific role of affective atmosphere, a latent, conversation-level emotional tendency shaped by conversational content and speaker interactions, remains insufficiently understood. This paper investigates what role atmosphere plays in conversational emotion understanding. We first propose AtmosERC, a graph-based atmosphere extractor that utilizes conversational context and speaker interactions to produce a compact representation of affective atmosphere. We then evaluate two downstream uses of the extracted atmosphere: initializing lightweight ERC model and providing emotion cues for LLM-based ERC. Experiments on three widely used ERC benchmarks show that the estimated atmosphere improves emotion prediction across both applications and supports more reliable predictions when local emotions deviate from the overall affective tendency.
Speech emotion recognition (SER) is commonly formulated as utterance-level classification, although conversational emotion depends on a speaker's usual vocal range and the emotional context established by previous utterances. Speech-language models provide strong pretrained acoustic and semantic representations, and can adapts them to SER labels via finetune, but this mechanism still missing per-dialogue state. We study whether test-time neural memory can supply this missing context while leaving the large audio language models (LALMs) backbone intact. Building on Titans, we introduce a plug-and-play Memory-as-a-Layer (MAL) adapter that writes dialogue history into a small neural memory and reads it back as an audio-token-aligned residual update, avoiding changes to the host model's token positions. Across different audio LLMs and emotion recognition datasets evaluations, our design improves SER performs across different evaluation metrics, supporting test-time memory as a residual contextual mechanism for conversational SER.
Daniel Chen, Qicong Hu, Yang Xiao +2
Department of Computer Science, University of Auckland, Auckland, New Zealand · School of Computer and Information Technology, Melbourne, Australia
Emotion Recognition in Conversation (ERC) has become a fundamental capability for large language models (LLMs) in human-centric interaction. Beyond accurate recognition, coherent emotional expression is also crucial, yet both are limited by the scarcity and static nature of high-quality annotated data. In this work, we propose SELF-EMO, a self-evolution framework grounded in the hypothesis that better emotion prediction leads to more consistent emotional responses. We introduce two auxiliary tasks, emotional understanding and emotional expression, and design a role-based self-play paradigm where the model acts as both an emotion recognizer and a dialogue responder. Through iterative interactions, the model generates diverse conversational trajectories, enabling scalable data generation. To ensure quality, we adopt a data flywheel mechanism that filters candidate predictions and responses using a smoothed IoU-based reward and feeds selected samples back for continuous self-improvement without external supervision. We further develop SELF-GRPO, a reinforcement learning algorithm that stabilizes optimization with multi-label alignment rewards and group-level consistency signals. Experiments on IEMOCAP, MELD, and EmoryNLP show that SELF-EMO achieves state-of-the-art performance, improving accuracy by +6.33% on Qwen3-4B and +8.54% on Qwen3-8B, demonstrating strong effectiveness and generalization.
Shaowei Zhang, Faqiang Qian, Yan Chen +5
1SenseTime · 2Tsinghua University · 3Shanghai Jiao Tong University +1