Selective, Risk-Controlled Emotion Forecasting in Conversation: From a Persistence Barrier to Departure Detection with Conformal Guarantees
Organizations: School of Computer Sciences, Universiti Sains Malaysia, 11800 Gelugor, Penang, Malaysia · School of Mathematical Sciences, Universiti Sains Malaysia, 11800 Gelugor, Penang, Malaysia
Abstract
Forecasting emotion in conversation is increasingly framed as predicting the next emotional state's value. We show this targets a hard quantity in a state-observed testbed and propose a learnable, deployable reformulation instead. On valence-arousal-dominance trajectories from a dyadic corpus, repeating a speaker's own previous state sets a persistence baseline that no frozen-encoder correction meaningfully improves upon and a state-space filter exceeds; the floor requires the historical state to be observed, since an estimated anchor collapses persistence below a direct forecaster. What survives is the timing of a departure from persistence, learnable on three of four corpora and near chance on the highest-base-rate corpus, a negative result we report. We recast the task as selective departure forecasting: a lead-time-weighted metric with a leakage protocol, and commitment policies whose best choice switches with the departure rate. A dialogue-level conformal risk control bounds the expected loss of committed predictions, meeting the target in all twelve configurations tested, though the guarantee is bought through abstention, with coverage and alarm recall falling sharply on the hardest corpus. One tested language model's confidence fails as a ranking signal where another succeeds, a model-dependent failure not yet attributable to family versus scale. Code and result logs are available from the corresponding author upon request.
Figures & tables
| Anchor | |||
| Oracle (gold label) | 0.625 | 0.609 | 0.552 |
| Estimated (same-turn probe) | 0.071 | 0.069 | 0.058 |
| Predictor | |||
| Same-speaker persistence (0 par.) | 0.625 | 0.609 | 0.552 |
| Same-speaker Kalman filter (1 hyperpar.) | 0.662 | 0.647 | 0.603 |
| Same-speaker AR(1) (2 par.) | 0.560 | 0.539 | 0.471 |
| Turn-level persistence (0 par.) | 0.446 | 0.549 | 0.426 |
| Linear extrapolation (0 par.) | 0.186 | 0.201 | 0.051 |
| Unconstrained direct value forecasters: | |||
| Channel (gap ) | CCC | pairs | Ratio to ceiling |
| Mean-label persistence | 0.627 | 9737 | – |
| Within-annotator | 0.620 | 20786 | – |
| Cross-annotator | 0.281 | 24916 | 68.4% |
| Agreement ceiling | 0.411 | 12456 | 100% |
| Dataset | Model | a | b | c | d |
| DailyDialog | GRU-MH | 0.337 | 0.343 | 0.336 | 0.304 |
| PsyDyn | 0.248 | 0.265 | 0.256 | 0.210 | |
| MELD | GRU-MH | 0.542 | 0.552 | 0.543 | 0.635 |
| PsyDyn | 0.659 | 0.702 | 0.642 | 0.702 |
| Dataset | Model | Mean risk | 95% CI | Cond. risk | Frac. | Coverage | |
| DailyDialog | GRU-MH | 0.1 | 0.086 | [0.061,0.113] | 0.154 | 0.85 | 0.169 |
| 0.2 | 0.166 | [0.127,0.209] | 0.217 | 0.95 | 0.308 | ||
| 0.3 | 0.247 | [0.212,0.287] | 0.279 | 1.00 | 0.471 | ||
| DailyDialog | PsyDyn | 0.1 | 0.074 | [0.050,0.097] | 0.140 | 1.00 | 0.160 |
| 0.2 | 0.156 | [0.133,0.185] | 0.179 | 1.00 | 0.388 | ||
| 0.3 | 0.203 | [0.182,0.225] | 0.206 | 1.00 | 0.773 |
| Dataset | Dialogues (tr/val/te) | Turns | Shift rate |
| DailyDialog | 13118 (10494/1311/1313) | 102979 | 0.173 |
| MELD (mp) | 1432 (1145/143/144) | 13708 | 0.554 |
| Hi-EF | 53 (42/5/6) | 4779 | 0.640 |
| EmoryNLP | 897 (717/89/91) | 12606 | 0.646 |
| IEMOCAP (5-fold CV) | 151 | 10039 | N/A (VAD) |
| Model | DailyDialog | MELD (dyadic) | MELD (multi-party) † | EmoryNLP (3-seed) |
| Random@prior | 0.179 | 0.581 | 0.581 † | |
| GRU direct | 0.393 | 0.685 | – † | |
| GRU multi-horizon | 0.387 | 0.697 | – † | |
| Free-form dynamics | 0.389 | 0.710 | – † | |
| OU-coupled (PsyDyn) | 0.371 | 0.688 | 0.665 † |
| Dataset | Model | AUROC [range] | MCC |
| DailyDialog | GRU-direct | 0.720 [0.717,0.726] | 0.247 |
| DailyDialog | GRU-MH | 0.716 [0.713,0.721] | 0.239 |
| DailyDialog | Free-form | 0.720 [0.719,0.722] | 0.242 |
| DailyDialog | PsyDyn | 0.687 [0.683,0.689] | 0.220 |
| MELD | GRU-direct | 0.529 [0.497,0.556] | 0.031 |
| MELD | GRU-MH | 0.516 [0.492,0.545] | 0.057 |
| Family | Dataset | Shift-F1 | ECE raw | ECE cal. | 50% | 70% | 90% |
| 14B | DailyDialog | 0.101 | 0.084 | 0.029 | |||
| 14B | MELD | 0.334 | 0.368 | 0.261 | |||
| 7B | DailyDialog | 0.274 | 0.704 | 0.549 | |||
| 7B | MELD | 0.719 | 0.330 | 0.221 |
Emotion Recognition in Conversations
Emotion Recognition in Conversations studies how dialogue context, speaker dynamics, and language, voice, and visual cues reveal changing emotional states.
See all 65 papers