Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimation
Organizations: Electronics and Telecommunications Research Institute
Abstract
Continuous valence-arousal estimation in real-world environments is challenging due to inconsistent modality reliability and interaction-dependent variability in audio-visual signals. Existing approaches primarily focus on modeling temporal dynamics, often overlooking the fact that modality reliability can vary substantially across interaction stages. To address this issue, we propose SAGE, a Stage-Adaptive reliability modeling framework that explicitly estimates and calibrates modality-wise confidence during multimodal integration. SAGE introduces a reliability-aware fusion mechanism that dynamically rebalances audio and visual representations according to their stage-dependent informativeness, preventing unreliable signals from dominating the prediction process. By separating reliability estimation from feature representation, the proposed framework enables more stable emotion estimation under cross-modal noise, occlusion, and varying interaction conditions. Extensive experiments on the Aff-Wild2 benchmark demonstrate that SAGE consistently improves concordance correlation coefficient scores compared with existing multimodal fusion approaches, highlighting the effectiveness of reliability-driven modeling for continuous affect prediction.
Figures & tables
| Model | Validation Set | ||
|---|---|---|---|
| Valence CCC | Arousal CCC | CCC Avg | |
| MM-CV-LC [ 45 ] | 0.469 | 0.649 | 0.559 |
| Netease Fuxi Virtual Human [ 46 ] | 0.464 | 0.640 | 0.552 |
| CtyunAI [ 48 ] | 0.550 | 0.681 | 0.616 |
| HFUT-MAC [ 47 ] | 0.554 | 0.659 | 0.607 |
| Situ-RUCAIM3 [ 27 ] | 0.588 | 0.668 | 0.628 |
| Model | Test Set | ||
|---|---|---|---|
| Valence CCC | Arousal CCC | CCC Avg | |
| MM-CV-LC [ 45 ] | 0.46 | 0.49 | 0.47 |
| Netease Fuxi Virtual Human [ 46 ] | 0.64 | 0.62 | 0.63 |
| CtyunAI [ 48 ] | 0.50 | 0.63 | 0.56 |
| HFUT-MAC [ 47 ] | 0.52 | 0.54 | 0.53 |
| Situ-RUCAIM3 [ 27 ] | 0.61 | 0.59 | 0.60 |