HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classification

Andrey V. Savchenko; Kseniia Tsypliakova

cs.CVMar 13, 2026

HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classification

Authors: Andrey V. Savchenko, Kseniia Tsypliakova

Organizations: Sber AI Lab · HSE University Moscow, Russia

Abstract

This article presents our results for the 10th Affective Behavior Analysis in-the-Wild (ABAW) competition. For frame-wise facial emotion understanding tasks (frame-wise facial expression recognition, valence-arousal estimation, action unit detection), we propose a fast approach based on facial embedding extraction with pre-trained EfficientNet-based emotion recognition models. If the latter model's confidence exceeds a threshold, its prediction is used. Otherwise, we feed embeddings into a simple multi-layered perceptron trained on the AffWild2 dataset. Estimated class-level scores are smoothed in a sliding window of fixed size to mitigate noise in frame-wise predictions. For the fine-grained violence detection task, we examine several pre-trained architectures for frame embeddings and their aggregation for video classification. Experimental results on four tasks from the ABAW challenge demonstrate that our approach significantly improves validation metrics over existing baselines.

Figures & tables

Figure 1: Proposed approach for EXPR classification.

Panel of Figure 2 — Figure 2: Our best single-stream frame-level violence detection pipeline. ConvNeXt-T (pretrained on ImageNet-1K) extracts 768-d per-frame features, processed by a 5-layer dilated TCN.

Method	Modality	Is ensemble?	F1-score $P_{EXPR}$	Accuracy
Baseline VGGFACE (MixAugment) [ 31 ]	Faces	No	25.0	-
CLIP+LSTM [ 5 ]	Video	Yes	34.1	-
CLIP [ 36 ]	Faces	No	36.0	-
EfficientNet-B0 [ 48 ]	Faces	No	40.2	-
V-NAW [ 34 ]	Video	No	41.81	-
SSL + Temporal+ Post-process [ 57 ]	Audio/video	Yes	44.43	-

Table 1: Main results of Expression Recognition for the Aff-Wild2’s validation set.

Method	CCC_V	CCC_A	$P_{VA}$
Baseline ResNet-50 [ 31 ]	0.24	0.20	0.22
MAVEN [ 1 ]	0.307	0.305	0.3061
DDAMFN+LSTM [ 5 ]	-	-	0.479
Best EfficientNet [ 51 ]	0.490	0.596	0.543
Time-aware Gated Fusion [ 35 ]	0.427	0.676	0.552
GRJCA [ 44 ]	0.459	0.652	0.555

Table 2: Main results of Valence-Arousal Estimation for the Aff-Wild2’s validation set.

Method	F1-score $P_{AU}$
Baseline VGGFACE [ 31 ]	39.0
CLIP [ 36 ]	43.0
DDAMFN+LSTM, threshold 0.5 [ 5 ]	41.1
DDAMFN+LSTM, best thresholds [ 5 ]	45.1
AUD-TGN [ 58 ]	53.7
Best EfficientNet [ 47 ]	54.5

Table 3: Main results of Action Unit Detection for the Aff-Wild2’s validation set.

Baseline
Method	$F1_{V}$	$F1_{NV}$	Macro $F1$
ResNet-50 + BiLSTM [ 32 ]	0.56	0.71	0.640
3D / video backbones
SlowFast-R50	0.303	0.661	0.482
R3D-18 + Conv1d	0.186	0.797	0.491
S3D + Conv1d	0.268	0.781	0.524

Table 4: Main validation results and ablation study for Fine-Grained Violence Detection on the official DVD validation set.

F1_{V}

and

F1_{NV}

denote per-class F1-scores for violence and non-violence, respectively.

Affective Computing

Affective computing develops systems that recognize, model, and respond to human emotions and affective states from behavior and physiological signals.

See all 141 papers

HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition
Jul 14, 2026Aleksei Bakin, Andrey V. SavchenkoMulti-Task Learning Multimodal Emotion Recognition
Central University, Moscow, Russia · Sber AI Lab, Moscow, Russia · HSE University, Laboratory of Algorithms and Technologies for Network Analysis, Nizhny Novgorod, Russia
Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach
Mar 13, 2026Elena Ryumina, Maxim Markitantov, Alexandr Axyonov +4Valence-Arousal Modeling Affective Computing
St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia · St. Petersburg Federal Research Center of the Russian Academy of Sciences St. Petersburg, Russia · ITMO University, St. Petersburg, Russia +1
Team LEYA in 10th ABAW Competition: Multimodal Ambivalence/Hesitancy Recognition Approach
Mar 13, 2026Elena Ryumina, Alexandr Axyonov, Dmitry Sysoev +4Ambivalence and Hesitancy Recognition Multimodal Classification
St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia · HSE University, St. Petersburg, Russia

Jul 14, 2026J/K move · Enter open · S save

HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition

Aleksei Bakin, Andrey V. Savchenko

Central University, Moscow, Russia · Sber AI Lab, Moscow, Russia · HSE University, Laboratory of Algorithms and Technologies for Network Analysis, Nizhny Novgorod, Russia

Multi-Task Learning Multimodal Emotion Recognition Facial Expression Recognition Multimodal Fusion Ambivalence and Hesitancy Recognition

Open paper AskPDF arXiv

This article presents our results for the 11th Affective Behavior Analysis in-the-Wild (ABAW) competition. For multi-task learning with simultaneous prediction of valence, arousal, facial expressions, and action units on s-Aff-Wild2 dataset, we use frozen lightweight facial extractors, MT-EmotiDDAMFN and MT-EmotiEffNet-B0, with separate heads and systematic post-processing: temporal Gaussian smoothing, per-class expression bias, AffectNet blending, per-AU threshold tuning, and weighted backbone fusion. On the official validation set, our ensemble significantly exceeds the performance of the ConvNeXt baseline. For ambivalence/hesitancy video recognition on the expanded BAH dataset, we extend the audiovisual pipeline to video-level Macro F1 by late fusion of face, HuBERT audio, and RoBERTa text classifiers, temporal aggregation, and a global-text gate. Frame-level Weighted F1 on validation set rises from 0.74 in ABAW-8 to 0.79, while the best public-test video-level Macro F1 reaches 0.73. In both tasks, competitive performance is achieved without fine-tuning heavy backbones. These results indicate that systematic prediction calibration and lightweight multimodal fusion can rival substantially heavier end-to-end approaches while offering improved efficiency and deployment flexibility.

Abstract

Affective Computing

Explore similar work

HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition

Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach

Team LEYA in 10th ABAW Competition: Multimodal Ambivalence/Hesitancy Recognition Approach