Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusion
Authors: Aislan Gabriel O. Souza, Agostinho Freire, Leandro Honorato Silva, Igor Lucas B. da Silva, João Vinícius R. de Andrade, Gabriel C. de Albuquerque, Lucas Matheus da S. Oliveira, Mário Stela Guerra, +1 more
Organizations: Universidade de Pernambuco (UPE), Escola Polit´ecnica de Pernambuco (POLI), Recife, Brazil
We address the Ambivalence/Hesitancy (A/H) Video Recognition Challenge at the 10th ABAW Competition (CVPR 2026). We propose a divergence-based multimodal fusion that explicitly measures cross-modal conflict between visual, audio, and textual channels. Visual features are encoded as Action Units (AUs) extracted via Py-Feat, audio via Wav2Vec 2.0, and text via BERT. Each modality is processed by a BiLSTM with attention pooling and projected into a shared embedding space. The fusion module computes pairwise absolute differences between modality embeddings, directly capturing the incongruence that characterizes A/H. On the BAH dataset, our approach achieves a Macro F1 of 0.6808 on the validation test set, outperforming the challenge baseline of 0.2827. Statistical analysis across 1{,}132 videos confirms that temporal variability of AUs is the dominant visual discriminator of A/H.
Figures & tables
Feature
Metric
A/H vs No A/H
∣r∣
AU06 (cheek raiser)
std
0.076 vs 0.059
0.186
AU09 (nose wrinkler)
std
0.095 vs 0.084
0.186
AU12 (smile)
std
0.089 vs 0.068
0.172
AU26 (jaw drop)
zcr
0.421 vs 0.384
0.168
AU02 (outer brow)
std
0.110 vs 0.102
0.149
Table 1: Top discriminative AU features (Mann-Whitney U, N=1,132 ). All significant after Bonferroni correction.
University of the Basque Country UPV/EHU, San Sebastian, Spain · Laboratory of LESIA, University of Biskra, Algeria · Lab. de Génie Electrique (LAGE), University Kasdi Merbah Ouargla, Ouargla, Algeria +2