cs.CVMar 15, 2026

Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusion

Authors: Aislan Gabriel O. Souza, Agostinho Freire, Leandro Honorato Silva, Igor Lucas B. da Silva, João Vinícius R. de Andrade, Gabriel C. de Albuquerque, Lucas Matheus da S. Oliveira, Mário Stela Guerra, +1 more

Organizations: Universidade de Pernambuco (UPE), Escola Polit´ecnica de Pernambuco (POLI), Recife, Brazil

Abstract

We address the Ambivalence/Hesitancy (A/H) Video Recognition Challenge at the 10th ABAW Competition (CVPR 2026). We propose a divergence-based multimodal fusion that explicitly measures cross-modal conflict between visual, audio, and textual channels. Visual features are encoded as Action Units (AUs) extracted via Py-Feat, audio via Wav2Vec 2.0, and text via BERT. Each modality is processed by a BiLSTM with attention pooling and projected into a shared embedding space. The fusion module computes pairwise absolute differences between modality embeddings, directly capturing the incongruence that characterizes A/H. On the BAH dataset, our approach achieves a Macro F1 of 0.6808 on the validation test set, outperforming the challenge baseline of 0.2827. Statistical analysis across 1{,}132 videos confirms that temporal variability of AUs is the dominant visual discriminator of A/H.

Figures & tables

Explore similar work

CardsList
  1. Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusion

    Jul 17, 2026Oussama Berhili, Yassine Ouzar, Larbi BoubchirMultimodal FusionAmbivalence and Hesitancy Recognition

  2. Conflict-Aware Multimodal Fusion for Ambivalence and Hesitancy Recognition

    Mar 16, 2026Salah Eddine Bekhouche, Hichem Telli, Azeddine Benlamoudi +3Ambivalence and Hesitancy RecognitionMultimodal Fusion

  3. Team LEYA in 10th ABAW Competition: Multimodal Ambivalence/Hesitancy Recognition Approach

    Mar 13, 2026Elena Ryumina, Alexandr Axyonov, Dmitry Sysoev +4Ambivalence and Hesitancy RecognitionMultimodal Classification