Organizations: Tianjin Normal University · Shenzhen University · The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · School of New Media and Communications, Tianjin University
Micro-gestures are subtle and transient movements triggered by unconscious neural and emotional activities, holding great potential for human-computer interaction and clinical monitoring. However, their low amplitude, short duration, and strong inter-subject variability make existing deep models prone to degradation under low-sample, noisy, and cross-subject conditions. This paper presents an active inference-based framework for micro-gesture recognition, featuring Expected Free Energy (EFE)-guided temporal sampling and uncertainty-aware adaptive learning. The model actively selects the most discriminative temporal segments under EFE guidance, enabling dynamic observation and information gain maximization. Meanwhile, sample weighting driven by predictive uncertainty mitigates the effects of label noise and distribution shift. Experiments on the SMG dataset demonstrate the effectiveness of the proposed method, achieving consistent improvements across multiple mainstream backbones. Ablation studies confirm that both the EFE-guided observation and the adaptive learning mechanism are crucial to the performance gains. This work offers an interpretable and scalable paradigm for temporal behavior modeling under low-resource and noisy conditions, with broad applicability to wearable sensing, HCI, and clinical emotion monitoring.
Figures & tables
Figure 1: Overview of existing methods and UAAI
Figure 2: Overall Framework. The framework enhances micro-gesture recognition performance through EFE-based temporal and spatial selection and uncertainty-aware augmentation under the active inference mechanism.
Method
GCN-NAS
MS-G3D
MSTCN-VAE
C3D
TSM
TRN
UAAI*
Modality
Skeleton
Skeleton
RGB
RGB
RGB
RGB
RGB
Accuracy (%)
58.85
64.75
42.59
45.90
58.69
59.51
63.47
Table 1: Comparison with state-of-the-art methods on the SMG dataset in different modalities. * denotes our framework.
Method
Uncertainty-aware
Temporal Selection
Spatial Selection
Accuracy (%)
Baseline
No
No
No
50.49
+Uncertainty-aware
Yes
No
No
57.54
+Temporal Selection
No
Yes
No
56.40
+Spatial Selection
No
No
Yes
55.40
UAAI
Yes
Yes
Yes
63.47
Table 2: Module Ablation Experiments
Figure 3: Active Observation Visualization
Frame Selection
Dataset
Accuracy (%)
Logic-in-Frames [ 12 ]
SMG
61.31
Video Tree [ 33 ]
SMG
59.51
Ours
SMG
63.47
Table 3: Comparison of frame selection methods
Figure 4: Convergence curves of UAAI under different training epochs. The upper row shows accuracy curves, and the lower row shows corresponding loss curves. The model converges stably after around 40 epochs.
Figure 5: Convergence curves with and without the UMIX.
M
Training wall-clock time(s)
Throughput (samples/s)
GPU utilization (%)
2
∼ 152
∼ 645
∼ 9.2
5
∼ 204
∼ 523
∼ 9.7
8
∼ 255
∼ 392
∼ 10.1
Table 4: Training cost under different M
Figure 6: Accuracy and loss Curves under different Monte Carlo sampling numbers M
Micro-gesture online recognition aims to temporally localize and classify subtle gestures in untrimmed videos. Owing to their extremely short duration, low motion amplitude, and ambiguous visual cues, capturing discriminative spatiotemporal representations remains highly challenging. Existing parameter-efficient adapters typically employ a single branch to model spatial and temporal cues jointly, which may fail to capture the fine-grained patterns of micro-gestures. To address this limitation, we propose a Spatial-Temporal Decoupled Adapter that decomposes video adaptation into independent temporal and spatial branches via lightweight depthwise convolutions. In addition, to address the long-tail distribution problem in the benchmark dataset, we introduce Adaptive Soft Balanced Augmentation, which dynamically allocates augmentation intensity based on class rarity and learning difficulty, without manual thresholds. Our method achieves an F1 score of 0.43808, ranking 1st in Track 2 of the 4th EI-MiGA-IJCAI Challenge.
Xucheng Shen, Kun Li, Fei Wang +3
Hefei University of Technology, Hefei, China · United Arab Emirates University, Al Ain, United Arab Emirates · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +1
Micro-gestures (MGs) are spontaneous and subtle body movements that frequently convey hidden human emotions. Recognizing MGs in untrimmed videos remains highly challenging due to their extremely low signal-to-noise ratio, severe long-tailed class distribution, and the inherent domain shift encountered in cross-subject evaluation scenarios. In this paper, we propose a comprehensive multi-modal framework for Track 1 of the 4th MiGA-IJCAI Challenge. To capture fine-grained representations, we design a saliency-guided multi-modal extraction pipeline integrating 68-keypoint skeleton joint coordinates, 3D heatmap volumes, and high-resolution RGB visual features. We introduce a gentle square-root smoothed weighting mechanism paired with an Orthogonal Semantic Embedding Loss to protect tail classes without compromising overall recognition capabilities. More importantly, to bridge the cross-subject generalization gap, we propose a Cross-Modal Pseudo-Labeling (CMPL) strategy for unsupervised domain adaptation, which significantly boosts single-modal robustness. A temperature-scaled soft-voting mechanism is finally utilized to alleviate overconfidence during late fusion. Extensive experiments demonstrate that our framework achieves a competitive F1-score of 68.13%, securing the 4th place.
Haoran Zhang, Haokun Zhang, Pengyu Liu +3
School of Computer Science and Information Engineering, Hefei University of Technology (HFUT), Hefei, China · School of Computer Science, University of Auckland(UOA), Auckland, New Zealand
In this paper, we present XInsight Lab's solution to the micro-gesture classification track of the 4th MiGA Challenge at IJCAI 2026, in which our solution ranked first and achieved a new state-of-the-art result. We propose a multimodal ensemble framework that integrates a self-supervised RGB-based model with supervised multi-stream models from previous solutions. The self-supervised RGB model is pretrained on 120K unlabeled clips via masked video modeling and then fine-tuned on iMiGUE. This simple yet effective RGB baseline achieves 69.224% top-1 accuracy on the iMiGUE test set, demonstrating the benefit of learning transferable representations from unlabeled in-domain videos. By incorporating this model as a complementary branch, the final ensemble reaches 74.419% top-1 accuracy, surpassing the previous state of the art by 1.206 percentage points. Experimental results on iMiGUE, including ablation studies on the ensemble strategy, validate the effectiveness of self-supervised RGB representation learning for micro-gesture recognition.
Tingyi Liu, Kun Li, Fei Wang +5
Hefei University of Technology, Hefei, China · United Arab Emirates University, Al Ain, United Arab Emirates · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +5