cs.CVMar 27, 2026

Dual-View Optical Flow for 4D Micro-Expression Recognition - A Multi-Stream Fusion Attention Approach

Authors: Luu Tu Nguyen, Thi Bich Phuong Man, Vu Tram Anh Khuong, Thanh Ha Le, Thi Duyen Ngo

Organizations: VNU University of Engineering and Technology, Faculty of Information Technology, Hanoi, Vietnam

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Micro-expression recognition is vital for affective computing but remains challenging due to the extremely brief, low-intensity facial motions involved and the high-dimensional nature of 4D mesh data. To address these challenges, we introduce a dual-view optical flow approach that simplifies mesh processing by capturing each micro-expression sequence from two synchronized viewpoints and computing optical flow to represent motion. Our pipeline begins with view separation and sequence-wise face cropping to ensure spatial consistency, followed by automatic apex-frame detection based on peak motion intensity in both views. We decompose each sequence into onset-apex and apex-offset phases, extracting horizontal, vertical, and magnitude flow channels for each phase. These are fed into our Triple-Stream MicroAttNet, which employs a fusion attention module to adaptively weight modality-specific features and a squeeze-and-excitation block to enhance magnitude representations. Training uses focal loss to mitigate class imbalance and the Adam optimizer with early stopping. Evaluated on the multi-label 4DME dataset, comprising 24 subjects and five emotion categories, in the 4DMR IJCAI Workshop Challenge 2025, our method achieves a macro-UF1 score of 0.536, outperforming the official baseline by over 50% and securing first place. Ablation studies confirm that both the fusion attention and SE components each contribute up to 3.6 points of UF1 gain. These results demonstrate that dual-view, phase-aware optical flow combined with multi-stream fusion yields a robust and interpretable solution for 4D micro-expression recognition.

Figures & tables

Figure 1: The workflow of our proposed method
Figure 2: The preprocessing pipeline for 4DME dataset
Figure 3: The architecture of Triple-stream MicroAttNet
Figure 4: The Fusion Attention used in MicroAttNet. It adaptively weights the input streams ( uu , vv , mm ) by generating soft attention weights, producing weighted outputs ( uwu_{w} , vwv_{w} , mwm_{w} ) that emphasize informative motion directions.
RankTeamScore
1Red-Green-Blue (ours)0.536
2WS0.518
3Infinite Messtropy0.476
4Mohammad Javad Shamani0.473
5Benchmark0.355
1MicroAttNet (ours late submission)0.554
Table 1: The top 5 leaderboard of the 4DMR IJCAI Workshop Challenge 2025
ConfigurationFusion AttentionSE BlockUF1 Score
MicroAttNet (ours)XX0.554
Without Fusion Attention X0.524
Without SE BlockX 0.518
Without both  0.510
Table 2: Performance comparison of different model configurations
Figure 5: Impact of Decision Threshold on Multi-label Performance (UF1)
More in this topic

Micro-Expression Recognition

See all 12 papers

Explore similar work

CardsList
Feb 27, 2026

Micro-expression Recognition Based on Dual-branch Feature Extraction and Fusion

Micro-expressions, characterized by transience and subtlety, pose challenges to existing optical flow-based recognition methods. To address this, this paper proposes a dual-branch micro-expression feature extraction network integrated with parallel attention. Key contributions include: 1) a residual network designed to alleviate gradient anishing and network degradation; 2) an Inception network constructed to enhance model representation and suppress interference from irrelevant regions; 3) an adaptive feature fusion module developed to integrate dual-branch features. Experiments on the CASME II dataset demonstrate that the proposed method achieves 74.67% accuracy, outperforming LBP-TOP (by 11.26%), MSMMT (by 3.36%), and other comparative methods.

Mingjie Zhang, Bo Li, Wanting Liu +5

School of Petroleum China University of Petroleum (Beijing) Karamay Campus Karamay, China · Artificial Intelligence Research Institute China Tower Corporation Limited Beijing, China · Low Carbon Business Department South Electric Power Science and Technology Development Co., Ltd. Guangzhou, China +1

Micro-Expression RecognitionFacial Expression RecognitionFeature Fusion
AskPDF
Jun 26, 2026

STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition

Micro-expression recognition is challenging due to subtle and short-lived facial muscle movements. Existing methods rely heavily on apex-onset frames, overlook fine-grained inter-frame dynamics, and separately model spatial and temporal information, limiting generalization across datasets. To address these challenges, we propose STAG, a dynamic ROI-AU-coupled spatial-temporal network that jointly models motion flow and adaptive facial connectivity. The framework extracts optical flow from discriminative frames using magnitude-based selection and temporal attention. A dual-branch architecture combines an enhanced graph attention network for structured spatial reasoning with a transformer encoder for temporal modeling. A bidirectional cross-attention module enables mutual refinement of spatial and temporal features, while AU-guided dynamic connectivity adapts facial region interactions according to muscle activation patterns. The transformer captures subtle temporal dynamics beyond apex-based approaches, improving semantic consistency and interpretability for explainable micro-expression recognition. The fused representation is optimized using focal loss and evaluated on CASME II, 4DME, DFME, NaME, SAMM, and SMIC-HS. Extensive experiments demonstrate improved robustness, generalization, interpretability, and computational efficiency, confirming the effectiveness of adaptive relational reasoning, AU-guided dynamic connectivity, and deep spatial-temporal feature fusion for accurate cross-dataset micro-expression recognition.

Nandani Sharma, Varun Sharma, Dinesh Singh

Vision Intelligence and Machine Learning (VIML) Group, School of Computing and Electrical Engineering, Indian Institute of Technology Mandi, Mandi, 175005, Himachal Pradesh, India · Indian Institute of Information Technology Bhagalpur, Sabour Rd, Bhagalpur, 813210, Bihar, India

Facial Expression RecognitionMicro-Expression RecognitionSpatiotemporal GNNs
AskPDF
Apr 20, 2026

MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition

Unlike macro-expression, micro-expression does not follow a strictly consistent mapping rule between emotions and Action Units (AUs). As a result, some micro-expressions share identical AUs yet represent completely opposite emotional categories, making them highly visually similar. Existing microexpression recognition (MER) methods mostly rely on explicit facial motion cues (e.g., optical flow, frame differences, AU features) while ignoring implicit emotion information. To tackle this issue, this paper presents a Motion Emotion Feature Decoupling Network (MEDN) for MER. We design a dual-branch framework to separately extract motion and emotion features. In the motion branch, an AU-detection task restricts features to the explicit motion domain, and orthogonal loss is adopted to reduce motion emotion feature coupling. For implicit emotion modeling, we propose a Sparse Emotion Vision Transformer (SEVit) that sparsifies spatial tokens to highlight local temporal variations with multi-scale sparsity rates. A Collaborative Fusion Module (CoFM) is further developed to fuse disentangled motion and emotion features adaptively. Extensive experiments on three benchmark datasets validate that MEDN effectively decouples motion and emotion features and achieves superior recognition performance, offering a new perspective for enhancing recognition accuracy and generalization.

Chenxing Hu, Kun Xie, Qiguang Miao +3

School of Computer Science and Technology, Xidian University, Xi’an, Shaanxi 710071, China · Xi’an Key Laboratory of Big Data and Intelligent Vision, Xi’an, Shaanxi 710071, China · Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University, Xi’an 710071, China +1

Disentangled Representation LearningVision TransformerFacial Action Unit DetectionFacial Expression RecognitionMicro-Expression Recognition
AskPDF