Facial Action Unit Detection

Ask about this topic

Overview

Facial Action Unit (AU) detection maps facial images or video to labels for visible muscle actions, often predicting several AUs at once and identifying subtle or brief movements. Methods use convolutional or transformer-based visual features, facial landmarks, temporal models, and graphs of AU co-occurrence; newer approaches also use language-based semantic priors, multimodal cues, and controlled synthetic data. Systems are commonly evaluated on BP4D and DISFA, with in-the-wild tests such as Aff-Wild2 and specialized protocols for micro-AUs. A central challenge is reliable generalization across people, datasets, and recording conditions, especially with scarce, imbalanced labels and evaluation results that can vary substantially across data splits.

Latest papers

CardsList

23 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding

    Oct 12, 2026Hao Yu, Haoyu Chen, Hui Wei +3Visual Representation LearningSelf-Supervised Visual Representation Learning

    ELLIS Institute Finland and CMVS, University of Oulu, Finland · DISI, University of Trento, Italy

  2. MoTop: Motion-Topological Model For Micro AU Detection

    Sep 25, 2026Huai-Qian Khor, Mengting Wei, Yante Li +2Facial Action Unit DetectionMotion Representation Learning

    CMVS, University of Oulu Oulu, Finland · AII, CAAS, Beijing, China Zhongyuan Research Center, CAAS, Henan, China · University of Malaya Kuala Lumpur, Malaysia +1

  3. A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources

    Sep 6, 2026Xuri Ge, Tianshuo Zhang, Ruihan Li +6Facial Expression RecognitionFacial Action Unit Detection

    Shandong University, Qingdao Key Laboratory of Trustworthy Artificial Intelligence, Shandong, China · University of Glasgow, United Kingdom · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

  4. Spatiotemporal Facial Action Unit Detection using Twin Cycle Autoencoders for Driver Monitoring

    Jul 18, 2026Sai Sidharth DAutoencodersFacial Action Unit Detection

  5. AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis

    Jul 17, 2026Dipit Saha, Mohammad Raihan Rashid, Shah Mohammad Abdul Mannan +2Multi-Task LearningFacial Action Unit Detection

    Bangladesh University of Engineering and Technology

  6. Task-Specific Feature Fusion Method for Multi-Task Affective Behavior Analysis

    Jul 15, 2026Jiajun Sun, Zhe GaoMulti-Task LearningFacial Action Unit Detection

    Shanghai Normal University, Shanghai, China

  7. AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow

    Jul 14, 2026Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Fadi Dornaika +1Multi-Task LearningFacial Action Unit Detection

    University of the Basque Country (UPV/EHU), Spain · Department of Innovation Engineering, University of Salento & Institute of Applied Sciences and Intelligent Systems (CNR), Italy · Faculty of Data Science and Computing, Universiti Malaysia Kelantan, Malaysia

  8. Causal Supervision of Attention for Affective Behaviour Analysis

    Jul 13, 2026Nemanja Rašajski, Konstantinos Makantasis, Antonios Liapis +1Causal Representation LearningFacial Action Unit Detection

    Institute of Digital Games, University of Malta, Malta · AI Department, University of Malta, Malta

  9. Facial Affect Analysis for Service-Oriented Systems: Advances, Challenges, and Future Visions

    Jun 13, 2026Spyridon Georgiou, Aggelos Psiris, Thomas Lagkas +4Software EngineeringFacial Action Unit Detection

    Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · Department of Informatics, Faculty of Science, Democritus University of Thrace, Kavala, Greece · Department of Networks and Digital Media, Kingston University, Kingston upon Thames, United Kingdom +1

  10. Horse Eye Blink Detection and Classification for Equine Affective State Assessment

    Jun 3, 2026João Alves, Signe Møller-Skuldbøl, Pia Haubro Andersen +1Facial Action Unit DetectionFacial Expression Recognition

    Visual Analysis and Perception Lab, Aalborg University · Department of Animal Biosciences, Swedish University of Agricultural Sciences

  11. Generating 3D models from sketches of human faces using a combined approach of Convolutional Neural Networks, Procedural Modeling, and Contour Mapping

    May 25, 2026Nancy IskanderFacial Action Unit Detection3D Face Reconstruction

    Graduate Department of Computer Science University of Toronto

  12. From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competition

    May 24, 2026Dimitrios Kollias, Panagiotis Tzirakis, Alan Cowen +10Facial Action Unit DetectionValence-Arousal Modeling

    Queen Mary University of London (QMUL), UK · Hume AI, USA · Google Deepmind, USA +7

  13. UAU-Net: Uncertainty-aware Representation Learning and Evidential Classification for Facial Action Unit Detection

    Apr 23, 2026Yuze Li, Zhilei LiuRepresentation LearningEvidential Deep Learning

    School of Artificial Intelligence Tianjin University Tianjin, China

  14. MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition

    Apr 20, 2026Chenxing Hu, Kun Xie, Qiguang Miao +3Disentangled Representation LearningVision Transformer

    School of Computer Science and Technology, Xidian University, Xi’an, Shaanxi 710071, China · Xi’an Key Laboratory of Big Data and Intelligent Vision, Xi’an, Shaanxi 710071, China · Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University, Xi’an 710071, China +1

  15. Bidirectional Learning of Facial Action Units and Expressions via Structured Semantic Mapping across Heterogeneous Datasets

    Apr 12, 2026Jia Li, Yu Zhang, Yin Chen +5Facial Action Unit DetectionFacial Expression Recognition

    School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China · School of Computer Science and Engineering, and the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, Nanjing 210096, China · Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China, and also with the University of Chinese Academy of Sciences, Beijing 100049, China

  16. ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning

    Apr 10, 2026Shifeng Liu, Zhengye Zhang, Sirui Zhao +7Facial Expression RecognitionAffective Computing

    State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, China · Tsinghua University, Beijing, China · University of Science and Technology of China, Hefei, China +2

  17. Beyond the Fold: Quantifying Split-Level Noise and the Case for Leave-One-Dataset-Out AU Evaluation

    Apr 2, 2026Saurabh Hinduja, Gurmeet Kaur, Maneesh Bilalpur +2Cross-ValidationFacial Action Unit Detection

    CGI Technologies and Solutions Inc · University of Pittsburgh · University of South Florida

  18. Micro-AU CLIP: Fine-Grained Contrastive Learning from Local Independence to Global Dependency for Micro-Expression Action Unit Detection

    Mar 17, 2026Jinsheng Wei, Fengzhou Guo, Yante Li +3Micro-Expression RecognitionFacial Action Unit Detection

    Nanjing university of posts and telecommunications

  19. HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classification

    Mar 13, 2026Andrey V. Savchenko, Kseniia TsypliakovaAffective ComputingFacial Expression Recognition

    Sber AI Lab · HSE University Moscow, Russia

  20. Hierarchical Granularity Alignment and State Space Modeling for Robust Multimodal AU Detection in the Wild

    Mar 11, 2026Jun Yu, Yunxiang Zhang, Naixiang Zheng +2Facial Action Unit DetectionMultimodal Robustness

    University of Science and Technology of China

  21. AULLM++: Structured-Token-Conditioned Large Language Models for Micro-Expression Action Unit Detection

    Mar 9, 2026Zhishu Liu, Kaishen Yuan, Bo Zhao +2Facial Action Unit DetectionFacial Expression Recognition

    Great Bay University, Dongguan, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

  22. Controlled Face Manipulation and Synthesis for Data Augmentation

    Feb 22, 2026Joris Kirchner, Amogh Gudi, Marian Bittner +1Synthetic Data AugmentationImage Editing

    Vicarious Perception Technologies (VicarVision) · Delft University of Technology

  23. Hierarchical Vision-Language Interaction for Facial Action Unit Detection

    Feb 16, 2026Yong Li, Yi Ren, Yizhe Zhang +5Facial Action Unit DetectionVision-Language Models

    the School of Computer Science and Engineering, and the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, Nanjing 210096, China. · the School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China. · Key Laboratory of Child Development and Learning Science (Ministry of Education), School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China. +1