cs.CVOct 12, 2026

A Multi-Task Deep Learning Framework for Pose-Invariant Face Recognition and Facial Landmark Localization

Authors: Ali Ashkeyev, Zukhra Abdiakhmetova

Organizations: Faculty of Information Technology, Kazakh-British Technical University, Almaty, Kazakhstan · Faculty of Information Technology and Artificial Intelligence, Abai Kazakh National Pedagogical University, Almaty, Kazakhstan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Face recognition systems often experience performance degradation under large pose variations due to changes in facial appearance and visibility of discriminative features. This study presents a multi-task deep learning framework that jointly addresses pose-invariant face recognition and facial landmark localization within a unified architecture. To enable simultaneous optimization of identity recognition and facial structure learning, the original Cross-Pose Labeled Faces in the Wild (CPLFW) dataset was adapted into a multi-task learning dataset by associating facial images with identity labels and five-point landmark annotations. The proposed framework consists of two task-specific branches operating on a common facial input: a ResNet50-based branch for identity recognition and an EfficientNetV2B0-based branch with spatial attention for facial landmark localization. The two objectives are jointly optimized using a weighted multi-task loss, allowing the model to learn complementary semantic and spatial facial representations. The proposed framework was evaluated on the prepared CPLFW-based multi-task dataset. Unlike the original CPLFW benchmark, which focuses primarily on identity recognition, the proposed dataset adaptation incorporates five-point facial landmark annotations to enable joint recognition and localization learning. Experimental results demonstrate that the model achieves a validation recognition accuracy of 81.58% and a facial landmark localization mean squared error of 96.38. The results indicate that combining identity classification and landmark regression within a single optimization framework provides an effective approach for learning robust facial representations under cross-pose conditions The proposed method demonstrates the potential of multi-task learning and attention-based feature refinement for unconstrained facial analysis.

Figures & tables

Fig. 1: Architecture of the proposed dual-branch multi-task framework for pose-invariant face recognition and facial landmark localization. The ResNet50 branch performs identity classification, while the EfficientNetV2B0 branch performs five-point landmark regression using spatial attention.
Fig. 2: Preparation pipeline of the adapted CPLFW dataset for multi-task learning.
Fig. 3: Training and validation accuracy of the face recognition branch during optimization.
Fig. 4: Training and validation loss of the face recognition branch.
Fig. 5: Training and validation mean squared error for facial landmark localization.
Fig. 6: Overall training and validation loss of the proposed multi-task framework.
MetricValue
Training Recognition Accuracy83.87%
Validation Recognition Accuracy81.58%
Training Landmark MSE ( pixel2\mathrm{pixel}^{2} )34.73
Validation Landmark MSE ( pixel2\mathrm{pixel}^{2} )96.38
TABLE I: Quantitative evaluation of the proposed multi-task framework.
Fig. 7: Qualitative visualization of identity prediction and facial landmark localization during training.
More in this topic

Face Recognition

Face recognition matches or identifies people from facial images, videos, and biometric representations across changing conditions.

See all 107 papers

Explore similar work

CardsList
Aug 11, 2026

Towards Unified Dynamic Face Landmark Detection

Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parameters need to be trained independently for each $N$-point'' benchmark dataset, and (2) a model trained on an NN-point'' dataset reliably outputs only the NN landmarks. In our work, we first conceptualize Face Part-Anchored Landmark Positions (FPALPs), wherein each landmark is treated as a progression value between zero (start) and one (end) along a face part's contour. Every landmark can be expressed in the FPALP format, irrespective of its source dataset, hence unlocking the ability to unify all $N$-point'' datasets into a single dataset. Secondly, we represent each landmark with an FPALP-based query, refine it progressively with a cross-modality decoder, and predict its coordinates based on the final representation. Our approach, called Unified Dynamic FLD, embodies these two design choices and streamlines the landmark detection pipeline by enabling (1) a single model to learn on any number of NN-point'' datasets, and (2) yield any number of specific landmark predictions by loading the designated landmark queries at runtime. Extensive experiments on multiple benchmark datasets show that our method delivers these benefits while remaining competitive with, and in several cases outperforming existing state-of-the-art methods.

Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang +2

University of Toronto · 2ModiFace · †Work completed while employed at ModiFace +1

Topic pending
AskPDF
Sep 9, 2026

FreqFLD: Towards All-in-One Facial Landmark Detection via Frequency Modulation

Recent progress in deep learning has significantly advanced facial landmark detection. However, most existing methods process features in a spatial-domain manner under a dataset-specific training paradigm, which overlooks the fact that facial landmark detection is inherently geometry-driven and sensitive to frequency variations, thereby limiting cross-dataset generalization under complex scenarios and hindering the development of a facial landmark detection model. To address this issue, we propose FreqFLD, a frequency-modulated framework towards All-in-One facial landmark detection. Specifically, FreqFLD introduces a Frequency Modulation Module (FreqMoM) to explicitly induce the frequency prior by decoupling and modulating low- and high-frequency components, which is then injected into subsequent feature modeling to enable balanced modeling of global facial structure and local landmark details. Furthermore, FreqFLD employs a Frequency-Modulated Mixture-of-Experts (FreqMoE), with expert selection adaptively conditioned on frequency-modulated priors, enabling flexible modeling of heterogeneous facial landmark patterns under diverse and challenging scenarios. To regularize frequency-consistent modeling under the All-in-One paradigm, we further introduce a Frequency-Consistent Routing (FreqCR) loss, which constrains the routing and assignment of frequency-aware experts to promote balanced expert utilization across diverse facial scenarios, thereby enabling stable expert specialization and achieving robust facial landmark detection. Extensive experiments demonstrate that the proposed FreqFLD achieves comparable performance on popular datasets. The code is available at: https://github.com/jkj1059657014/FreqFLD.

Shun Ren, Kaijie Jin, Shengkai Hu +5

College of Computer and Information Technology, China Three Gorges University, Yichang 443002, China · School of Information Engineering, Zhongnan University of Economics and Law, Wuhan 430073, China · National Institute of Natural Hazards, Ministry of Emergency Management of China, Beijing 100085, China +3

Mixture of ExpertsFrequency-Domain Feature LearningDomain Generalization
AskPDF
May 17, 2025

Facial Recognition Leveraging Generative Adversarial Networks

Face recognition performance based on deep learning heavily relies on large-scale training data, which is often difficult to acquire in practical applications. To address this challenge, this paper proposes a GAN-based data augmentation method with three key contributions: (1) a residual-embedded generator to alleviate gradient vanishing/exploding problems, (2) an Inception ResNet-V1 based FaceNet discriminator for improved adversarial training, and (3) an end-to-end framework that jointly optimizes data generation and recognition performance. Experimental results demonstrate that our approach achieves stable training dynamics and significantly improves face recognition accuracy by 12.7% on the LFW benchmark compared to baseline methods, while maintaining good generalization capability with limited training samples.

Zhongwen Li, Zongwei Li, Xiaoqi Li

School of Cyberspace Security, Hainan University, Haikou, China

Face RecognitionData AugmentationSynthetic Data AugmentationGenerative Adversarial Networks
AskPDF