Identical twin face verification represents an extreme fine-grained recognition challenge where even state-of-the-art systems fail due to overwhelming genetic similarity. Current face recognition methods achieve over 99.8% accuracy on standard benchmarks but drop dramatically to 88.9% when distinguishing identical twins, exposing critical vulnerabilities in biometric security systems. The difficulty lies in learning features that capture subtle, non-genetic variations that uniquely identify individuals. We propose the Asymmetric Hierarchical Attention Network (AHAN), a novel architecture specifically designed for this challenge through multi-granularity facial analysis. AHAN introduces a Hierarchical Cross-Attention (HCA) module that performs multi-scale analysis on semantic facial regions, enabling specialized processing at optimal resolutions. We further propose a Facial Asymmetry Attention Module (FAAM) that learns unique biometric signatures by computing cross-attention between left and right facial halves, capturing subtle asymmetric patterns that differ even between twins. To ensure the network learns truly individuating features, we introduce Twin-Aware Pair-Wise Cross-Attention (TA-PWCA), a training-only regularization strategy that uses each subject's own twin as the hardest possible distractor. Extensive experiments on the ND_TWIN dataset demonstrate that AHAN achieves 92.3% twin verification accuracy, representing a 3.4% improvement over state-of-the-art methods.
Figures & tables
Figure 1: Visualization of the generated attention map.
Figure 2: Overview of the AHAN architecture. The model processes face images through a Vision Transformer ( Dosovitskiy 2020 ) backbone and three parallel streams: (1) Global Self-Attention for holistic features, (2) Hierarchical Cross-Attention (HCA) for multi-scale region analysis, and (3) Facial Asymmetry Attention Module (FAAM) for asymmetry signatures. Twin-Aware Pair-Wise Cross-Attention (TA-PWCA) uses the subject’s twin as a distractor during training only.
Component
Setting
Value
Optimizer
Adam
lr=1e-4, wd=5e-4
Schedule
Cosine
100 epochs
Batch Size
64
4 accum steps
HCA Regions
4
eyes, nose, mouth, jaw
HCA Scales
3
1× , 2× , 4×
TA-PWCA Prob
0.5
Training only
Table 1: Key hyperparameters and training configuration.
Method
Backbone
Key Innovation
ArcFace
ResNet-100
Angular margin
CosFace
ResNet-100
Cosine margin
AdaFace
IR-101
Adaptive margin
CurricularFace
IR-101
Curriculum learning
UniFace
ResNet-100
Unified cross-entropy
TransFace
ViT-B/16
Vision transformer
Table 2: Comparison with baseline methods
Method
General Verification
Twin Verification
Hard Twin Verification
Acc
AUC
TAR@1%FAR
Acc
AUC
TAR@1%FAR
Acc
AUC
TAR@1%FAR
Non Transformer-based Face Recognition
ArcFace
98.5
99.4
95.6
88.9
93.8
82.4
85.3
90.6
78.4
MagFace
98.4
99.3
95.4
88.5
93.4
81.9
85.0
90.3
78.0
AdaFace
98.2
99.2
95.1
88.2
93.1
81.5
84.7
90.0
77.6
CosFace
98.0
99.1
94.8
87.5
92.5
80.6
84.1
89.4
76.8
Table 3: Main results on ND_TWIN dataset. Best results in bold.
Configuration
TAR@ FAR=0.1%
EER (%)
TVA (%)
Baseline (ViT-B)
52.1
24.7
81.2
+ HCA
63.4
19.3
87.1
+ FAAM
58.9
21.8
84.7
+ TA-PWCA
67.8
17.1
89.9
+ HCA + FAAM
69.2
16.4
90.4
+ HCA + TA-PWCA
74.6
14.2
91.8
Table 4: Ablation study results on Hard Twin Verification.
Past literature on face recognition for monozygotic (("identical") twins points to facial marks and mirror asymmetry as possible directions for improved accuracy of twins recognition. The Celeb Twins Test Set (CTTS) contains web-scraped image pairs for 80 sets of celebrity twins. It is the only twins test set with meta-data for twins with distinguishing skin marks and possible mirror asymmetry. CTTS is organized in the manner of face verification test sets such as LFW, CALFW, CPLFW, CFP-FP, and AgeDB-30. Current deep CNN matchers can achieve over 76% accuracy in classifying CTTS same-person / different-person image pairs. We show that current matchers do not make use of skin marks, or asymmetry, and discuss reasons for this. Finally, we discuss the feasibility of using generative AI tools such as Grok, ChatGPT and Gemini to create images of imagined monozygotic twins as a means to increase representation of twins in face recognition training sets.
Michael Zang, Haiyu Wu, Mrinal Sharma +1
Department of Computer Science & Engineering University of Notre Dame
Heterogeneous Face Recognition (HFR) aims at matching face images captured across different sensing modalities, such as thermal-to-visible or near-infrared-to-visible, enhancing the usability of face recognition systems in challenging real-world conditions. Although recent HFR methods have achieved significant improvements in performance, many rely on computationally expensive models, making them impractical for deployment on resource-limited edge devices. In this work, we introduce a lightweight yet effective HFR framework by adapting a hybrid CNN-Transformer model originally developed for RGB homogeneous face recognition. Our approach enables efficient end-to-end training with only a small amount of paired heterogeneous data, while still maintaining strong performance on standard RGB face recognition benchmarks. This makes it suitable for both homogeneous and heterogeneous settings. Comprehensive experiments on several challenging HFR and face recognition benchmarks show that our method achieves state-of-the-art or competitive performance while keeping computational requirements low.
Anjith George, Sebastien Marcel
Idiap Research Institute, Martigny, Switzerland · Université de Lausanne (UNIL), Lausanne, Switzerland
In face recognition systems, facial templates are widely adopted for identity authentication due to their compliance with the data minimization principle. However, facial template inversion technologies have posed a severe privacy leakage risk by enabling face reconstruction from templates. This paper proposes a Layer-Based Facial Template Inversion (LBFTI) method to reconstruct identity-preserving fine-grained face images. Our scheme decomposes face images into three layers: foreground layers (including eyebrows, eyes, nose, and mouth), midground layers (skin), and background layers (other parts). LBFTI leverages dedicated generators to produce these layers, adopting a rigorous three-stage training strategy: (1) independent refined generation of foreground and midground layers, (2) fusion of foreground and midground layers with template secondary injection to produce complete panoramic face images with background layers, and (3) joint fine-tuning of all modules to optimize inter-layer coordination and identity consistency. Experiments demonstrate that our LBFTI not only outperforms state-of-the-art methods in machine authentication performance, with a 25.3% improvement in TAR, but also achieves better similarity in human perception, as validated by both quantitative metrics and a questionnaire survey.
Zixuan Shen, Zhihua Xia, Kaikai Gan +1
College of Cyber Security, Jinan University · Guangzhou, Guangdong, China