Scaling up data and model capacity has driven substantial progress in visual representation learning. However, conventional pretext tasks exhibit diminishing returns on fine-grained facial analysis. We attribute this to a scaling bottleneck: these tasks model each image's appearance and are dominated by identity, pose, and illumination, so subtle within-identity facial differences contribute little to the training signal. We introduce DIME, a differential masked autoencoder for large-scale facial representation learning. DIME builds on a simple observation: two images of the same person share identity-specific appearance, so their difference exposes facial changes. DIME reconstructs both masked images in an identity-matched pair and, through an Explicit Differential Direction Supervision (EDDS) objective, aligns the direction of their reconstructed difference with the true difference, while source isolation makes each reconstruction depend only on its own image. Shared appearance cancels in this difference, and because EDDS ignores magnitude, subtle facial changes count as much as large pose or illumination changes. DIME uses identity labels only to form pretraining pairs, requires no dense annotations, and learns dense facial representations with emergent part-level semantics. We pretrain DIME on 20 million high-resolution facial images consolidated from existing face datasets. DIME's AU detection performance continues to improve as pretraining data scales from 6M to 20M images, with its BP4D F1 increasing by 5.9 points, while MAE gains only 0.5 points on the same subsets. DIME also outperforms specialized facial methods on AU detection, AU intensity estimation, cross-domain anti-spoofing, landmark localization, head pose estimation, face parsing, and face generation.
Figures & tables
Fig. 2 : Emergent capacity of DIME. (a) Cosine similarity between a marked query token (red cross) and all patches, demonstrating that DIME learns robust part-level semantic consistency across distinct facial parts (brighter colors indicate higher similarity). (b) UMAP-PCA [ 12 ] visualizations of dense features from DIME-L, DINOv3-L [ 13 ] , and MAE-L [ 3 ] . DIME exhibits more semantically differentiated and structurally aligned facial representations.
Fig. 3 : Overall architecture of DIME. For simplicity, we illustrate one mixed token grid. In practice, the two reciprocal mixed grids and their complementary decoder branches are batched and processed in parallel to improve efficiency.
Fig. 4 : EDDS in masked image modeling. (a) From the context Vℓ (dots) of two complementary masked views, the model infers the direction of change δℓ at the hidden patch ℓ . EDDS scores a prediction by 1−cosθℓ , so the optimum points along the posterior mean uℓ , whose length reflects predictability. (b) Under MSE, predicting no change costs ∝∥δℓ∥2 , which nearly vanishes for subtle changes such as AUs; under EDDS it costs 1 at every supported patch, whatever the magnitude, recovering the lost signal.
Model
Backbone
Embed. dim
Depth
Decoder dim/depth
Params
DIME-S
ViT-S/16
384
12
384/6
23 M
DIME-B
ViT-B/16
768
12
512/8
86 M
DIME-L
ViT-L/16
1024
24
512/8
307 M
TABLE I : Architectures of DIME. The decoder column reports its embedding dimension and block depth. The Params column counts the encoder only; the decoder and prediction heads, used only in pretraining, are excluded.
Method
AU1
AU2
AU4
AU6
AU7
AU10
AU12
AU14
AU15
AU17
AU23
AU24
Avg. ↑
SD ↓
CISNet [ 48 ]
54.8
48.3
57.2
76.2
76.5
85.2
87.2
66.2
50.9
65.0
47.7
56.5
64.3
13.4
D-PAttNet [ 49 ]
50.7
42.5
59.0
79.4
79.0
85.0
89.3
67.6
51.6
65.3
49.6
54.5
64.5
14.9
CaFNet [ 50 ]
55.1
49.3
57.7
78.3
78.6
85.1
86.2
67.4
52.0
64.4
48.3
56.2
64.9
13.4
ME-GraphAU [ 51 ]
52.7
44.3
60.9
79.9
80.1
85.3
89.2
69.4
55.4
64.4
49.8
55.1
65.5
14.4
MAE-Face [ 6 ]
65.2
55.7
66.9
79.5
79.5
85.2
90.0
67.5
59.7
67.5
54.8
60.9
69.4
11.1
FMAE [ 26 ]
59.2
50.0
62.7
80.0
79.2
84.7
89.8
63.5
52.8
65.1
55.3
56.9
66.6
12.8
TABLE II : Per-AU F1 scores (%) on BP4D [ 1 ] . Best and second-best results are in vermilion and deep blue , respectively.
Method
AU1
AU2
AU4
AU6
AU7
AU10
AU12
AU14
AU15
AU17
AU23
AU24
Avg. ↑
SD ↓
FACS3D-Net [ 58 ]
43.0
38.1
49.9
82.3
85.1
87.2
87.5
66.0
48.4
47.4
50.0
31.9
59.7
19.8
ML-GCN [ 59 ]
40.2
36.9
32.5
84.8
88.9
89.6
89.3
81.2
53.3
43.1
55.9
28.3
60.3
23.6
MS-CAM [ 60 ]
38.3
37.6
25.0
85.0
90.9
90.9
89.0
81.5
60.9
40.6
58.2
28.0
60.5
24.9
SEV-Net [ 61 ]
47.9
40.8
31.2
86.9
87.5
89.7
88.9
82.6
39.9
55.6
59.4
27.1
61.5
23.3
MAE-Face [ 6 ]
53.0
44.5
49.4
87.7
88.8
91.4
90.8
83.2
45.2
50.0
61.2
45.3
65.9
19.6
FMAE [ 26 ]
53.9
45.5
45.9
86.2
88.3
91.2
89.9
82.3
51.3
56.3
60.7
42.7
66.2
18.8
TABLE III : Per-AU F1 scores (%) on BP4D+ [ 2 ] . Best and second-best results are in vermilion and deep blue , respectively.
Table 7
Fig. 5 : Per-AU F1 scores on BP4D (left), BP4D+ (middle), and DISFA (right). Dashed boxes highlight groups containing challenging AUs, where DIME improves detection beyond generic visual encoders and prior facial self-supervised models.
ICM→O
OCM→I
OCI→M
OMI→C
Average
Method
HTER ↓
AUC ↑
HTER ↓
AUC ↑
HTER ↓
AUC ↑
HTER ↓
AUC ↑
HTER ↓
AUC ↑
LBP-TOP [ 7 ]
30.17
77.61
29.14
71.69
36.90
70.80
33.52
73.15
32.43
73.31
SA-FAS [ 68 ]
10.00
96.23
6.58
97.54
5.95
96.55
8.78
95.37
7.83
96.42
PDLE [ 69 ]
11.42
95.52
8.62
97.60
5.41
98.85
10.05
94.27
8.88
96.56
UDG-FAS [ 70 ]
10.97
95.36
5.86
98.62
5.95
98.47
9.82
96.76
8.15
97.30
GAC-FAS [ 71 ]
8.60
97.16
4.29
98.87
5.00
97.56
8.20
95.16
6.52
97.19
TABLE VI : Cross-domain face anti-spoofing results. The generic visual encoders and DIME variants are evaluated using a unified downstream protocol [ 9 ] . Best and second-best results are highlighted in vermilion and deep blue , respectively.
WFLW [ 79 ]
Method
NME ↓
FR ↓
AUC ↑
HRNet-W18 [ 80 ]
4.60
–
–
STAR [ 76 ]
4.02
2.32
60.50
ADNet [ 81 ]
4.14
2.72
60.22
LDEQ [ 82 ]
3.92
2.48
62.40
Faceptor-Full [ 83 ]
4.03
1.92
60.24
TABLE VII : Facial geometry estimation: sparse geometry (landmark detection), global orientation (head pose estimation), and dense semantic structure (face parsing). Best and second-best results are in vermilion and deep blue .
Fig. 6 : Selected face images generated with the DIME-L encoder in the RAEv2 framework with 100K training steps.
RAEv2 [ 99 ]
REPA [ 100 ]
Rep. encoder
Steps
256/FID
512/FID
256/FID
512/FID
DINOv3-L [ 13 ]
100K
12.48
14.83
13.46
16.15
DINOv3-L [ 13 ]
200K
9.83 ↓ 2.65
11.53 ↓ 3.30
10.61 ↓ 2.85
11.80 ↓ 4.35
DINOv3-L [ 13 ]
400K
8.53 ↓ 3.95
8.94 ↓ 5.89
8.22 ↓ 5.24
8.36 ↓ 7.79
DINOv2-L [ 16 ]
100K
14.83 ↑ 2.35
16.79 ↑ 1.96
15.31 ↑ 1.85
18.82 ↑ 2.67
MAE-L [ 3 ]
100K
16.72 ↑ 4.24
18.22 ↑ 3.39
18.34 ↑ 4.88
21.31 ↑ 5.16
TABLE VIII : Face-generation FID [ 101 ] (lower is better) on FFHQ at 256×256 and 512×512 under RAEv2 [ 99 ] and REPA [ 100 ] . Gray arrows give the FID change relative to DINOv3-L at 100K steps. Best and second-best results are in vermilion and deep blue , respectively. Steps: training steps.
Fig. 7 : Face-generation dynamics on FFHQ. (a) Training loss and FID of RAEv2 using MAE, DINOv3, and DIME encoders. DIME consistently converges faster and achieves lower FID. (b) Generated samples from 1K to 50K training steps using DIME (top) and DINOv3 (bottom). DIME produces coherent facial structure earlier and resolves fine details sooner, consistent with its faster FID reduction.
Transfer
Unit
F1 ↑
ICC ↑
NME ↓
HTER ↓
Single (MAE [ 3 ] )
61.9
.619
4.31
11.32
Same image
62.3
.623
4.29
10.91
Random ID
62.8
.630
4.23
10.38
Same ID
62.9
.630
4.22
10.32
Same–random
+0.1
+.001
−.01
−0.06
TABLE IX : Controlled ablation study of DIME. Panels (a)–(c) examine the pretraining unit, the interaction between identity matching and EDDS, and source isolation. Panels (d)–(f) evaluate complementary supervision, facial-change information retained in frozen features, and sensitivity to the source-mixing ratio. All variants: ViT-S/16, 100 epochs, 30% of Face-20M.
Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.
Frozen self-supervised vision models can align parts of generic objects, but it remains unclear whether this correspondence extends to human faces, where global layout is shared while identity-specific appearance varies sharply. We test whether frozen DINOv3 features define a region-level facial coordinate system: a feature space in which eyes, brows, nose, mouth, skin, and hair remain distinguishable across people and across time without face-specific training. Using DINOv3 ViT-L/16 patch embeddings and FaRL only as a face-part labeling interface, we evaluate cross-identity nearest-neighbor matching and temporal label propagation on 200 CelebDF-v2 real videos. DINOv3 achieves 83.0% region-level semantic accuracy under unconstrained cross-identity matching, compared with a 23.0% area-weighted random baseline, and 95.5% temporal tracking accuracy without a learned temporal module. A no-FaRL control collapses to 0.9%, showing that FaRL supplies semantic initialization while DINOv3 supplies dense spatial correspondence. The strongest correspondence appears at an intermediate layer: block 18 gives a 4.93x same-region versus cross-region discrimination ratio, compared with 1.48x at the final block. Against CLIP ViT-L/14, DINOv3 shows only a small aggregate advantage but a +16.8 pp gain on anatomical regions, indicating that image-level contrastive supervision captures coarse facial layout but not fine-grained anatomical identity. These results establish frozen DINOv3 as a strong zero-shot representation for region-level facial correspondence and identify intermediate self-supervised features as the most useful layer for dense face analysis.
Izaldein Al-Zyoud, Abdulmotaleb El Saddik
MCRLab, School of Electrical Engineering and Computer Science University of Ottawa, Ottawa, ON, Canada
Although recent deepfake detectors achieve high overall accuracy, their errors remain unevenly distributed across demographic subgroups, with real faces from certain groups more often misclassified as fake. Existing fairness-aware detectors typically regularise the entire feature representation, without identifying or controlling the specific components that drive unfair predictions. Such coarse intervention can over-suppress useful forgery cues while leaving demographic structure in component-specific subspaces. To address this, we identify two subgroup-sensitive components: multi-scale spatial features, which encode local facial and forgery patterns, and fine-tuning-induced residual features, which adapt the backbone to the unfair training distribution. We propose FairReL, a fairness-aware representation-learning framework that targets both components with dedicated demographic supervision. FairReL uses an SVD-decomposed foundation-model backbone to isolate the fine-tuning-induced residual representation, and introduces two complementary losses. Group-Conditional Wavelet Decorrelation (GCWD) suppresses subgroup-imbalanced structure across spatial wavelet sub-bands, while Subspace-Localised Mean Alignment (SLMA) aligns subgroup means within each real/fake class in the residual representation. Experiments on FF++, Celeb-DF, DFD and DFDC show that, against the state-of-the-art fairness-aware detector, FairReL improves unseen-dataset AUC by 3.9% while reducing subgroup FPR disparity by 10.2%. Code is available at https://github.com/xiaoman89/FairReL .
Xiaoman Lu, Jiaqi Li, Shuntian Zheng +2
Department of Computer Science University of Warwick Coventry, United Kingdom · School of Computer Science University of Birmingham Birmingham, United Kingdom
In this work, we introduce DifFoundMAD, a parameter-efficient D-MAD framework that exploits the generalisation capabilities of vision foundation models (FM) to capture discrepancies between suspected morphs and live capture images. In contrast to conventional D-MAD systems that rely on face recognition embeddings or handcrafted feature differences, DifFoundMAD follows the standard differential paradigm while replacing the underlying representation space with embeddings extracted from FMs. By combining lightweight finetuning with class-balanced optimisation, the proposed method updates only a small subset of parameters while preserving the rich representational priors of the underlying FMs. Extensive cross-database evaluations on standard D-MAD benchmarks demonstrate that DifFoundMAD achieves consistent improvements over state-of-the-art systems, particularly at the strict security levels required in operational deployments such as border control: The error rates reported in the current state-of-the-art were reduced from 6.16% to 2.17% for high-security levels using DifFoundMAD.
Lazaro J. Gonzalez-Soler, André Dörsch, Christian Rathgeb +1
da/sec - Biometrics and Security Research Group Darmstadt, Germany