The absence of large-scale masked face datasets challenges masked face detection and recognition. We propose a two-step generative data augmentation framework combining rule-based mask warping with unpaired image-to-image translation via GANs, producing masked face samples that go beyond rule-based overlays. Trained on about 19,100 images in the target domain (3.8% of IAMGAN's scale), or, including out-of-domain transfer pretraining, 59,600 and 11.8%, the proposed approach yields consistent improvements over rule-based warping alone and achieves results complementary to IAMGAN's, showing that both steps contribute. Evaluation is conducted directly on the generated samples and is qualitative; quantitative metrics like FID and KID were not applied as any real reference distribution would unfairly favor the model with closer training data. We introduce a non-mask preservation loss to reduce non-mask distortions and stabilize training, and stochastic noise injection to enhance sample diversity. Note: The paper originated as a coursework project completed under resource constraints. Following scholarship termination, the author took on part-time employment to maintain research continuity, which led to a mid-semester domain pivot from medical imaging to masked face tasks due to company data restrictions. The work was completed alongside concurrent coursework with delayed compute access and without AI assistance. It was submitted at the semester end to meet a publication requirement, accepted without revision requests, and selected among the best papers for an extended submission to Springer Nature Computer Science, which was not pursued due to continued funding absence. Downstream evaluation on recognition or detection performance was not completed by the submission deadline. The note is added in response to subsequent comparisons and criticisms that did not account for these conditions.
Figures & tables
Figure 1: Examples from dataset B.
Figure 2: Generator architecture in AttentionGAN. Our noise inputs are depicted in red.
Figure 3: Feed noise to the last two content-generating layers without training.
Figure 4: Uniform-colored masks generated at epoch 279.
Figure 5: Models drastically distort inputs at two different epochs. Outputs like images 2 and 4 intermittently appear as the training progresses.
Epochs
Lambda A /Lambda B
Noise Input
Non-Mask Change Loss
Dataset Size
Training Data Selection Restrictions Added to B
1 ∼ 60
10/10
No
No
9,517
Simple mask, fully occluded
61 ∼ 90
10/10
No
No
8,938
Front facing
91 ∼ 140
8/8
No
No
1,695
No pitch/roll, light-colored medical mask
141 ∼ 298
8/8
No
Yes
1,695
None
299 ∼ 510
8/8
Yes
Yes
1,695
None
Table 1: Transfer learning from trial-and-error experiments. All epochs use Learning Rate 0.0002 and Lambda_identity 0.5.
Figure 6: Results produced by the model at epoch 313. Input and output images are paired side by side.
Figure 7: Results produced by the model at epoch 476. Input and output images are paired side by side.
Figure 8: Noisy output in epoch 476 test result. (a) Red and white blocks in hair. (b) Patterns on the mask.
Generative Adversarial Networks (GANs) can help overcome data scarcity in computer vision tasks by generating additional training samples. In this work, we explore generative data augmentation in two low-resource domains: Bangla handwritten character recognition and chest X-ray image analysis. We use DCGAN-based models trained on 64x64 images to generate synthetic samples and evaluate their quality using Inception Score (IS), Fréchet Inception Distance (FID), and visualization methods such as t-SNE and UMAP. To measure practical usefulness, we train image classifiers using real data and a combination of real and synthetic data. Experimental results show that synthetic augmentation improves data diversity and consistently increases classification performance in limited-data settings. We also investigate training stability techniques, including gradient penalty and spectral normalization, and perform ablation studies on synthetic-to-real data ratios and sample filtering strategies. In addition, we discuss challenges related to medical image evaluation, dataset licensing, and privacy concerns of synthetic data. Our approach is simple, reproducible, and provides a strong baseline for generative augmentation in resource-constrained imaging applications.
Md. Sohanuzzaman Soad, Mahady Al Hady, S M Rafiuddin Rifat +1
Department of Computer Science and Engineering University of Asia Pacific Dhaka, Bangladesh
The shortage of legally compliant data for face recognition training has sparked growing interest in using synthetic data as an alternative. While recent diffusion-based methods enable the generation of photorealistic face images with strong identity adherence and data diversity, their downstream recognition performance still exhibits a significant synthetic-real gap. This paper identifies visual tendency as a previously underexplored limitation, whereby synthetic data exhibit an unrealistic prevalence of visual attributes and thus deviate from the real-data distribution. Visual tendency can be attributed to the generator's conditioning on identity embeddings, through which co-occurring residual visual cues are unintentionally absorbed into learned identity semantics. To discourage the generator from exploiting such visual cues, this paper proposes SteerFace, a simple and efficient training framework that perturbs identity embeddings by steering them toward random orthogonal directions on the embedding hypersphere. The perturbation serves as an identity-preserving regularizer that penalizes the generator's reliance on non-identity components, as supported by theoretical analysis. This paper further introduces an adaptive strategy that learns perturbation strengths with both sample-wise preference and favorable overall statistics. Extensive experiments show that SteerFace effectively mitigates visual tendency, outperforms prior methods in downstream face recognition, and generalizes well across different training datasets and generation pipelines.
Yuxi Mi, Qiuyang Yuan, Jianqing Xu +5
Fudan University Shanghai, China · Youtu Lab, Tencent Shanghai, China · WeChat Pay Lab33, Tencent Shenzhen, China
Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on external foundation models or datasets, whose usage is restricted in many scenarios due to policy or legal constraints. We propose ScoreMix, a self-contained synthetic generation method to produce hard synthetic samples for recognition tasks by leveraging the score compositionality of diffusion models. The approach mixes class-conditioned scores along reverse diffusion trajectories, yielding domain-specific data augmentation without external resources. We systematically study class-selection strategies and find that mixing classes distant in the discriminator's embedding space yields larger gains, providing up to 3% additional average improvement, compared to selection based on proximity. Interestingly, we observe that condition and embedding spaces are largely uncorrelated under standard alignment metrics, and the generator's condition space has a negligible effect on downstream performance. Across 8 public face recognition benchmarks, ScoreMix improves accuracy by up to 7 percentage points, without hyperparameter search, highlighting both robustness and practicality. Our method provides a simple yet effective way to maximize discriminator performance using only the available dataset, without reliance on third-party resources. Paper website: https://parsa-ra.github.io/scoremix/.