Synthetic Image Oversampling for Rare-Class Representation Learning in Deep Facial Phenotyping

Miles Liao; Zhanliang Wang; Hongzhuo Chen; Kai Wang

cs.CVOct 16, 2026

Synthetic Image Oversampling for Rare-Class Representation Learning in Deep Facial Phenotyping

Authors: Miles Liao, Zhanliang Wang, Hongzhuo Chen, Kai Wang

Organizations: Hopkins Authentic Research Program for Students, Hopkins School, New Haven, CT 06515, USA · Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, USA · Department of Mathematics, University of Pennsylvania, Philadelphia, PA 19104, USA · Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA

Abstract

Recent deep facial phenotyping methods can support the recognition of genetic syndromes, but patient images are often scarce and unevenly distributed across disorders. Synthetic facial images offer a way to supplement the training data without collecting additional patient photographs. Their usefulness, however, depends on whether they improve recognition of real patients: generated images may introduce artifacts or repeat features already represented in the training set. It is therefore necessary to examine both how much synthetic augmentation to use and which disorders benefit. We investigate disease-specific synthetic oversampling using images from the Phenotypic Disease Image Database to augment a subset of the GestaltMatcher Database. We compare increasing augmentation levels in an image-based model, GestaltMatcher-Arc, and a multimodal model, GestaltMML, while keeping the real training and patient-disjoint evaluation partitions fixed. Repeated training runs assess overall and disease-specific accuracy, and an exploratory embedding analysis examines changes in class compactness and separation. Augmentation improves mean recognition accuracy in both training setups, although gains remain modest in GestaltMatcher-Arc and continue across the tested levels in GestaltMML. These aggregate improvements are accompanied by uneven disease-level responses, including reduced accuracy for some disorders. In the embedding analysis, disorders with fewer real training images tend to gain more class separation, but geometric changes do not consistently track classification performance. Together, the findings support synthetic oversampling for limited-data facial phenotyping and emphasize the need to assess its amount and benefit for individual disorders within each training setup.

Figures & tables

Fig. 1: Study design. A patient-disjoint GMDB partition contains 1,289 real training images and 558 real evaluation images. Nested PDIDB subsets augment the training data under PD25, PD50, PD75, and PD100. GestaltMatcher-Arc and GestaltMML are trained with three seeds per condition. The real partitions and synthetic subsets are fixed across training seeds.

Disorder	Train	Eval.	PD25	PD50	PD75	PD100
Cornelia de Lange	312	135	46	91	137	182
Williams-Beuren	191	82	47	93	140	186
Noonan	145	62	48	95	143	190
Kabuki	144	62	40	81	121	161
KBG	125	54	47	94	140	187
Angelman	111	49	42	85	127	169

TABLE I: Dataset composition. Train and Eval. are the fixed, patient-disjoint real-image partitions. The Baseline condition consists of the real training images only, while PD25–PD100 indicate the numbers of synthetic images added under the four augmentation conditions.

Fig. 2: PDIDB synthetic facial images. Two examples are shown for each of the five disorders with the fewest real training images. Synthetic images are used only for training.

Fig. 3: Classification accuracy across augmentation conditions. Bars show mean overall and mean per-class Top-1 accuracy across three training seeds; error bars show one sample standard deviation. Checkpoints are selected and evaluated on the same fixed real-image partition.

Fig. 4: Disease-specific changes in Top-1 accuracy from Baseline. Values are mean

\pm

sample standard deviation of paired percentage-point differences across three training seeds. Each difference is calculated against the Baseline run with the same seed. Disorders are ordered by real training image count, from largest to smallest.

Fig. 5: Shared UMAP projection of seed-11 GestaltMatcher-Arc embeddings at Baseline and PD50. Each point represents one of the same 558 real evaluation images; colors indicate disorder. Quantitative metrics use the original $L_{2}$ -normalized, 512-dimensional embeddings.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Condition	Top-1	Mean Top-1	Top-3	Top-5
GestaltMatcher-Arc	Baseline	89.19 $\pm$ 0.92	87.23 $\pm$ 1.18	97.25 $\pm$ 0.27	98.27 $\pm$ 0.27
	PD25	89.96 $\pm$ 0.65	89.15 $\pm$ 0.80	97.01 $\pm$ 0.27	98.45 $\pm$ 0.37
	PD50	90.20 $\pm$ 0.58	88.30 $\pm$ 0.73	97.61 $\pm$ 0.10	98.86 $\pm$ 0.21
	PD75	90.02 $\pm$ 0.45	88.24 $\pm$ 1.08	97.25 $\pm$ 0.37	98.69 $\pm$ 0.27
	PD100	89.84 $\pm$ 0.52	88.31 $\pm$ 0.24	97.13 $\pm$ 0.36	98.63 $\pm$ 0.10
GestaltMML	Baseline	68.82 $\pm$ 1.53	64.02 $\pm$ 2.14	86.26 $\pm$ 0.83	93.55 $\pm$ 1.12

Appendix

TABLE II: Overall classification performance across synthetic augmentation conditions. Top-

k

denotes sample-weighted Top-

k

accuracy, while Mean Top-1 denotes Top-1 accuracy averaged equally across the 10 disease classes. Values are mean

\pm

sample SD across three training seeds, in percentages. Green indicates the highest-observed mean and red indicates the lowest-observed mean within each architecture and metric.

Synthetic Data Augmentation

Synthetic data augmentation creates or modifies training examples to improve learning when real data are scarce, imbalanced, costly, or sensitive.

See all 456 papers

FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases
Oct 7, 2026Hongzhuo Chen, Zhanliang Wang, Florent Pollet +9Rare Disease Diagnosis Synthetic Data Evaluation
Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA, USA · Applied Mathematics and Computational Science Graduate Program, University of Pennsylvania, Philadelphia, PA, USA · Department of Biomedical Informatics, Columbia University, New York, NY, USA +6
Multi-Level Evidence Aggregation for Robust Facial Phenotype Retrieval in Rare Genetic Disorder Prioritization
Aug 11, 2026Alexander Hustinx, Carolin Kaffiné, Behnam Javanmardi +2Rare Disease Diagnosis Image Matching
Institute for Genomic Statistics and Bioinformatics, University Hospital Bonn, Bonn, Germany
Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks
May 12, 2026Daniil Dushenev, Nazariy Karpov, Daniil Zinovjev +2Class-Imbalanced Learning Synthetic Data Augmentation
National University of Science and Technology MISIS, Moscow, Russia

Oct 7, 2026J/K move · Enter open · S save

FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases

Hongzhuo Chen, Zhanliang Wang, Florent Pollet, Mian Umair Ahsan+8

Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA, USA · Applied Mathematics and Computational Science Graduate Program, University of Pennsylvania, Philadelphia, PA, USA · Department of Biomedical Informatics, Columbia University, New York, NY, USA+6

Rare Disease Diagnosis Synthetic Data Evaluation Synthetic Data Generation Synthetic Data Augmentation

Open paper AskPDF arXiv

Many rare genetic diseases are associated with recognizable craniofacial features. However, traditional approaches for describing facial morphology rely largely on qualitative clinical observation and free-text descriptions, which are often subjective, non-standardized, and difficult to reproduce across observers and institutions. Although the Human Phenotype Ontology (HPO) provides controlled terms for describing facial features, these terms are typically categorical rather than quantitative and may vary depending on examiner experience and interpretation. Here, we present FaceKit, a computational framework for quantitative facial phenotyping from frontal facial photographs. FaceKit extracts standardized measurements of facial landmarks and derived 120 morphological features, then reports feature-level z-scores representing deviation from population reference distributions. The reference distributions are built from the FairFace dataset spanning diverse ancestral groups. We evaluated FaceKit on a curated subset of the GestaltMatcher Database covering 50 rare-disease cohorts. In addition to quantitative facial analysis, FaceKit includes synthetic facial image generation to support rare disease model development and data augmentation. We also performed privacy evaluation to assess whether synthetic images reveal identifiable information from real patient photographs and could compromise patient privacy. Across disease case studies, FaceKit-derived quantitative measurements captured known facial features associated with rare genetic disorders and provided objective support for clinical phenotyping. Together, these results establish FaceKit as a useful tool for quantitative phenotyping, and has the potential to improve rare disease diagnosis, support genotype-phenotype studies, and enable more reproducible clinical characterization across diverse patient populations.

Abstract

Synthetic Data Augmentation

Explore similar work

FaceKit: a Toolkit for Interpretable Facial Phenotyping, Synthetic Image Generation and Privacy Analysis in Rare Diseases

Multi-Level Evidence Aggregation for Robust Facial Phenotype Retrieval in Rare Genetic Disorder Prioritization

Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks