SEEC: A Large-Scale Dataset and Benchmark for Self-Supervised External-Eye Representation Learning
Authors: George R. Nahass, Michelle Fleishaker, Sanjay Ganesh, Sasha Hubschman, R. V. Paul Chan, Chad A. Purnell, Pete Setabutr, Ann Q. Tran, Sathya N. Ravi
Organizations: Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA · Department of Computer Science, University of Illinois Chicago, Chicago, IL, USA · Artificial Intelligence in Ophthalmology, University of Illinois Chicago, Chicago, IL, USA · Department of Plastic Surgery, University of Illinois Chicago, Chicago, IL, USA
External-eye photography captures clinically relevant periocular anatomy at low acquisition cost, yet no large-scale standardized resource exists for evaluating self-supervised external-eye representations. We introduce a 4.35-million-image Standardized External-Eye Corpus (SEEC) derived from fourteen heterogeneous public face datasets, together with a benchmark for external-eye representation learning. For evaluation, we provide held-out evaluation distributions, landmark-annotated anatomical probe sets, and clinical disease-classification experiments spanning ptosis, thyroid eye disease, and craniofacial syndromes. We evaluate representations at two complementary levels: (a) the representation level, by assessing embedding geometry across multiple data distributions, and (b) the functional level, by assessing anatomical recovery, out-of-distribution anatomical transfer, and disease classification. Our models include ResNet-101 and ViT-B encoders pretrained using InfoNCE, VICReg, and LeJEPA objectives and compared with DINOv2 and MAE. Our results indicate that representation-level and functional-level evaluations are dissociable: broader embedding-space utilization does not consistently correspond to anatomical transfer or disease-classification performance, and no single encoder is universally optimal. Domain-specific external-eye encoders outperform DINOv2 and MAE on cross-dataset anatomical transfer despite being pretrained at substantially smaller scale, while external-eye-pretrained ViT models match or outperform these general-purpose baselines on frozen disease-classification probes. We release benchmark code, evaluation scripts, and preprocessing pipelines to reproduce SEEC at https://monkeygobah.github.io/seec-project/
Figures & tables
Figure 1: Overview of SEEC construction and benchmark resources. (A) SEEC construction pipeline. Fourteen public facial-image datasets were sequentially standardized and curated to produce 4.35 million unilateral 224×224 periocular images and a 135,000-image high-resolution subset. For datasets requiring automated periocular filtering, manual annotation and blinded regrading were used to train dataset-specific classifiers with source-specific filtering thresholds. (B) Source-image retention across major stages of SEEC construction. Counts through periocular filtering represent bilateral source images prior to left/right splitting. (C) Benchmark resources used for self-supervised pretraining and evaluation. Fixed SEEC subsets of 10,000, 100,000, and 1 million images were used for pretraining; held-out open-source and clinical datasets were used for embedding-geometry, anatomical-landmark, and disease-classification evaluations.
Dataset
Role
N
Labels
Public Release
SEEC
Master Corpus
4,352,692
Unlabeled
Partial†
Pretrain-10K
Pretraining
10,000
Unlabeled
Partial†
Pretrain-100K
100,000
Pretrain-1M
1,000,000
Holdout
Open-Source Geometry
421,730
Unlabeled
Partial†
Open-HR
134,969
Table 1: Dataset components and their roles in the benchmark. † indicates components released directly where source licenses permit redistribution of derived images; for the remaining public-source components, we provide preprocessing code, manifests, crop coordinates, and reconstruction tools to allow authorized users to rebuild the benchmark. ‡ indicates clinical components used under institutional data-use restrictions and not publicly released.
Figure 2: Overview of the benchmark tasks. (A) Encoders are pretrained under a controlled self-supervised benchmark strategy. (B) Frozen embeddings are evaluated using geometry metrics across distributions. (C) Frozen embeddings are used for anatomical landmark probing in in-distribution and cross-dataset transfer settings. (D) Disease classification is evaluated using frozen linear probes and five-epoch fine-tuning. For embeddings, border color denotes the evaluation dataset, and fill color denotes the pretraining objective or baseline model.
Figure 3: Effective rank normalized by feature dimension for ResNet-101 (top) and ViT-B (bottom) projector representations across training-set scale, SSL objective, initialization, and evaluation datasets.
Figure 4: Within-dataset LM-Celeb landmark probe performance for ResNet-101 and ViT encoders. Lines show SSL objectives across training-set scale; dashed references show init-only and external ViT baselines.
Model
Initialization
Objective
Mean L2
Mean MAE
Iris Centroid
Medial Canthus
Lateral Canthus
ViT
DinoV2
DinoV2
5.04
3.23
4.91
5.55
5.67
MAE
MAE
8.89
5.88
7.53
9.80
10.47
ResNet-101
ImageNet
Baseline
12.08
7.90
10.95
15.04
13.36
InfoNCE
6.04
3.87
5.40
6.55
6.67
LeJEPA
6.82
4.36
6.17
8.81
7.47
VicReg
5.51
3.51
5.13
7.00
6.01
Table 2: Celeb-to-CFD landmark transfer performance for frozen ResNet-101 and ViT encoders. Probes were trained and selected on LM-Celeb and evaluated on LM-CFD test images; errors are reported in 224-pixel crop coordinates.
Figure 5: Representative Celeb-to-CFD landmark predictions from out-of-distribution landmark probes. Columns show probes trained on 1M-image random-initialized ResNet-101 and ViT models using VICReg, InfoNCE, or LeJEPA. For each objective, the top row shows the best CFD test example and the bottom row shows the worst example by mean landmark L2 error. Ground-truth landmarks are shown in teal and predictions in red.
Model
Objective
Accuracy
Recall
F1
Training of Encoder and Classifier for 5 Epochs
ViT-B/16
DINOv2
0.40
0.26
0.34
MAE
0.89
0.85
0.89
ResNet-101
InfoNCE
0.71
0.66
0.70
LeJEPA
0.56
0.47
0.53
VICReg
0.71
0.66
0.70
ViT-B/16
InfoNCE
0.87
0.82
0.88
Table 3: Disease classification performance on held-out external-eye images using frozen linear probes and five-epoch joint training of the full backbone and linear classifier. Accuracy, F1, and recall are reported on the held-out test set; F1 and recall are averaged across disease classes. Bold and italics indicate best and second best respectively for each experimental setting.
Figure 6: Qualitative visualization of disease-labeled embeddings after supervised fine-tuning. t-SNE projections are shown for MAE and VICReg ViT-B encoders after five epochs of fine-tuning on the disease classification task. Points represent external-eye images and are colored by disease class. t-SNE is shown for qualitative visualization only; quantitative comparisons are based on held-out classification metrics in Table 3 .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Source Images
Exact Duplicates
Images After Duplicate Removal
Grayscale (%)
RGB (%)
CFD
1,207
0
1,207
0.00
100.00
CFD-I
146
0
146
0.00
100.00
CFD-MR
88
0
88
0.00
100.00
Tufts
560
0
560
0.00
100.00
FEI
2,800
0
2,800
0.00
100.00
FIML
7,049
555
6,494
100.00
0.00
Appendix
Table 4: Source dataset composition following duplicate removal. The table reports the number of images contributed by each source dataset, the number of exact duplicates identified within each source dataset, the number of images retained after duplicate removal, and image-format distributions.
Dataset
Initial Positives (Grader 1)
Initial Negatives (Grader 1)
Positive Agreement ( n=100 )
Negative Agreement ( n=100 )
Negative Regrade Agreement (All Negatives)
CelebA
1,449
1,051
0.99
0.57
0.47
FIML
940
1,560
0.97
0.68
0.66
IMDB-WIKI
952
1,548
1.00
0.59
0.55
MORPH
2,399
101
0.99
0.53
0.28
UMD Faces
1,309
1,191
0.99
0.59
0.44
UTKFace
1,808
692
1.00
0.39
0.33
Appendix
Table 5: Manual periocular annotation and intergrader agreement by source dataset. Grader 1 labeled 2,500 randomly sampled images per dataset as periocular or non-periocular. A second blinded grader independently reassessed randomly sampled positive and negative images and all images initially classified as negative.
Dataset
Positive
Negative
Gray
CelebA
1,449
494
557
FIML
940
1,028
532
IMDB-WIKI
952
845
703
MORPH
2,399
28
73
UMD Faces
1,309
523
668
UTKFace
1,808
228
464
Appendix
Table 6: Final periocular annotation distribution following blinded regrading. Images were categorized as periocular, non-periocular, or ambiguous (gray).
Dataset
Accuracy
Precision
Recall
F1
AUROC (95% CI)
Selected Threshold
Selected FPR
Selected TPR
CelebA
0.95
0.97
0.93
0.95
0.99 [0.98–1.00]
0.92
0.014
0.85
FIML
0.97
0.96
0.97
0.97
0.99 [0.98–1.00]
0.95
0.028
0.93
IMDB-WIKI
0.95
0.96
0.94
0.95
0.99 [0.98–1.00]
0.70
0.024
0.92
MORPH
1.00
1.00
1.00
1.00
1.00 [1.00–1.00]
0.50
0.000
1.00
UMD Faces
0.96
0.95
0.97
0.96
1.00 [0.99–1.00]
0.74
0.013
0.97
UTKFace
0.85
0.80
0.94
0.86
0.95 [0.88–0.99]
0.89
0.000
0.59
Appendix
Table 7: Performance and selected operating thresholds of dataset-specific ResNet-18 classifiers for periocular image detection. Accuracy, precision, recall, F1 score, and area under the receiver operating characteristic curve (AUROC) with 95% confidence intervals are reported on the held-out test sets. FPR and TPR denote false-positive and true-positive rates, respectively, at the selected operating threshold.
Figure 7: Qualitative assessment of CNN-based periocular filtering. (A) Representative low-confidence images retained by the dataset-specific classifiers near the selected probability threshold. (B) Example distributions of predicted periocular probabilities for held-out positive and negative samples and manually designated ambiguous images. These analyses were used to assess whether the selected operating thresholds limited retention of non-periocular images while preserving images with discernible periocular anatomy.
Dataset
Original Images
Post- Deduplication
Post-Alignment/ Preprocessing
Post-Periocular Filtering
224 × 224 Unilateral
512 × 512 High Resolution
CFD
1,207
1,207
1,207
1,207
2,414
2,414
CFD-I
146
146
146
146
292
292
CFD-MR
88
88
88
88
176
176
Tufts
560
560
336
336
672
672
FEI
2,800
2,800
1,182
1,182
2,364
4
FIML
7,049
6,494
6,317
2,870
5,740
5,732
Appendix
Table 8: Source-level image counts across the principal stages of SEEC construction. Counts are reported for the original source collections, after exact duplicate removal, after face detection, alignment, and source-specific preprocessing, after periocular filtering, and after generation of the standardized unilateral 224×224 and high-resolution 512×512 corpora.
Dataset
Source use terms
Redistribution provision
CelebA
Non-commercial research use
Restricted
UTKFace
Non-commercial research use
Not expressly specified
FFHQ
Mixed Creative Commons/source-image licenses
Permitted subject to applicable license terms
VGGFace2
Research use
Not expressly specified
Tufts Face Database
Non-commercial research and educational use
Restricted
Chicago Face Database (CFD)
Non-commercial scientific research use
Restricted
Appendix
Table 9: Source-specific use and redistribution terms for datasets contributing to SEEC. “Restricted” indicates that redistribution is expressly limited or prohibited by the source terms; “Not expressly specified” indicates that research use is permitted but redistribution of derived images is not explicitly addressed.
Figure 8: Top-20 explained variance of projected embeddings across architecture, objective, initialization, pretraining scale, and evaluation distribution. Lower values indicate less concentration of variance in the leading covariance directions.
Figure 9: Standard deviation of sampled pairwise cosine similarities after centering and row-normalizing projected embeddings. Lower values indicate less angular concentration among samples.
Figure 10: Within-dataset LM-CFD landmark probe performance for ResNet-101 and ViT-B/16 encoders. ViT models achieve strong landmark prediction performance, with external DINOv2 and MAE baselines shown as dashed references.
Self-supervised learning (SSL) offers a promising approach to medical image representation learning, yet augmentation strategies of SSL frameworks validated on natural images do not reliably transfer to specialized medical imaging domains. We analyze DINO, a knowledge-distillation based SSL framework, and empirically demonstrate that its default augmentation strategy produces unreliable representations for retinal fundus images, degrading downstream glaucoma classification performance. Guided by the principle that effective cropped views should share minimal redundant information while retaining clinically relevant signal, we redesign DINO's augmentation strategy. Our approach biases the learned representations toward diagnostically critical features rather than low-level image statistics. We evaluate our method for glaucoma classification by training a linear classifier on the frozen representations, achieving strong generalization across seven benchmark datasets. URL for code and weights: https://github.com/ashish-256/task_dino
Ashish Kumar Meena, Chandra Sekhar Seelamantula
Department of Electrical Engineering, Indian Institute of Science, Bengaluru, India
Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pretraining strategies influence the transferability of learned representations to weakly supervised ophthalmic imaging tasks. We investigate this question in ultra-widefield (UWF) retinal imaging by evaluating foundation model representations within a patch-based multiple instance learning (MIL) framework for disease classification on UWF images. We compare Vision Transformer encoders pretrained with supervised, Masked Autoencoder (MAE), and self-distillation objectives, while keeping the downstream aggregation architecture unchanged. Within a controlled comparison of ViT-B encoders pretrained on ImageNet-1k, the choice of pretraining objective substantially influenced frozen representation transfer, with supervised and self-distillation-based models outperforming MAE. A contemporary DINOv3 model pretrained at a larger scale achieved the strongest overall performance, with a quadratic weighted kappa of 0.863 for five-class diabetic retinopathy grading, comparable with DINOv1. Attention analysis further revealed distinct patch-aggregation behaviours associated with the different pretrained representations, while partial fine-tuning substantially reduced the performance gap for MAE. These findings suggest that pretraining strategy influences both representation transferability and the subsequent aggregation of patch-level evidence within MIL, resulting in differences in downstream classification performance.
Mingya Alexa Gong, Da Ma, Lovre Antonio Budimir +7
Institute of Ophthalmology, University College London, London, United Kingdom · Wake Forest University School of Medicine, Winston-Salem, NC, USA · Virginia Tech-Wake Forest University School of Biomedical Engineering and Sciences, Blacksburg, VA, USA +5
Annotated medical images remain scarce and costly to obtain, which motivates self-supervised learning (SSL). We present a controlled empirical study of two SSL frameworks, SimSiam and SimCLR, for retinal fundus classification, asking whether retina-specific augmentations (Graham's local-mean subtraction and CLAHE) improve representations beyond standard augmentation and grayscale conversion. ResNet-18 backbones are pretrained on 48K unlabeled fundus images and evaluated by linear probing and fine-tuning on EDID (multi-disease) and RetinaMNIST (DR grading). On EDID, grayscale conversion accounts for most of SimSiam's linear-probe gain, and Graham or CLAHE add little on top of it. After fine-tuning, SimSiam with Graham+Gray exceeds an ImageNet-supervised baseline on RetinaMNIST (+3.0 accuracy points) but only matches it on EDID. Retina-specific augmentation therefore helps inconsistently and should be validated per SSL method and task.
Bekzat Nurlanbekova, Fung Fung Ting
∗† School of Information Technology Monash University Malaysia, Malaysia