What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering
Authors: Amir Hossein Saleknia, Mohammad Sabokrou
Organizations: School of Electrical Engineering, Iran University of Science and Technology (IUST), Tehran, Iran · Okinawa Institute of Science and Technology (OIST), Okinawa, Japan
In computer vision, a prevailing method for quantifying dataset bias is to train a model to distinguish between datasets. High classification accuracy is then interpreted as evidence of meaningful semantic differences. This approach assumes that standard image augmentations successfully suppress low-level, non-semantic cues, and that any remaining performance must therefore reflect true semantic divergence. We demonstrate that this fundamental assumption is flawed within the domain of large-scale natural image collections. High classification accuracy is often driven by resolution-based artifacts, which are structural fingerprints arising from native image resolution distributions and interpolation effects during resizing. These artifacts form robust, dataset-specific signatures that persist despite conventional image corruptions. Through controlled experiments, we show that models achieve strong dataset classification even on non-semantic, procedurally generated images, proving their reliance on superficial cues. To address this issue, we revisit this decades-old idea of dataset separability, but not with supervised classification. Instead, we introduce an unsupervised approach that measures true semantic separability. Our framework directly assesses semantic similarity by clustering semantically-rich features from foundational vision models, deliberately bypassing supervised classification on dataset labels. When applied to major web-scale datasets, the primary focus of this work, the high separability reported by supervised methods largely vanishes, with clustering accuracy dropping to near-chance levels. This reveals that conventional classification-based evaluation systematically overstates semantic bias by an overwhelming margin.
Figures & tables
Figure 1: The "Name That Dataset" game [ 36 ] aware of resolution distributions. These images are sampled from the YFCC and CC datasets. We maintain their relative sizes. Additionally, we provide a plot showing the resolution distribution of the training samples from each dataset. Can you guess which dataset each image is from? Once resolution effects are considered, dataset separability becomes significantly more apparent. (Answer: YFCC: 3, 5, 6, 8, 9; CC: 1, 2, 4, 7)
Figure 2: The overview of our proposed unsupervised semantic bias assessment pipeline. Images from multiple datasets are embedded via a generic pretrained feature extractor (e.g., DINOv2), clustered using K-Means, and evaluated using clustering accuracy and normalized mutual information (NMI). Dimensionality reduction via UMAP is applied as an auxiliary step for computational tractability before clustering. See A for comprehensive robustness analysis.
ConvNeXt V2
EfficientVit
MViTv2
YCD
84.21%
85.01%
86.53%
YCDLW
68.20%
69.85%
68.02%
Table 1: The results of three baseline models on the YCD and YCDLW image sets.
Figure 3: Normalized confusion matrices (per-class percentage) for three models evaluated on YCD. Each matrix shows the performance of: (left) ConvNeXt V2-Tiny, (center) MViTv2-Tiny, and (right) EfficientVit-B3.
Figure 4: Normalized confusion matrices (per-class percentage) for three models evaluated on YCDLW. Each matrix shows the performance of: (left) ConvNeXt V2-Tiny, (center) MViTv2-Tiny, and (right) EfficientVit-B3.
Figure 5: KDE plots of average image resolution for YFCC, CC, and DataComp datasets, and misclassified samples from three baseline models combined.
Figure 6: KDE plots of average image resolution for YFCC, CC, DataComp, WIT, and LAION datasets, along with the resolution distribution of misclassified samples aggregated across all three baseline models.
Figure 7: Several samples of generated fake images at 100×100 resolution.
Figure 8: Distribution of average image resolutions across datasets, with DataComp dominating at resolution 100 and CC at resolution 640.
Model
Resolution
YFCC
CC
DataComp
ConvNeXt V2
100
80
112
808
640
264
510
226
MViTv2
100
153
101
746
640
247
493
260
EfficientVit
100
102
67
831
640
196
571
233
Table 2: Predictions for YFCC, CC, and DataComp at two resolutions (100 and 640) for each model.
Model
Normal Training
Two-Step Resizing
ConvNeXt V2
84.21%
70.91%
MViTv2
86.53%
68.36%
EfficientVit
85.01%
71.64%
Table 3: Performance of three baseline models on YCD with Normal training vs Two-step resizing.
Figure 9: Residual images highlighting artifacts introduced by resizing.
Model
Normal Images
Residual Images
ConvNeXt V2
84.21%
88.46%
MViTv2
86.53%
89.37%
EfficientVit
85.01%
88.87%
Table 4: Classification performance of baseline models on normal vs. residual images.
Figure 10: KDE plots of the average image resolution in the CC dataset using different sample sizes (20, 200, 2000, and full dataset).
Dataset
Task
Acc(%)
NMI (%)
MIT-67
Scene Classification
69.65
82.32
Stanford-40
Action Recognition
70.89
75.81
CIFAR-10
Object Classification
81.80
88.73
OxfordIIITPet
Fine-grained Classification
84.08
91.18
Table 5: Clustering accuracy and NMI using DINOv2-S features followed by UMAP and k-means clustering.
YCD
Method
Acc (%)
NMI (%)
Random Chance
33.33
N/A
DINOv2 + Fine-tuning
85.67
N/A
DINOv2 + Linear Probing
66.91
N/A
DINOv2 (Ours)
46.95
6.60
YCDLW
Table 6: Clustering accuracy and NMI using DINOv2-S features followed by UMAP and K -means clustering on the YCD and YCDLW image sets. NMI is only applicable to unsupervised clustering. Accuracy for supervised models reflects direct label prediction; for clustering, it is computed via optimal label matching.
Dataset
DINOv2-S
DINOv2-B
DINOv2-L
YCD
46.95
46.40
45.99
YCDLW
31.47
31.35
29.87
MIT-67
69.65
74.50
81.79
Stanford-40
70.89
76.26
76.88
CIFAR-10
81.80
92.92
96.43
Oxford-Pet
84.08
85.37
88.67
Table 7: Clustering accuracy (%) across DINOv2 variants and datasets.
Dataset
DINOv2-S
DINOv2-B
DINOv2-L
YCD
6.60
6.74
6.58
YCDLW
6.32
6.12
5.38
MIT-67
82.32
85.53
88.20
Stanford-40
75.81
80.15
82.85
CIFAR-10
88.73
96.43
98.12
Oxford-Pet
91.18
92.21
93.05
Table 8: NMI (%) across DINOv2 variants and datasets.
Dataset
DINOv2-B
ConvNeXtV2-B
Acc (%)
NMI (%)
Acc (%)
NMI (%)
YCD
46.40
6.74
47.34
7.61
YCDLW
31.35
6.12
29.68
6.18
Table 9: Comparison of clustering performance between DINOv2 and ConvNeXtV2 on YCD and YCDLW.
Figure 11: Example image from the DataComp dataset before and after applying super-resolution. The semantic content remains unchanged while resolution details vary.
Method
Acc (%)
NMI (%)
Random Chance
50.00
N/A
DINOv2 + Fine-tuning
95.36
N/A
DINOv2 + Linear Probing
76.61
N/A
DINOv2 + k-NN
65.27
N/A
DINOv2 (Ours)
50.01
0.00
Table 10: Clustering accuracy and NMI between original and super-resolved versions of the DataComp dataset.
Figure 12: Visual characterization of semantic bias. Correctly clustered images reveal the dominant themes behind residual separability: CC is characterized by a strong presence of people and indoor images , DataComp by commercial, object-centric photography, and YFCC by natural and urban views .
Figure 13: Pipeline for semantic characterization using CLIP. Image-prompt similarity is used to assign each correctly clustered image to a dominant semantic theme for quantitative analysis.
Dataset
Semantic Category (%)
Person/Indoor
Commercial
Scenic/Outdoor
CC
86.01
8.50
5.49
DataComp
11.72
84.05
4.23
YFCC
10.82
3.37
85.81
Table 11: Quantitative semantic characterization of correctly clustered images.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Model
Reduced Dimension
20
30
40
50
YCD
DINOv2-S
46.95
46.99
46.92
46.93
DINOv2-B
46.40
46.43
46.45
46.47
YCDLW
DINOv2-S
31.47
31.65
31.25
31.10
DINOv2-B
31.35
31.50
31.15
30.95
CIFAR-10
DINOv2-S
81.80
81.74
81.76
81.86
Appendix
Table 12: Clustering accuracy (%) across UMAP reduced dimensions.
Dataset
Model
Reduced Dimension
20
30
40
50
YCD
DINOv2-S
6.60
6.64
6.58
6.58
DINOv2-B
6.74
6.72
6.73
6.76
YCDLW
DINOv2-S
6.32
6.45
6.25
6.15
DINOv2-B
6.12
6.25
6.05
5.95
CIFAR-10
DINOv2-S
88.73
88.83
88.84
88.62
Appendix
Table 13: NMI (%) across UMAP reduced dimensions.
Setting
YCD
CIFAR-10
Acc
NMI
Acc
NMI
UMAP (20 dim) - our standard
46.95
6.60
81.80
88.73
PCA (20 dim)
39.96
4.17
75.61
73.92
No reduction (raw features)
40.02
4.19
75.10
74.07
Appendix
Table 14: Clustering performance (%) with different dimensionality reduction strategies using DINOv2-S features.
Algorithm
YCD
CIFAR-10
Acc
NMI
Acc
NMI
K-means (our standard)
46.95
6.60
81.80
88.73
Agglomerative Clustering
44.49
4.91
84.48
88.22
Spectral Clustering
47.06
7.81
71.18
77.80
Appendix
Table 15: Clustering performance (%) with different clustering algorithms using UMAP-reduced DINOv2-S features.
k
2
3
4
5
6
7
8
9
10
NMI (%)
9.02
6.60
6.29
6.50
6.34
6.37
6.54
6.45
7.43
Appendix
Table 16: NMI (%) for YCD image sets with varying number of clusters using DINOv2-S features.
We present LatentDiff, a scalable framework for semantic dataset comparison that operates directly in the latent space of pretrained vision encoders. By combining sparse autoencoder-based divergence testing with density ratio estimation, LatentDiff identifies interpretable semantic differences between datasets at a fraction of the computational cost of caption-based alternatives. We also introduce Noisy-Diff, a benchmark capturing realistic sparse distribution shifts that cause existing methods to struggle. Experiments demonstrate that LatentDiff achieves superior accuracy while remaining robust to settings where an extremely small fraction of images (from 5% to <1% ) differ semantically.
James Flora, Kowshik Thopalli, Akshay R. Kulkarni +2
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, USA · Lawrence Livermore National Laboratory, Livermore, California, USA · University of California San Diego, California, USA
Vision-Language Models (VLMs) such as CLIP are now foundational to multimodal systems, yet their robustness to spurious correlations remains poorly understood at scale. We present the first large-scale empirical study of 194 publicly available VLMs, including 16 model families, covering a wide range of model sizes, 24 training datasets, and three evaluation benchmarks, namely ImageNet (overall performance), CelebA (typical single-attribute bias), and UrbanCars (complex multi-attribute biases). Across these settings, the Spearman correlation between model scale and performance weakens as evaluation shifts from ImageNet (ρ=0.68) to single-attribute (ρ=0.48) and further to multi-attribute (ρ=0.05) bias benchmarks. In contrast, properties of the training data (size and quality) show more consistent relationships with worst-group accuracy across both bias benchmarks. Notably, curated datasets yield improvements of up to 25% over uncurated alternatives at a comparable scale. Finally, the effect of architectural choices (e.g., patch size, image resolution) is highly context-dependent, varying with the nature of the benchmark, including the type of bias and its spatial distribution within images.
Ioannis Sarridis, Ioannis Kompatsiaris, Symeon Papadopoulos
Information Technologies Institute, CERTH Thessaloniki, Greece
Labeled datasets reflect the biases of their annotation pipelines, which sometimes introduce label bias: group-conditional label errors that cause systematic performance disparities across demographic subgroups. Label bias in image segmentation remains underexplored, as even detecting it typically requires clean, unbiased annotations, which are not readily available. We present a data-centric adaptation of Confident Learning to segmentation, allowing detection of label bias directly in the training data without a clean, unbiased ground truth. By comparing the provided training labels to the model's confident predictions, we isolate directional errors that quantify the presence and nature of bias, where standard overlap metrics like Dice fail. We further show that label bias influences subgroup separability in the encoder's feature space, an artifact we leverage for bias mitigation rather than suppressing it. We evaluate three datasets, spanning from synthetic to real-life bias, showing how our framework reliably detects and mitigates bias without access to clean labels, achieving equitable performance across experimental conditions.
Aditya Parikh, Stella Frank, Sneha Das +1
Department of Applied Mathematics and Computer Science Technical University of Denmark