Machine learning progress has historically prioritized model-centric innovations, yet achievable performance is frequently capped by the intrinsic complexity of the data itself. In this work, we isolate and quantify the impact of instance density (measured by face count) as a primary driver of data complexity. Rather than simply observing that ``crowded scenes are harder,'' we rigorously control for class imbalance to measure the precise degradation caused by density alone. Controlled experiments on the WIDER FACE and Open Images datasets, restricted to exactly 1 to 18 faces per image with perfectly balanced sampling, reveal that model performance degrades monotonically with increasing face count. This trend holds across classification, regression, and detection paradigms, even when models are fully exposed to the entire density range. Furthermore, we demonstrate that models trained on low-density regimes fail to generalize to higher densities, exhibiting a systematic under-counting bias, with error rates increasing by up to 4.6x, which suggests density acts as a domain shift. These findings establish instance density as an intrinsic, quantifiable dimension of data hardness and motivate specific interventions in curriculum learning and density-stratified evaluation.
Figures & tables
Fig. 1: Visualizing Instance Complexity (WIDER FACE): Increasing density creates intrinsic difficulties (occlusion, scale, entanglement) regardless of model capacity.
Fig. 2: Exp 1 : Misclassifications rate (%) vs. classification pair in the n -vs- n+1 chain on WIDER FACE and Open Images.
Fig. 3: Exp 2 : Classification accuracy vs. gap size k for low-base ( n=1 ) and high-base ( n=10 ) series on WIDER FACE and Open Images.
Fig. 4: Exp 2 : Average accuracy and MCC across all gaps.
Fig. 5: Exp 3 : MAE vs. true face count when training only on 1 to 9 faces.
Fig. 6: Exp 3 : Prediction bias (pred - true count) vs. true face count.
Fig. 7: Exp 4 : MAE in Low (1 to 6), Medium (7 to 12), and High (13 to 18) bins after full balanced training.
Fig. 8: Exp 4 : MSE vs. exact face count (CSRNet, full end-to-end training).
Fig. 9: Exp 5 : MAE vs. true face count for three state-of-the-art off-the-shelf detectors on WIDER FACE and Open Images.
Fig. 10: Exp 6 (control) : Prediction bias (predicted − true count) vs. true face count when training on the full balanced 1 to 18 distribution.
Fig. 11: Exp 7: Stability Analysis. Comparison of prediction bias between our Balanced Model (100 samples/count) and a model trained on the Full Biased WIDER FACE set (all available images, thousands per count).
Estimated density is often interpreted as indicating how typical a sample is under a model. Yet deep models trained on one dataset can assign higher density to simpler out-of-distribution (OOD) data than to in-distribution test data. We refer to this behavior as the OOD anomaly. Prior work typically studies this phenomenon within a single architecture, detector, or benchmark, implicitly assuming certain canonical densities. We instead separate the trained network from the density estimator built from its representations or outputs. We introduce two estimators: Jacobian-based estimators and autoregressive self-estimators, making density analysis applicable to a wide range of models. Applying this perspective to a range of models, including iGPT, PixelCNN++, Glow, score-based diffusion models, DINOv2, and I-JEPA, we find the same striking regularity that goes beyond the OOD anomaly: lower-complexity samples receive higher estimated density, while higher-complexity samples receive lower estimated density. This ordering appears within a test set and across OOD pairs such as CIFAR-10 and SVHN, and remains highly consistent across independently trained models. To quantify these orderings, we introduce Spearman rank correlation and find striking agreement both across models and with external complexity metrics. Even when trained only on the lowest-density (most complex) samples - or even a single such sample - the resulting models still rank simpler images as higher density. These observations lead us beyond the original OOD anomaly to a more general conclusion: deep networks consistently favor simple data. Our goal is not to close this question, but to define and visualize it more clearly. We broaden its empirical scope and show that it appears across architectures, objectives, and density estimators.
Current research on deep machine unlearning primarily focuses on improving or evaluating the overall effectiveness of unlearning methods while overlooking the varying difficulty of unlearning individual training samples. As a result, the broader feasibility of machine unlearning remains under-explored. This paper studies the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis over various unlearning algorithms and datasets. In particular, we summarize four factors that make unlearning a data point difficult, and we empirically show that these factors are independent of a specific unlearning algorithm but only relevant to the target model and its training data. Given these findings, we argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.
Hammad Rizwan, Mahtab Sarvmaili, Hassan Sajjad +1
Department of Computer Science, Dalhousie University, Canada.
Despite steady progress in face recognition, current face recognition models still suffer from significant demographic biases. While approaches for bias mitigation have been proposed, existing methods often impose constraints on the training procedure and result in the degradation of recognition accuracy. To address this issue, we here introduce a method that reduces racial bias in pre-trained face recognition models without compromising their accuracy. To this end, we model face embeddings of each person by von Mises-Fisher (MF) distribution. We next observe the dependency between demographic attributes and the density of MF distributions, and propose DenseFace, a probabilistic face matching procedure that accounts for differences in MF distributions. Our extensive experiments demonstrate DenseFace to consistently reduce racial bias in strong face recognition models varying in network architectures, training datasets and loss functions. Notably, DenseFace preserves recognition accuracy and requires no retraining of the underlying face recognition model. Our work also investigates previously adopted bias measures and makes suggestions.
Mansur Bultygov, Vadim Seliutin, Dmitry Nekhaev +1
VisionLabs · Amazon Web Services · Mohamed bin Zayed University of Artificial Intelligence