With limited annotation budgets, choosing which images to label determines how much a model improves. Data-selection methods that use features from a separately trained model, or scene descriptions written by vision-language models, have been successful, but those signals do not directly capture changes in the model being improved. The target model's own internal features reflect what it has learned so far and change with retraining, making them a natural cue for choosing the next training data. However, feature rarity alone does not reveal the errors that matter for performance. Here we link internal features to prediction errors and their expected impact on performance and select images for labeling and retraining without using labels for candidate images. We evaluated the method with an object detector on two datasets and two pairs of random seeds. Adding internal features improved the identification of prediction errors in 15 of 16 conditions. When performance was averaged over successive labeling rounds, the method outperformed selection based only on feature rarity in all four evaluation settings and ranked among the top two of six methods. With other conditions held fixed, performance after retraining was again higher than with rarity-based selection, even though the latter collected more errors. With longer retraining, the proposed method ranked first among six methods. These results suggest that linking a model's internal features to its errors and their effects on performance may help select training images that improve performance, thereby allowing the model's current state to guide which images are labeled next.
Figures & tables
Figure 1Figure 2
Dataset
Seeds
Pro- posed
Ran- dom
External k -center
BADGE
Output entropy
Rarity only
Rank
BDD100K
11/1
13.929
13.308
13.829
12.803
12.809
12.881
1st
BDD100K
18/0
13.248
13.182
13.707
12.736
12.298
12.134
2nd
COCO 2017
11/1
15.839
15.959
14.707
15.664
15.484
15.228
2nd
COCO 2017
18/0
15.735
15.541
14.346
15.336
14.887
14.850
1st
Table 3
Figure 4Figure 5Figure 6
Method
Information used
Selection rule and role
Random sampling
Candidate image IDs and the image-selection seed
Draws uniformly at random from the candidate set. Serves as the reference set before the proposed method replaces images.
k -center selection on features from a separate model
Image features from a ResNet-50 that is separate from the detector under training
Normalizes the 2,048-dimensional features of a ResNet-50 pretrained on ImageNet-1K and repeatedly selects the image farthest in feature space from the labeled set. Uses neither the outputs nor the internal features of the target detector.
Output entropy
Final outputs of the target detector
Aggregates, per image, the binary entropy computed from the top-class confidence of each predicted box. Uses neither internal features nor calibrated error probabilities.
Selection by rarity of the internal representation
Internal features of the target detector
Computes the minimum cosine distance between each image’s internal representation and those of the images labeled so far, and selects the most distant images. Uses neither output probabilities, error types, nor error probabilities.
BADGE adapted to object detection
Internal features and top-class confidence of the target detector
L2-normalizes the mean internal feature of each image, scales it by 1 minus the maximum box confidence, and draws the batch by k -means++ seeding on these vectors. Retains the uncertainty and diversity components of BADGE without loss gradients (Methods). Compared as the pre-specified representative of existing methods that use the internal representation.
Proposed method
Internal features, outputs, box geometry and neighbor information of the target detector, and a separate 1,500-image labeled set for error estimation
Estimates the error type and probability of each box in the unlabeled candidates and computes an image priority from the effect of each error type, the sufficiency of the evidence and the estimated annotation effort. Starts from the same random draw and replaces, under constraints, only images whose priority difference is confirmed by the stability bounds.
Annotation errors are widespread in computer vision datasets and can significantly degrade the performance of systems trained on them, particularly in complex tasks such as object detection. Several approaches exist to identify annotation errors, including training-free feature-space methods which provide a fast and interpretable way to analyze annotations. However, the behavior on object detection annotations, which include semantic and spatial information, remains largely unexplored. In this work we analyze the applicability of feature-space-based approaches for detecting annotation errors in object detection datasets. By adapting an existing feature-space method, we show that such approaches reliably expose semantic mislabel, while positional errors remain difficult to detect. We evaluate this behavior across multiple pretrained embedding models, synthetic noise types (symmetric, asymmetric, and positional), and real-world annotation errors using VOC2012 and KITTI. All code and real-world corruptions are publicly available at the following repository: https://github.com/ ChristianSieberichs/BoundingBox_corruption_detection
Christian Sieberichs, Simon Geerkens, Thomas Waschulzik +2
University of Applied Sciences Düsseldorf · Siemens Mobility GmbH · Goethe University Frankfurt
Data quality is a critical factor in the effectiveness of machine learning models. Label errors, present even in widely used benchmarks, introduce noise into training data and reduce model generalization. In this work, we conduct a comparative analysis of two automatic label error detection methods - Confident Learning and Dataset Cartography - on three Russian text classification corpora of varying size, number of classes, and domain: ru_emotion_e-culture (49,123 examples, emotion classification), RuCoLA (8,524 examples, linguistic acceptability), and TERRa (2,337 examples, textual entailment recognition). We use the pre-trained rubert-base-cased model fine-tuned on each corpus. To verify the meaningfulness of filtering, we conduct control experiments with random removal of an equivalent number of examples. Results show that the effectiveness of both methods depends strongly on dataset characteristics: on large corpora with low noise levels, filtering does not improve performance, while on small datasets with high noise, Confident Learning achieves a significant F1-macro improvement. Dataset Cartography demonstrates more conservative behavior, removing fewer examples. Across all corpora, targeted removal by both methods outperforms random removal, confirming the meaningfulness of the approaches.
Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently. We quantify this requirement for the area under the generalized risk-coverage curve (AUGRC). A prelabel lower bound rules out insufficient budgets. With all labels known, a covering linear program bounds the minimum number of labels sufficient to fix the winner (the certificate size) within K−1 labels for K candidates. For fixed K, independent uniform orders and identical predictions, the prelabel bound approaches one quarter of the pool. With iid Bernoulli errors independent of the orders, every exact acquisition policy reads almost all labels asymptotically, although a two-candidate certificate needs only half. Across 108 feature-panel comparisons on nine datasets, disagreement labels settle every accuracy choice but no AUGRC choice. A 20% budget is ruled out in 96 conditions; certificates need 56-57% on average. On ten conditions with pretrained image classifiers, confidence-score choice reads 68-91% of 10,000 labels for exact selection and 50-67% with AUGRC tolerance 5×10−4. An exact stopping test works with any acquisition order. Together, these results link confidence ranks to label budgets and certified model comparison.