Auditing Quality Filters for Long-Tail Human Data Curation
Organizations: Gritt Robotics
Abstract
Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF, and groups similar poses. Low poses account for only about 2 percent of the retained examples. This low share may partly reflect the pipeline's quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene's vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation.
Figures & tables
| Capture source | Inst. | Low | Share [95% CI] |
|---|---|---|---|
| Gritt dataset | 3,258 | 0 31 | 0.95% [0.54, 0 1.56] |
| External construction dataset | 3, 504 | 0 53 | 10.52% [7.24, 14.20] |
| Full retained library | 3,762 | 0 84 | 2.23% [1.51, 0 3.23] |
| Check | Estimated joints used | Ratio |
|---|---|---|
| Original pose labels | pose vector and alignment | (0.32) |
| Box shape | none | 0.26 |
| Vertical body span | span only | 0.43–0.47 |
| Scene-based alignment | pose vector only | 0.66 |
| Image appearance | none | 0.50 |
| Comparison | Change (pp) | 95% interval |
|---|---|---|
| Original to new upright models | +17.1 | [+13.6, +20.6] |
| Standing to mined poses | -17.4 | [-20.3, -14.4] |
| Restricted subset: original scale | -29.2 | [-38.1, -20.8] |
| Restricted subset: standing shrunk | -17.7 | [-25.6, -10.2] |
| Restricted subset: posed enlarged | -24.6 | [-33.3, -16.4] |
Training Data Curation
Training data curation improves the quality, diversity, organization, and reliability of datasets used to train machine-learning models.
See all 156 papers