SteelHuman: Predicting Operator Estimates of Steel-Scrap Composition from Sparse RGB Views
Organizations: Faculty of Electrical Engineering and Information Technologies Ss. Cyril and Methodius University Skopje, North Macedonia
Abstract
Steel scrap composition estimation is an important part of yield planning and fair supplier compensation. Measuring the composition of a multi-ton pile is an inherently laborious process, so the assessment is often left to trained human operators. Recent works tackle composition-level tasks, but require structured acquisition and multi-stage preprocessing. We study a simpler approach to steel scrap load assessment by aggregating features from sparse, handheld views of the pile. On 600 out-of-fold piles, frozen DINOv2 tokens aggregated uniformly across views reach 13.4% total variation (TV) distance against operator estimates. Most remaining error lies in allocation among four thickness-defined heavy melting scrap (HMS) classes, yet image-dependent allocation reduces within-HMS TV from 25.1% under a training-set split prior to 11.2%. Due to the lack of public datasets and benchmarks for predicting composition, external evaluation relies on the public DOES dataset intended for classification. A classifier using the same frozen features reaches 93.97% accuracy, exceeding the published results we compare against. The composition model also aligns with related DOES categories without adaptation.
Figures & tables
| Class | Active | Mean share (%) | Homogeneous |
|---|---|---|---|
| Stamped | 99 | 13.0 | 96 |
| HMS mm | 596 | 49.6 | 118 |
| HMS 4–6 mm | 506 | 15.1 | 0 |
| HMS 2–4 mm | 465 | 16.6 | 0 |
| HMS 0–2 mm | 251 | 3.3 | 0 |
| Cast iron | 365 | 2.4 | 9 |
| All piles ( ) | Homogeneous ( ) | Heterogeneous ( ) | ||||
|---|---|---|---|---|---|---|
| Method | Mean | Median | Mean | Median | Mean | Median |
| TV-optimal prior | 40.44 | 36.00 | 67.76 | 39.00 | 28.73 | 26.00 |
| Mean pooling | ||||||
| Self-attention | ||||||
| Gated ABMIL | ||||||
| Mean pooling, no projector | ||||||
| Representation | All | Hom. | Hetero. |
|---|---|---|---|
| DINOv2 ViT-S [ 19 ] | 14.72 | 10.56 | 16.39 |
| DINOv2 ViT-L [ 19 ] | 12.83 | 5.15 | 15.92 |
| DINOv2 ViT-g [ 19 ] | 12.84 | 5.67 | 15.72 |
| DINOv3 ViT-L [ 26 ] | 13.30 | 6.04 | 16.22 |
| ConvNeXt-V2-L [ 36 ] | 15.61 | 9.31 | 18.14 |
| CLIP ViT-L [ 23 ] | 15.87 | 11.62 | 17.58 |
| Metric | Model | HMS prior |
|---|---|---|
| Within-HMS TV (%) | 11.19 | 25.12 |
| Conditional HMS TV (%) | 14.13 | 30.74 |
| Conditional (class steps) | 0.227 | 0.534 |
| DOES-trained method | Accuracy (%) |
|---|---|
| Original DOES, PreActResNet18 [ 24 ] | 68.09 |
| CLRiuS, 10 labeled tiles/class [ 25 ] | 75.89 |
| LASER, pretrained CLIP [ 15 ] | 92.80 |
| Esfandiari et al. [ 8 ] , ViT-L teacher | 92.37 |
| Esfandiari et al. [ 8 ] , distilled DeiT | 88.81 |
| Ours: frozen DINOv2-L, linear head | 91.53 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Analysis / inclusion rule | Piles |
|---|---|
| Primary evaluation | 600 |
| Homogeneous | 180 |
| Heterogeneous | 420 |
| Conditional HMS: target HMS | 515 |
| Four-HMS RMSE: heterogeneous, HMS | 405 |
| View subsets: heterogeneous, | 356 |
| Method | All piles | Heterogeneous |
|---|---|---|
| Self-attention | ||
| Gated ABMIL | ||
| No projector |
| Features | All | Hom. | Hetero. |
|---|---|---|---|
| Final layer | 12.83 | 5.15 | 15.92 |
| DINOv2 ViT-L layers | |||
| Layer 4 | 23.47 | 30.17 | 20.78 |
| Layer 11 | 15.04 | 10.86 | 16.71 |
| Layer 17 | 14.23 | 10.07 | 15.90 |
| DINOv2-L layer fusion | |||