Accuracy Is Not Value: Choosing Weak Supervisors by What the Student Inherits
Organizations: Brown University
Abstract
Weak supervisors are usually judged by their own errors, but what matters is what a student learns from them. For squared loss and any linear learner shared by all supervisors, we show that a supervisor's value is determined by its errors after they pass through the student's learning operator. Consequently, the inner products of the supervisors' raw errors can be compatible with every feasible value of the student's risk, and weighting supervisors by them can be arbitrarily worse than the best weighting in relative excess risk. This student-relative value is not identifiable from unlabeled data, but what is missing is small: one finite-dimensional projection of the target decides whether training on a weak aggregate helps, another decides how to weight the supervisors. Known teacher training rules, or a label-model condition stated in student-relative form, shrink it further. Clean labels close the remaining gap at a cost set by the geometry of the unresolved directions rather than by the dimension of the student's representation, with matching worst-case lower bounds. In Gaussian designs we also quantify when exact teacher structure beats ordinary stacking and when weak supervisors, combined with clean labels, beat every procedure that uses the clean labels alone. The guarantees extend to fitted ridge students and finite unlabeled data.
Figures & tables
| Decision | Geometry | Anchor cost |
|---|---|---|
| Gain signs, one or all ( Theorem 6.1 ) | margin ; width of queries | ; |
| Affine selection ( Theorem 6.2 ) | unresolved trace | |
| Local selection ( Theorem 6.2 ) | tangent cone at the oracle | |
| Passive anchors ( Section 6.4 ) | cell coverage |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Work | Target object | Observables | Clean labels | Learner | Ident. | What this paper adds |
|---|---|---|---|---|---|---|
| Misfit–gain ( Charikar et al., 2024 ; Mulgund and Pabbaraju, 2025 ) | gain of one supervisor; choice among weak models by misfit | student and supervisor predictions | none | convex class / projection | no | joint aggregation; identification boundaries; label cost |
| Representations ( Xue et al., 2025 ) | prediction gap of weakly vs. cleanly trained students | representations | none | ridge head | no | absolute risk and decisions, which the gap does not determine; non-identification |
| Label models ( Dawid and Skene, 1979 ; Ratner et al., 2016 ) | supervisor accuracies | agreement patterns | none | none or any | yes (under independence) | what independence must control; post-learning value |
| Ensemble W2S ( Sang et al., 2024 ; Agrawal et al., 2024 ; Liu and Alahi, 2024 ) | a combined pseudo-label | weak predictions | none or few | fine-tuned model | no | the objective to optimize and when it is observable |
| Stacking, aggregation ( Wolpert, 1992 ; Breiman, 1996 ; Tsybakov, 2003 ) | best combination of fixed predictors | labeled data | all | none | no | which moments are needed; instead of ; unlabeled Gram |
| Forecast combination ( Bates and Granger, 1969 ) | combined forecast error | raw error covariance | historical outcomes | none | no | the transmitted, not the raw, error is optimized |
Weakly Supervised Learning
Weakly supervised learning trains models from incomplete, noisy, coarse, or indirect labels instead of dense ground truth.
See all 163 papers