Learning under noisy supervision is governed by a feedback-truth gap

Elan Schonfeld; Elias Wisnia

cs.LGFeb 18, 2026

Learning under noisy supervision is governed by a feedback-truth gap

Authors: Elan Schonfeld, Elias Wisnia

Organizations: Department of Biology, Columbia University, New York, NY, USA · Department of Chemistry, Columbia University, New York, NY, USA

Abstract

When feedback is absorbed faster than task structure can be evaluated, the learner will favor feedback over truth. A two-timescale model shows this feedback-truth gap is inevitable whenever the two rates differ and vanishes only when they match. We test this prediction across neural networks trained with noisy labels (30 datasets, 2,700 runs), human probabilistic reversal learning (N = 292), and human reward/punishment learning with concurrent EEG (N = 25). In each system, truth is defined operationally: held-out labels, the objectively correct option, or the participant's pre-feedback expectation - the only non-circular reference decodable from post-feedback EEG. The gap appeared universally but was regulated differently: dense networks accumulated it as memorization; sparse-residual scaffolding suppressed it; humans generated transient over-commitment that was actively recovered. Neural over-commitment (~0.04-0.10) was amplified tenfold into behavioral commitment (d = 3.3-3.9). The gap is a fundamental constraint on learning under noisy supervision; its consequences depend on the regulation each system employs.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Samples	Features	Classes	OpenML ID
Ionosphere	351	34	2	59
Glass	214	9	6	41
Sonar	208	60	2	40
WDBC	569	30	2	1510
Vehicle	846	18	4	54

Appendix

Table 1: Primary benchmark datasets.

Human probabilistic reversal learning (Eckstein 2022)
Analysis	$N$	Measure	Statistic	$p$ -value	Effect size
PRL AUG ${}_{\text{pos}}$	292	$0.049\pm 0.032$	$t(291)=26.4$	$1.1\times 10^{-79}$	$d=1.55$
PRL $T^{*}$ recovery	292	$9.9\pm 13.9$ trials	99.3%	–	–
Human reward/punishment learning (Stolz 2022)
REW commitment	25	$0.567\pm 0.171$	$t(24)=16.6$	$5.9\times 10^{-15}$	$d=3.32$
PUN commitment	25	$0.628\pm 0.160$	$t(24)=19.6$	$1.4\times 10^{-16}$	$d=3.92$

Appendix

Table 2: Summary of all primary statistical results across experiments.

Figure 1: Supplementary Figure 1

|

PRL window sensitivity analysis. Robustness of the feedback–truth gap to rolling window size in the probabilistic reversal learning dataset (

N=292

). Gap magnitude (AUG

{}_{\text{pos}}

) and temporal structure remain qualitatively consistent across window sizes from 4 to 16 trials, confirming that the observed dynamics are not artifacts of the chosen smoothing parameter. Main text uses 8-trial window (highlighted).

Learning with Noisy Labels

Learning with noisy labels studies how to train reliable models when annotations are incorrect, incomplete, subjective, or weakly supervised.

See all 150 papers

Human Supervision as an Information Bottleneck: A Unified Theory of Error Floors in Human-Guided Learning
Feb 26, 2026Alejandro Rodriguez DominguezInformation Bottleneck Learning with Noisy Labels
Miralta Finance Bank S.A., Spain · Department of Computer Science, University of Reading, UK
A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits
Sep 8, 2026Xiangnan Zhang, Jingxin Liu, Ranqi Lu +6Neural Processes Temporal Credit Assignment
School of Future Technologies, Beijing Institute of Technology, Beijing, 100081, China. · Key Laboratory of Brain Health Intelligent Evaluation and Intervention (Beijing Institute of Technology), Ministry of Education, Beijing, 100081, China. · School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China. +5
Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning
Aug 4, 2026Wenxiao Fan, Kan LiInstance Reweighting Noisy-Label Learning
School of Computer Science, Beijing Institute of Technology

Feb 26, 2026J/K move · Enter open · S save

Human Supervision as an Information Bottleneck: A Unified Theory of Error Floors in Human-Guided Learning

Alejandro Rodriguez Dominguez

Miralta Finance Bank S.A., Spain · Department of Computer Science, University of Reading, UK

Information Bottleneck Learning with Noisy Labels RL from Human Feedback

Open paper AskPDF arXiv

Large language models are trained primarily on human-generated data and feedback, yet they exhibit persistent errors arising from annotation noise, subjective preferences, and the limited expressive bandwidth of natural language. We argue that these limitations reflect structural properties of the supervision channel rather than model scale or optimization. We develop a unified theory showing that whenever the human supervision channel is not sufficient for a latent evaluation target, it acts as an information-reducing channel that induces a strictly positive excess-risk floor for any learner dominated by it. We formalize this Human-Bounded Intelligence limit and show that across six complementary frameworks (operator theory, PAC-Bayes, information theory, causal inference, category theory, and game-theoretic analyses of reinforcement learning from human feedback), non-sufficiency yields strictly positive lower bounds arising from the same structural decomposition into annotation noise, preference distortion, and semantic compression. The theory explains why scaling alone cannot eliminate persistent human-aligned errors and characterizes conditions under which auxiliary non-human signals (e.g., retrieval, program execution, tools) increase effective supervision capacity and collapse the floor by restoring information about the latent target. Experiments on real preference data, synthetic known-target tasks, and externally verifiable benchmarks confirm the predicted structural signatures: human-only supervision exhibits a persistent floor, while sufficiently informative auxiliary channels strictly reduce or eliminate excess error.

Abstract

Learning with Noisy Labels

Explore similar work

Human Supervision as an Information Bottleneck: A Unified Theory of Error Floors in Human-Guided Learning

A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits

Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning