From Human Corrections to Learned Segmentations: A Critical Mathematical Review of Information, Learning, and Human Effort

Jon Sporring

cs.LGOct 14, 2026

From Human Corrections to Learned Segmentations: A Critical Mathematical Review of Information, Learning, and Human Effort

Authors: Jon Sporring

Organizations: Department of Computer Science, University of Copenhagen, DK-2100 Copenhagen, Denmark

Abstract

When does a human correction become useful learning, and what establishes better segmentation at matched human effort? This critical narrative review follows feedback through its selection, learner update, and evaluation. It compares active learning, machine teaching, interactive segmentation, and sequential predictive evaluation using a common mathematical account of target information, predictive error, response predictability, and human work. An exact Bayesian update converts evaluation-label information into expected log-loss improvement; an excess-risk decomposition explains what an approximate learner can lose. Conditional information decompositions separate the roles of selected locations, labels, and displayed context. Empirical comparisons show why response protocols, update mechanisms, and effort denominators matter, distinguishing current-mask refinement from reusable learning and simulated interaction from human evidence. We identify what logs can reveal, what requires independent evaluation, and what remains model-dependent. The resulting research priorities are to test what selection reveals, whether using it improves assimilation, and whether the resulting interaction policy benefits segmentation at matched effort. These are distinct claims: predicting a teacher's responses need not identify human intentions or establish better learning. No new acquisition algorithm or empirical superiority is claimed.

Figures & tables

Symbol	Meaning and principal relation
Images, targets, and predictions
$J,\mathcal{X},\mathcal{Y}$	Fixed image collection, address domain, and label alphabet; $J:\mathcal{X}\to\mathbb{R}^{d}$ .
$X\sim\nu$	Evaluation address and weights; $\nu$ is not the view-selection distribution.
$F\sim\mu$ , $f$	Random interpretation across sessions and its fixed session realization.
$P_{F}$ , $P_{f}$	Random target label distribution and its realization; $P_{f}:\mathcal{X}\to\mathcal{P}(\mathcal{Y})$ .
$T_{F}$ , $T_{f}$	Random and realized evaluation-equivalence classes; members of $T_{f}$ agree with $P_{f}$ $\nu$ -almost everywhere.

Table 1: Core symbols, meanings, and relations.

Study	Selection, feedback, and update	Finding, uncertainty, and scope
Crayons [ 20 ]	Human-painted examples and corrections train a reusable pixel classifier; the comparison changes the fitting algorithm, not a query policy.	Ten users, counterbalanced order: skin-classification time to 97.5% reference agreement averaged 3:06/3:51 minutes for the stronger-generalizing algorithm versus 10:35/10:26 for the faster-fitting alternative in the two order groups (Tables 1–2). No intervals or tests are reported. Slower individual updates can therefore accompany faster interactive completion.
RootPainter [ 59 ]	Humans select foreground/background marks after viewing predictions; sparse corrections accumulate in reusable-model training. Initial clear examples precede corrective annotation.	Two author-annotators, three datasets: time-matched corrective versus dense annotation gave held-out measurement $R^{2}$ of 0.80 (SE 0.04) versus 0.75 (SE 0.07), with no significant strategy effect at the 0.05 threshold (Table 3). Corrective annotation came first. The comparison establishes neither superiority nor equivalence.
RootPainter3D [ 60 ]	A physician selects spatial corrections; a model is trained throughout annotation of successive CT volumes. Both completed contours and the reusable predictor change.	One physician, 933 hearts: the last ten averaged 122 seconds with assistance versus 421 manually (Fig. 5); no timing interval is reported. In 20 paired contour comparisons, corrected-output Dice was 0.952 (SD 0.007) against manual contours (Table 1). Timings include inspection but exclude inactivity of at least 20 seconds; the design does not isolate selection modelling.
BIFSeg [ 65 ]	Human bounding boxes and scribbles guide segmentation and image-specific network adaptation. Parameters change, but the intervention is targeted to the current image.	Two clinical users. With identical initial predictions and one-round scribbles, tumour-core Dice was 87.49% versus 85.93% for CRF, and whole-tumour Dice 88.11% versus 85.18% ( $p<0.05$ , paired tests; Table VI). This is evidence of better use of the same corrections for current-case refinement, not accumulated learning across cases.
Cellpose 2.0 [ 48 ]	A human corrects and completes instance masks; accumulated masks retrain a reusable model before the next image. The design compares online/offline annotation and pretrained/from-scratch initialization.	One annotator, two categories: online/pretrained annotation used 167 versus 663 and 158 versus 897 manually drawn objects compared with offline annotation (Fig. 5). Test performance approached within-annotator agreement with 100–200 objects; cost-ratio intervals are not reported. Pretraining and repeated fitting contribute jointly; object counts are not session time.
MONAI Label [ 17 ]	The framework supports system query selection and human clicks/scribbles, with mask refinement and reusable-model retraining. The reported workflow is not one fixed acquisition method.	DeepGrow validation Dice rose from 0.891 to 0.967 (2D) and 0.730 to 0.959 (3D) across annotation stages (Table 2), without uncertainty intervals. These are workflow demonstrations, not controlled acquisition-policy comparisons. Inconsistent validation counts and timing totals in the source prevent a precise cost-effect estimate.

Table 2: Study-level evidence: selection, assimilation, and human effort. Prompt refinement, current-image adaptation, and reusable-model training are distinct update scopes. “Time” and counts are not interchangeable effort measures.

Approach	Decision and observation	Issue in the corrective loop
BALD [ 30 ]	Learner requests a label; beliefs determine its expected informativeness.	A teacher may choose a different support within the displayed view.
BatchBALD [ 36 ]	Learner requests a batch; joint label information accounts for redundancy.	A jointly informative batch is not necessarily the subset the human will label.
Variance reduction [ 13 ]	Query scored through its expected effect on a fitted predictor.	Improvement depends on the loss, model assumptions, and actual update.
Annotator-aware AL [ 70 ]	Learner selects an instance and an annotator using reliability estimates.	Reliability and the human’s choice of what to correct are distinct mechanisms.
Iterative teaching and teacher-aware learning [ 40 , 71 ]	Teacher selects supervision; learner may interpret its selection mechanism.	Spatial support, displayed prediction, and human effort require an explicit specialization.
Interactive segmentation [ 12 , 59 ]	Human or simulated teacher supplies spatial feedback to a mask or reusable model.	Per-image refinement and learning for subsequent views require different evaluation.

Table 3: Representative approaches and the assumptions needed to transfer their interpretation to corrective learning.

Setting or claim	Representation or additional requirement
Fixed-pool additive log loss	Directly represented by $P_{f}$ , $\nu$ , and the learner’s predictions.
Joint-mask uncertainty; topology or expected Dice/IoU	Pixel marginals do not determine the joint mask distribution; use a joint target and a suitable loss.
Generalization to unseen images	Specify an image population and an independent evaluation protocol; random views of the fixed pool are insufficient.
Continuous target families	Mutual information can be defined through joint/product KL; finite discrete entropy differences require additional restrictions.
Changing human interpretation	Introduce target dynamics and the evaluation reference; asynchronous training alone does not make the target change.

Table 4: Scope of the formal core and limits of transfer.

Panel of Figure 1 — Figure 1: A four-pixel interaction with a uniform source over 16 deterministic targets and one faithful correction.

Panel of Figure 2 — Figure 2: Evaluation weights determine which targets are equivalent and how much the same correction reduces distortion.

Panel of Figure 3 — Figure 3: Expected gain and distortion after one more correction depend on the teacher’s response policy, even with faithful labels.

Panel of Figure 4 — Figure 4: Equal information gain can give different distortion reductions when evaluation weights differ.

Panel of Figure 5 — Figure 5: More target information need not produce a larger distortion reduction when learner beliefs differ from the source posterior.

Panel of Figure 6 — Figure 6: The four-pixel BALD comparison changes both the selection rule and the response opportunities; it is not a matched-policy experiment.

Panel of Figure 7 — Figure 7: A normalized response model makes same-stream scoring operational without estimating the teacher’s target or information gain.

Panel of Figure 8 — Figure 8: Under a known selection convention, the chosen address identifies the target even though both targets agree on the supplied labels.

Quantity	Interpretation	Related approach and qualification
Expected gain	Target information acquired	BALD/BatchBALD under matching beliefs, observation models, and target relevance [ 30 , 36 ] .
Expected distortion	Predictive error	Variance reduction for squared error under negligible bias, not general KL [ 13 ] .
Gain/work	Information per declared cost unit	Cost-aware acquisition; CEREALS uses informativeness and predicted annotation cost, not exact target gain [ 44 ] .
Distortion reduction/work	Predictive improvement per declared cost unit	Related to ROI acquisition; Haertel et al. consider accuracy benefit and annotation cost [ 28 ] .
Prequential length	Predictability of a response stream	Applies sequential predictive evaluation to support and labels [ 14 , 9 ] .
Length/work	Code bits per declared cost unit	A response-model diagnostic, not a measure of target gain.

Table 5: Quantities and their relation to established approaches.

Quantity	Available evidence	What does not follow
View, checkpoint, support, labels, times	Direct logs, if the interface records them.	Display is not inspection; non-correction is not acceptance.
$W_{n}$ and measured $C_{n}$	Support counts or a declared action/timing protocol.	Equal marks need not mean equal human effort.
Response log loss	Score causal forecasts on the same recorded stream.	A better response predictor need not identify gain or improve learning.
Predictive risk difference	Representative, independently assessed labels for declared checkpoints.	Training corrections alone need not represent the evaluation distribution.
Source posterior, exact target gain, entropy floor	A specified source and observation model or additional identifying evidence.	A single correction log generally does not identify these quantities.
Responses to undisplayed views	Interventions, or justified overlap and response-model assumptions.	Replaying recorded corrections does not reveal counterfactual human choices.

Table 6: Evidence required to operationalize the framework.

Human-in-the-Loop AI

Human-in-the-loop AI combines human feedback, oversight, and decision-making with machine learning and autonomous systems.

See all 289 papers

Feb 17, 2026

Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries

Integrating human expertise into machine learning systems often reduces the role of experts to labeling oracles, a paradigm that limits the amount of information exchanged and fails to capture the nuances of human judgment. We address this challenge by developing a human-in-the-loop framework to learn binary classifiers with rich query types, consisting of item ranking and exemplar selection. We first introduce probabilistic human response models for these rich queries motivated by the relationship experimentally observed between the perceived implicit score of an item and its distance to the unknown classifier. Using these models, we then design active learning algorithms that leverage the rich queries to increase the information gained per interaction. We provide theoretical bounds on sample complexity and develop a tractable and computationally efficient variational approximation. Through experiments with simulated annotators derived from crowdsourced word-sentiment and image-aesthetic datasets, we demonstrate significant reductions on sample complexity. We further extend active learning strategies to select queries that maximize information rate, explicitly balancing informational value against annotation cost. This algorithm in the word sentiment classification task reduces learning time by more than 57% compared to traditional label-only active learning.

Belén Martín-Urcelay, Yoonsang Lee, Matthieu R. Bloch +1

School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA.

Human-in-the-Loop Annotation Active Learning

Ask PDF

Feb 20, 2026

Enhancing Goal Inference via Correction Timing

Corrections offer a natural modality for people to provide feedback to a robot, by (i) intervening in the robot's behavior when they believe the robot is failing (or will fail) the task objectives and (ii) modifying the robot's behavior to successfully fulfill the task. Each correction offers information on what the robot should and should not do, where the corrected behavior is more aligned with task objectives than the original behavior. Most prior work on learning from corrections involves interpreting a correction as a new demonstration (consisting of the modified robot behavior), or a preference (for the modified trajectory compared to the robot's original behavior). However, this overlooks one essential element of the correction feedback, which is the human's decision to intervene in the robot's behavior in the first place. This decision can be influenced by multiple factors including the robot's task progress, alignment with human expectations, dynamics, motion legibility, and optimality. In this work, we investigate whether the timing of this decision can offer a useful signal for inferring these task-relevant influences. In particular, we investigate three potential applications for this learning signal: (1) identifying features of a robot's motion that may prompt people to correct it, (2) quickly inferring the final goal of a human's correction based on the timing and initial direction of their correction motion, and (3) learning more precise constraints for task objectives. Our results indicate that correction timing results in improved learning for the first two of these applications. Overall, our work provides new insights on the value of correction timing as a signal for robot learning.

Anjiabei Wang, Shuangge Wang, Tesca Fitzgerald

Yale University, New Haven, CT, United States

Robot Learning Human-Robot Collaboration RL from Human Feedback

Ask PDF

Oct 15, 2026

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards

Reward learning typically assumes the human's input covers everything that matters for the task. In practice, humans can only attend to so much at once, so their teaching naturally captures some features---or aspects of the task---better than others. Prior work has shown how the robot can identify which features were poorly supervised directly from its teaching data. But the current solution is to ask the human to try again, putting the burden on someone who already struggled to convey those features. We propose shifting this burden to the robot. Our key insight is that the same information that tells the robot what it doesn't know can help it ask better questions. Preference queries offer a low-effort channel for resolving this uncertainty, but only if the robot knows where to direct them and the human knows what to attend to when answering. We act on this insight in two ways: we build an informed prior that targets preference queries at the features that need resolving, and we accompany each query with an explanation that focuses the human's attention on exactly those features. In simulation, the informed prior beats the standard active preference learning baseline by up to

30.8\%

in the low-query regime, where reducing human effort matters most. In a user study, the informed prior and explanations each significantly improve reward recovery and amplify each other.

Helena Merker, Nick Walker, Andreea Bobu

Massachusetts Institute of Technology

Active Preference Learning Preference Learning Robot Learning Preference Elicitation Reward Design for RL

Ask PDF

Abstract

Human-in-the-Loop AI

Explore similar work

Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries

Enhancing Goal Inference via Correction Timing

Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards