When Lower Harm Misleads: Auditing Concept Corrections
Organizations: Carnegie Mellon University · The Hong Kong Polytechnic University · Mohamed bin Zayed University of Artificial Intelligence · University of Chinese Academy of Sciences ·
Abstract
An expert who corrects a concept prediction, such as a BI-RADS descriptor in breast ultrasound, expects the diagnosis to improve. Objectives that reduce the harm of such corrections are judged by harm and accuracy, yet harm also falls when a model responds less to every correction or starts from a worse uncorrected prediction, the reference for harm. We propose a correction audit, a reproducible evaluation protocol that separates these explanations: it splits each correction into harm and useful gain on identical events, separates the starting prediction from the correction response, and compares objectives with amplitude baselines matched in source harm. On three breast-ultrasound sources, worst-case correction training, evaluated from the mean-case model's start, lowers harm but gives up more useful gain in all 21 source comparisons pooled over six seeds, raising the corrected negative log-likelihood (NLL); from the same start, selecting between the objectives by harm picks the less helpful model on all three sources. On CUB, a monotonic harm penalty at its reported weight lowers path-level harm in all 80 concept sets while raising the NLL in all 80, almost entirely before any correction. At a fixed start, rescaling both responses to the same source harm reproduces the penalty's mean harm, useful gain and NLL on single-concept corrections within a prespecified margin of 0.0005 nat, although the two responses point in different directions. Lower harm therefore signals better corrections only when it is read together with useful gain, absolute risk and the start it is measured from.
Figures & tables
| corrected NLL ( nat) by evaluation source | |||
|---|---|---|---|
| Training | Private250 | BUSI647 | BUS-BRA |
| P | |||
| I | |||
| B | |||
| NLL | Rescued | |||
|---|---|---|---|---|
| Private250, linear | ||||
| Private250, MLP | ||||
| BUS-BRA, linear | ||||
| BUS-BRA, MLP | ||||
| BUSI647, linear | ||||
| BUSI647, MLP |
| M E | Reported in [ 25 ] | Ours |
|---|---|---|
| Harmful-path prevalence (pp) | ||
| Max. sampled partial harm | ||
| Accuracy before correction (pp) | ||
| Harm and accuracy favor | The audit step that changes the reading |
|---|---|
| Clinical, at Mean’s start: Worst, lower harm in 21 of 21 | Gain : more gain lost, so corrected NLL is higher in 21 of 21 |
| Clinical: the model that harm selects on each source | Gain : higher corrected NLL in every seed, fewer rescued predictions |
| CUB: M, with lower path harm in 80 of 80 and accuracy barely changed ( pp) | Split : NLL higher in 80 of 80, almost all before correction; amplitude : scaled E has M’s harm at lower NLL |
| CUB, fixed start: the penalty, with 8.5% of the harm | Amplitude : matched scaling reproduces the mean , and NLL |
| CUB: IntCEM, with 12.6 points fewer harmful paths | Split : partially corrected NLL nat, almost all starting risk |
Appendix figures & tables37 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Family | Points | Unres. | , unpenalized | , penalized |
|---|---|---|---|---|---|
| Frozen, endpoint | 300k | 0 | ( – ) | ( – ) | |
| Frozen, endpoint | 300k | 0 | ( – ) | ( – ) | |
| Frozen, partial-state | 300k | 10 | ( – ) | ( – ) | |
| Frozen, partial-state | 300k | 0 | ( – ) | ( – ) | |
| BUS-BRA, both rules | , | 480k | 0 | – | – |
| Joint, temperature | 800k | 0 | ( – ) | ( – ) |
| BUS-BRA (732 cases) | BUSI647 (452 images) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Annotation vs. diagnosis | Concept forecast | Annotation vs. diagnosis | Concept forecast | |||||||
| Field | AUROC | MI share | Acc. | Majority | AUROC † | AUROC | MI share | Acc. | Majority | AUROC † |
| Shape | ||||||||||
| Margin (pattern) | ||||||||||
| Stratum | Share/acc. | NLL | |
|---|---|---|---|
| BUS-BRA | |||
| low | / | [ , ] | [ , ] |
| middle | / | [ , ] | [ , ] |
| high | / | [ , ] | [ , ] |
| BUSI647 | |||
| low | / | [ , ] | [ , ] |
| Train evaluate | partial NLL | ||
|---|---|---|---|
| Train evaluate | Coverage | partial NLL | ||
|---|---|---|---|---|
| Train evaluate | 2026 | 2027 | 2028 | 2029 | 2030 | 2031 | Seeds and units |
|---|---|---|---|---|---|---|---|
| [ , ] | |||||||
| [ , ] |
| Train evaluate | : Mean / Worst | : Mean / Worst | AUROC: Mean / Worst |
|---|---|---|---|
| Shared CorrectiveMean start (primary) | ||||||
| BUS-BRA, linear | ||||||
| incomplete (999) | ||||||
| BUS-BRA, MLP | ||||||
| BUSI647, linear | ||||||
| Shared CEM start (primary) | ||||||
| NLL | ||||||
| incomplete (998) | incomplete (989) | |||||
| incomplete (998) | incomplete (989) | |||||
| Shared CEM start (primary) | ||||||
|---|---|---|---|---|---|---|
| NLL | ||||||
| Source | Readout | Checkpoint | NLL | Rescued (pp) | Broken (pp) | Worst picked | ||
|---|---|---|---|---|---|---|---|---|
| BUS-BRA | linear | selected | 22 / 4 | |||||
| BUS-BRA | MLP | selected | 20 / 4 | |||||
| BUSI647 | linear | selected | 16 / 2 | |||||
| BUS-BRA | BUSI647 | |||
|---|---|---|---|---|
| Start, query order | linear | MLP | linear | MLP |
| Own start, own uncertainty (planned primary) | ||||
| of which before correction | ||||
| Shared start, own uncertainty | ||||
| Unscaled NLL | Matched source harm, | ||||
|---|---|---|---|---|---|
| Source | Readout | selected | last | NLL | |
| BUS-BRA | linear | Tab. 1 | |||
| MLP | |||||
| BUSI647 | linear | Tab. 1 | |||
| Readout | NLL | Accuracy | Rescued | Broken | ||
|---|---|---|---|---|---|---|
| linear | ||||||
| MLP | ||||||
| Readout | NLL | Rescued | |||
|---|---|---|---|---|---|
| linear | 1 | ||||
| linear | 2 | ||||
| linear | 3 | ||||
| Readout | Start and query order | NLL | NLL before | ||
|---|---|---|---|---|---|
| linear | Shared start, shared order | ||||
| linear | Shared start, own uncertainty | ||||
| linear | Shared start, random | ||||
| Readout | Start and query order | Harm pick before-NLL pick | Before-NLL pick NLL pick |
|---|---|---|---|
| linear | Shared start, shared order | ||
| linear | Shared start, own uncertainty | ||
| linear | Shared start, random | ||
| Readout | Start and query order | NLL | Rescued | Worst picked | ||
|---|---|---|---|---|---|---|
| linear | Shared start, shared order | 23/0 | ||||
| linear | Shared start, own uncertainty | 24/9 | ||||
| linear | Shared start, random | 24/6 | ||||
| Readout | NLL | Rescued | ||
|---|---|---|---|---|
| linear | ||||
| MLP | ||||
| Outcome | Linear: harm pick | Linear: NLL pick | MLP: harm pick | MLP: NLL pick |
|---|---|---|---|---|
| NLL before | ||||
| NLL after | ||||
| Raw | Temperature-controlled | |||
| M E | M E | E β E | M E β | |
| Max. sampled partial harm | [ , ] | [ , ] | [ , ] | [ , ] |
| Harmful-path prevalence (%) | [ , ] | [ , ] | [ , ] | [ , ] |
| Per-event partial harm | – | [ , ] | [ , ] | [ , ] |
| Partial harm (%) | – | [ , ] | [ , ] | [ , ] |
| Pre-intervention NLL | [ , ] | [ , ] | [ , ] | |
| M E | Reported in [ 25 ] | Ours, raw | Ours, temperature-controlled |
|---|---|---|---|
| Harmful-path prevalence (pp) | [ , ] | [ , ] | [ , ] |
| Max. sampled partial harm ( nat) | [ , ] | [ , ] | [ , ] |
| Task accuracy before correction (pp) | [ , ] | ||
| Concept mean squared error | [ , ] | ||
| Pre-intervention NLL ( nat) | n.r. | [ , ] | [ , ] |
| Partially corrected NLL ( nat) | n.r. | [ , ] | [ , ] |
| Monotonic penalty minus endpoint ( ) | Minus L2 control | |||||
| NLL | NLL | |||||
| Max. partial harm | Prevalence (pp) | Pre-intervention | Partially corrected | Fully corrected | Partially corrected | |
| 1 | ||||||
| 3 | ||||||
| Fit split | Test split | |||
| Range | Range | |||
| Observational | 0.00 | 0.00–0.02 | 0.17 | 0.06–0.32 |
| Endpoint (E) | 0.40 | 0.08–0.85 | 0.51 | 0.37–0.72 |
| Endpoint mono. (M) | 0.83 | 0.65–0.98 | 0.70 | 0.49–0.89 |
| M E | 80/80 | 79/80 | ||
| Penalized unpenalized ( nat) | |||
|---|---|---|---|
| Operating point | NLL | ||
| (unmatched) | |||
| Point | NLL | Label | ||
| Endpoint background, | ||||
| [ , ] | [ , ] | [ , ] | – | |
| [ , ] | [ , ] | [ , ] | E | |
| [ , ] | [ , ] | [ , ] | E | |
| [ , ] | [ , ] | [ , ] | E, W P | |
| [ , ] | [ , ] | [ , ] | E, W P | |
| Combination | NLL | ||
|---|---|---|---|
| Endpoint, (primary) | , , | , , | , , |
| Endpoint, | , , | , , | , , |
| Partial-state, | , , | , , | , , |
| Partial-state, | , , | , , | , , |
| Point | Arm | NLL | |||
|---|---|---|---|---|---|
| Unpenalized | |||||
| Penalized | |||||
| Unpenalized | |||||
| Penalized | |||||
| Unpenalized | |||||
| Penalized |
| Combination | |||||
|---|---|---|---|---|---|
| Endpoint, | |||||
| Endpoint, | |||||
| Partial-state, | |||||
| Partial-state, |
| Frozen, endpoint | Frozen, partial-state | Jointly trained, temperature | ||||
|---|---|---|---|---|---|---|
| Point | ||||||
| [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | [ , ] | |
| Seed | Harm lower | NLL higher | partial NLL |
|---|---|---|---|
| Original | 10/10 | 10/10 | [ , ] |
| Second | 10/10 | 10/10 | [ , ] |
| Third | 10/10 | 10/10 | [ , ] |
| All | 30/30 | 30/30 | [ , ] |
| All a | [ , ] | ||
| All b | [ , ] |
| Point | NLL | Strata | ||
| [ , ] | [ , ] | [ , ] | 0/0/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| Point | NLL | Strata | ||
| [ , ] | [ , ] | [ , ] | 0/0/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| [ , ] | [ , ] | [ , ] | 10/10/10 | |
| Penalized unpenalized ( nat) | |||
|---|---|---|---|
| Amplitude control | NLL | ||
| Native units | |||
| Global scale ( ) | |||
| Per field or global ( / ) | |||
| Point | NLL | large harm (pp) | ||
|---|---|---|---|---|
| [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | |
| [ , ] | [ , ] | [ , ] | [ , ] | |
| Start | Image set | NLL before | partial NLL | ||
|---|---|---|---|---|---|
| Native | Pooled (130) | [ , ] | [ , ] | [ , ] | [ , ] |
| Private250 (40) | [ , ] | [ , ] | [ , ] | [ , ] | |
| BUSI647 (90) | [ , ] | [ , ] | [ , ] | [ , ] | |
| Fixed | Pooled (130) | [ , ] | [ , ] | [ , ] | |
| Private250 (40) | [ , ] | [ , ] | [ , ] | ||
| BUSI647 (90) | [ , ] | [ , ] | [ , ] |
Human-in-the-Loop Evaluation
Human-in-the-Loop Evaluation studies how human judgment and feedback can assess, guide, and monitor AI systems in realistic settings.
See all 174 papers