When Does Data Value Reduce to Class Balance? A Coverage View of Per-Point Data Valuation

Yingjie Bai; Weiming Zhi

cs.LGOct 26, 2026

When Does Data Value Reduce to Class Balance? A Coverage View of Per-Point Data Valuation

Authors: Yingjie Bai, Weiming Zhi

Organizations: School of Computer Science, The University of Sydney

Abstract

Per-point data-valuation scores -- Data Shapley, influence, leverage -- are routinely used to select training data: keep the points with the top $k$ scores. We argue that top- $k$ by a fixed, mode-blind per-point score is the wrong selection rule precisely in the regime where data selection matters. The reason is that a point's value is not a fixed number but depends on what else has been selected -- a near-duplicate of an already-chosen point adds almost nothing, however high its score -- and a per-point scalar cannot express this interaction. The theory quantifies this failure exactly: in an equal latent-mode model, fixed top- $k$ selection incurs a closed-form excess risk over adaptive coverage because it repeatedly spends budget on already-covered modes. The gap is largest when the budget is tight, the learner representation is redundant, and missing a mode is costly. The practical rule is simple: use balance when the labels available at selection identify the learner's modes, and use explicit coverage when they do not. On standard classification tasks with strong pretrained features, class-balancing the same per-point scores recovers the missing coverage and matches or outperforms generic set coverage across our datasets. Without labels, label-free coverage improves accuracy by up to +0.20 while raw top- $k$ can fall below random; when labels are coarser than the modes, coverage within coarse strata performs best. We turn this distinction into a pre-specified diagnostic and validate it across both favorable and stress-test settings. The limits are part of the claim: the advantage is representation-relative. Coverage fails when the relevant modes are not expressed in the learner geometry, as with spurious subgroups, or when data quality varies independently of coverage -- exactly as the theory predicts.

Figures & tables

Figure 1: Data value is a context function, and a fixed scalar top- $k$ pays an exact coverage price. Left: per-point valuation (Shapley, LOO/influence, leverage, probe-gap) compresses each training point to one number $v_{i}$ , and its top- $k$ concentrates on a few modes of the pool (circled points), under-covering the learner’s modes: the scalar discards the set-level interaction. Middle: a point’s value is the context function $g_{i}(S)=\mathrm{R}(S)-\mathrm{R}(S\cup\{i\})$ — equal to $\rho$ while the point’s mode is uncovered, $0$ once a same-mode duplicate is selected. Top right: the exact result — for any symmetric per-point score, top- $k$ covers only the occupancy $m_{\star}(k)$ ( $r(1-(1-1/r)^{k})$ as $N\to\infty$ ) versus adaptive greedy’s $\min(k,r)$ , giving $\mathbb{E}[\text{excess risk}]=\rho\,\delta_{N}(k)$ (a lower bound for non-coordinated scores, Definition 1 ); empirically, at $k=C=10$ Shapley covers $6.0$ and leverage $6.7$ of $10$ modes versus greedy’s $9.7$ , and the synthetic binding-budget gap is $0.895$ vs $0.637$ ( $+0.258$ ). Bottom right: the gap matters under a binding budget, a redundant learner geometry, and a hard task, and the correction is the modes-vs-labels boundary: balance the same scores when the labels name the learner’s modes; use learner-geometry coverage when labels are missing or coarser than the modes (coarse-label CIFAR-100: coverage within coarse strata $0.419$ vs. coarse-balanced prototypicality $0.240$ fine accuracy); and expect no coverage handle when the representation does not express the modes (Waterbirds).

$k$	random	LOO	leverage	proto	marginal	shapley	greedy-FL	greedy-proj	FL $-$ best
5	0.419	0.249	0.444	0.417	0.293	0.390	0.490	0.490	$+0.047$
10	0.601	0.263	0.600	0.637	0.417	0.615	0.895	0.889	$+0.258$
15	0.690	0.267	0.741	0.719	0.500	0.678	0.909	0.906	$+0.168$
20	0.821	0.364	0.802	0.847	0.526	0.757	0.928	0.886	$+0.081$
30	0.821	0.343	0.819	0.907	0.612	0.837	0.908	0.862	$+0.001$
50	0.897	0.464	0.894	0.951	0.729	0.924	0.942	0.870	$-0.010$

Table 1: Test accuracy by selector vs. budget (

r=C=10

). Greedy-FL (context-aware coverage) beats every per-point top-

k

at the binding budget; the gap is

\delta

-shaped (peaks at

k=C

\to 0

k\gg C

). Greedy-proj is the theory’s exact selector (greedy trace-reduction / OMP on the feature second moment, Eq. 1 ); in this near-block regime it matches facility location at binding budgets (

0.889

0.895

k=C

; both cover

{\approx}9.7

–

9.8

10

classes), as Lemma 4 predicts; App. F (Table 10 ) delimits where the correspondence breaks on real features. Gaps are computed from unrounded means, so they can differ by

0.001

from differences of the displayed values.

noise	block-contrast	eff. rank	gap @ $k{=}C$
0.10	0.416	15.5	$+0.349$
0.18	0.231	22.8	$+0.295$
0.30	0.105	27.7	$+0.035$
0.50	0.040	29.4	$-0.024$
0.80	0.014	29.8	$+0.003$

Table 2: Redundancy drives the gap. As noise rises the block-contrast (off-diagonal mass) falls and the effective rank rises; the gap

=

greedy-FL

-

leverage-top-

k

k=C

collapses.

Figure 2: The interaction gap, its mechanism, and its cause (synthetic,

r=C=10

). (a) The gap greedy

-

best-scalar is

\delta

-shaped: it peaks at the binding budget

k=C

(

+0.258

) and vanishes as

k\gg C

; top-

k

-by-Shapley even falls below random near the binding budget (red). (b) Mechanism: at

k=C

greedy coverage spans

9.7/10

classes while every per-point scalar concentrates and covers far fewer (Shapley, the most mode-informative, covers fewest,

6.0

). (c) Cause: sweeping embedding noise, the

k=C

gap grows with the off-diagonal Gram mass (redundancy) and collapses to

{\approx}0

as the effective rank rises from

15.5

29.8

— a controlled causal law, not a cross-task correlation.

	CIFAR-10	Oxford Pets	CIFAR-100	Flowers-102
greedy $-$ best raw scalar (oracle)	$+0.196{\scriptstyle\,\pm 0.042}$	$+0.166{\scriptstyle\,\pm 0.021}$	$+0.119{\scriptstyle\,\pm 0.012}$	$+0.223{\scriptstyle\,\pm 0.006}$
greedy $-$ balanced prototypicality	$-0.092{\scriptstyle\,\pm 0.020}$	$-0.004{\scriptstyle\,\pm 0.015}$	$-0.081{\scriptstyle\,\pm 0.012}$	$-0.171{\scriptstyle\,\pm 0.006}$

Table 3: Modes equal labels, using ViT features at budget

k=C

, with standard errors over five seeds. A raw per-point top-

k

loses to greedy facility location, but class-balancing the same scores matches or beats greedy facility location; in this regime the apparent interaction gap is class imbalance, fixed by balancing. The comparison is deployable , not oracle: the balanced row is a single pre-specified selector , class-balanced prototypicality (which coincides with the best balanced scalar on every dataset, Table 11 ); the raw row keeps the oracle- best raw scalar, which is conservative for our claim (coverage beats even the best raw score). Per-selector matched comparisons: Table 11 .

selector ( $k=C$ )	CIFAR-10 (acc / cov $/10$ )	CIFAR-100 (acc / cov $/100$ )
random	$0.541$ / $6.3$	$0.265$ / $64.0$
KNN-Shapley, raw top- $k$	$0.433$ / $5.3$	$0.245$ / $60.3$
leverage, raw top- $k$	$0.352$ / $4.7$	$0.190$ / $55.3$
KNN-Shapley, class-balanced	$0.682$ / $10.0$	$0.363$ / $100.0$
greedy facility location	$\mathbf{0.779}$ / $9.0$	$\mathbf{0.391}$ / $76.7$

Table 4: A real Data-Shapley method (KNN-Shapley) concentrates like the other scalars (frozen ViT,

k=C

3

seeds; accuracy / classes covered). Values are computed on a held-out validation split drawn from the training pool , disjoint from the final test set. Raw top-

k

trails random and covers few classes; balancing the same values recovers; greedy coverage is strongest. The mechanism is the per-point ranking, not the particular score.

test accuracy
$k=C$	random	leverage	typicality	$k$ -center	greedy-FL
CIFAR-100	$0.274{\scriptstyle\pm.009}$	$0.175{\scriptstyle\pm.007}$	$0.224{\scriptstyle\pm.006}$	$0.247{\scriptstyle\pm.004}$	$\mathbf{0.405}{\scriptstyle\pm.004}$
Food-101	$0.175{\scriptstyle\pm.004}$	$0.129{\scriptstyle\pm.005}$	$0.063{\scriptstyle\pm.002}$	$0.204{\scriptstyle\pm.008}$	$\mathbf{0.254}{\scriptstyle\pm.006}$
Flowers-102	$0.490{\scriptstyle\pm.016}$	$0.375{\scriptstyle\pm.006}$	$0.224{\scriptstyle\pm.014}$	$0.642{\scriptstyle\pm.011}$	$\mathbf{0.686}{\scriptstyle\pm.006}$
classes covered ( $/C$ )
CIFAR-100	$64.6{\scriptstyle\pm 1.6}$	$58.2{\scriptstyle\pm 1.4}$	$51.8{\scriptstyle\pm 0.8}$	$65.6{\scriptstyle\pm 0.8}$	$\mathbf{77.8}{\scriptstyle\pm 1.4}$

Table 5: No labels to balance (frozen ViT features,

k=C

, mean

\pm

se over

5

seeds). With no labels, label-free greedy coverage beats random on every dataset by many standard errors, while a raw per-point top-

k

(leverage / global typicality) concentrates and falls below random on both accuracy and classes covered on all three datasets — the concentration mechanism on real data.

selector ( $k=C_{\rm fine}$ )	fine accuracy	fine classes ( $/100$ )
coarse-balanced random (scalar)	$0.266$	$68.3$
coarse-balanced prototypicality (scalar)	$0.240$	$57.0$
coarse-balanced $k$ -center (coverage)	$0.251$	$73.3$
coarse-balanced facility location (coverage)	$\mathbf{0.419}$	$\mathbf{83.0}$
label-free greedy coverage	$0.392$	$79.7$
oracle (fine-balanced)	$0.474$	$100.0$

Table 6: Modes not equal labels: CIFAR-100 selected with only the

20

coarse superclasses, evaluated on the

100

fine classes at

k=C_{\rm fine}

(

3

seeds). Balancing the coarse labels with a scalar is insufficient (it cannot resolve fine modes inside a superclass), but pairing the coarse quotas with an explicit within-coarse coverage step (facility location) is the best practical selector — above label-free greedy coverage — and trails only the unavailable fine-label oracle. The correction that matters is coverage/diversity, not scalar balancing. Not all diversity objectives are equally effective, though: within-coarse

k

-center improves fine-class coverage (

73.3

) but selects boundary or atypical points and scores poorly (

0.251

), whereas facility location better represents the dense fine modes and gives the best practical accuracy.

Figure 3: Modes not equal labels: coverage, not scalar balancing, is the needed correction (CIFAR-100 coarse-to-fine). Selectors use only the

20

coarse superclasses, evaluation is on the

100

fine classes. (a) Scalar coarse balancing (orange/red) is insufficient; coarse-balanced facility location (purple) — coverage within each superclass — is the best practical selector, above label-free greedy coverage (blue) and second only to the unavailable fine-label oracle (green). (b) The advantage is coverage of finer modes: the coverage selectors span the most fine classes at every budget, decisively at the binding budget

k=C_{\rm fine}

selector ( $k=C$ )	test accuracy	% selected corrupted
random	$0.156{\scriptstyle\pm.009}$	$34\%$
quality (typicality) top- $k$	$0.226{\scriptstyle\pm.004}$	$0\%$
$k$ -center (coverage)	$0.011{\scriptstyle\pm.002}$	$99\%$
greedy facility location (coverage)	$0.011{\scriptstyle\pm.003}$	$97\%$
quality-filter $+$ coverage	$\mathbf{0.387}{\scriptstyle\pm.007}$	$0\%$

Table 7: Quality and coverage are orthogonal: a controlled demonstration (CIFAR-100,

30\%

corrupted feature-outliers,

k=C

5

seeds). Pure coverage selects almost only corrupted points and collapses to chance; quality-then-coverage avoids them and is best — the decoupled recipe.

setting	$C$	eff. rank	off-diag	NMI $(C)$	ARI $(C)$	wLMM	wSPLIT	branch
CIFAR-10	$10$	$23.6$	$0.38$	$0.749$	$0.681$	$15.9$	$0.12$	balance (labels $=$ modes)
Oxford Pets	$37$	$26.4$	$0.37$	$0.841$	$0.670$	$12.3$	$0.16$	balance
CIFAR-100	$100$	$25.6$	$0.38$	$0.638$	$0.288$	$10.3$	$0.17$	balance
Flowers-102	$102$	$26.9$	$0.39$	$0.864$	$0.616$	$5.9$	$0.33$	balance
Food-101	$101$	$24.8$	$0.37$	$0.545$	$0.189$	$10.6$	$0.19$	balance
CIFAR-100, coarse labels	$20$	$25.6$	$0.38$	$0.466$	$0.287$	$14.2$	$0.13$	balance (misroute)

Table 8: Diagnostic validation, including the settings where it is stressed (frozen ViT,

3

seeds). Effective rank / off-diagonal Gram mass measure redundancy;

a=

NMI of

k

-means

(C)

vs. the selection-time labels is the routing statistic (Definition 2 ); wLMM/wSPLIT are the reported caution signals. Top: all five modes-

=

-labels datasets have

a\geq 0.545>\tau=0.45

and route to balance , matching the best selector family (Tables 11 and 14 ). Bottom: the stress rows. With coarse labels on identical CIFAR-100 features,

a

drops

0.638\to 0.466

and wLMM rises

10.3\to 14.2

, but

a

lands

0.016

above the pre-fixed

\tau

, so the router misfires (the correct branch is stratified coverage, Table 6 ); any

\tau\in(0.47,0.54)

would separate every setting correctly, but we do not move the threshold post hoc — instead the near-

\tau

value plus the paired wLMM jump trigger the pre-stated ambiguity rule (report both selectors). On Waterbirds the available 2-class labels align with the geometry (

a=0.672

; balancing them is indeed the better safe correction, § 4.3 ), while the audit row shows the true subgroup modes are not expressed (

a_{g}=0.398

) — no statistic of features and available labels can route around representation-hidden modes.

dataset	learner	random	leverage	$k$ -center	greedy-FL	gap (FL $-$ rand)
CIFAR-100	ridge	$0.274$	$0.175$	$0.247$	$\mathbf{0.405}$	$+0.131$
	MLP	$0.267{\scriptstyle\pm.007}$	$0.186{\scriptstyle\pm.010}$	$0.200{\scriptstyle\pm.008}$	$\mathbf{0.397}{\scriptstyle\pm.010}$	$+0.130$
Flowers-102	ridge	$0.490$	$0.375$	$0.642$	$\mathbf{0.686}$	$+0.195$
	MLP	$0.473{\scriptstyle\pm.016}$	$0.357{\scriptstyle\pm.006}$	$0.567{\scriptstyle\pm.015}$	$\mathbf{0.653}{\scriptstyle\pm.007}$	$+0.180$

Table 9: Beyond the ridge probe: the gap is a property of the data, not the estimator. Label-free selection at

k=C

, scored with the closed-form ridge probe vs. an SGD-trained MLP (

5

seeds,

\pm

se on the MLP). The coverage gap (greedy-FL

-

random) is nearly identical under both learners, and a raw leverage top-

k

stays below random under both. Gaps are computed from unrounded means, so they can differ by

0.001

from differences of the displayed values.

Figure 4: Coverage predicts accuracy. The

244

(fraction of classes covered, test accuracy) pairs pooled across the six settings shown — the main selector experiments, every selector family and budget (

k=C,2C,4C

), including the coarse-to-fine and KNN-Shapley suites; no new runs. Within each setting coverage and accuracy are positively rank-correlated (per-setting Spearman

r

inset, mean

+0.80

), the empirical counterpart of Proposition 1 : mode coverage is the operative proxy for downstream risk. Raw scalar top-

k

(red) concentrates — low coverage, low accuracy; balanced (green) and coverage (blue) selectors restore both.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

$k=C$	CIFAR-100	Flowers-102	Food-101
random	$0.280{\scriptstyle\pm.014}$ / $66.7$	$0.486{\scriptstyle\pm.028}$ / $60.7$	$0.179{\scriptstyle\pm.005}$ / $66.3$
$k$ -center	$0.243{\scriptstyle\pm.004}$ / $65.0$	$0.639{\scriptstyle\pm.015}$ / $76.7$	$0.195{\scriptstyle\pm.010}$ / $60.3$
greedy-proj (literal OMP)	$0.243{\scriptstyle\pm.004}$ / $64.0$	$0.459{\scriptstyle\pm.006}$ / $56.7$	$0.163{\scriptstyle\pm.005}$ / $65.0$
greedy-FL	$\mathbf{0.403}{\scriptstyle\pm.001}$ / $\mathbf{78.3}$	$\mathbf{0.683}{\scriptstyle\pm.011}$ / $\mathbf{78.7}$	$\mathbf{0.254}{\scriptstyle\pm.010}$ / $\mathbf{70.7}$

Appendix

Table 10: The theory’s literal selector vs. the empirical one, on real features (label-free selection, frozen ViT,

k=C

3

seeds, accuracy / classes covered). Unlike in the block regime (Lemma 4 , Table 1 ), raw greedy projection underperforms facility location on real features and behaves like the

k

-center family: real feature spectra are heavy-tailed, so trace-reduction spends its budget on spectral extremes (atypical directions), while the modes that carry the labels are density modes, which RBF facility location targets. This is the same boundary-point pathology the paper documents for

k

-center (Table 6 ) and quality-blind coverage (Table 7 ): the correspondence holds where the lemma’s condition

m(\kappa(0)-\kappa(\sqrt{2}))\gg L\eta

holds, and degrades with it.

raw scalar / random
selector ( $k=C$ )	CIFAR-10	Oxford Pets	CIFAR-100	Flowers-102
random	$0.554{\scriptstyle\pm.017}$	$0.453{\scriptstyle\pm.026}$	$0.246{\scriptstyle\pm.010}$	$0.479{\scriptstyle\pm.005}$
leverage	$0.373{\scriptstyle\pm.037}$	$0.413{\scriptstyle\pm.015}$	$0.193{\scriptstyle\pm.010}$	$0.364{\scriptstyle\pm.013}$
prototypicality	$0.353{\scriptstyle\pm.021}$	$0.187{\scriptstyle\pm.010}$	$0.231{\scriptstyle\pm.007}$	$0.100{\scriptstyle\pm.007}$
LOO / influence	$0.419{\scriptstyle\pm.060}$	$0.235{\scriptstyle\pm.029}$	$0.275{\scriptstyle\pm.005}$	$0.459{\scriptstyle\pm.006}$
Data Shapley	$0.563{\scriptstyle\pm.032}$	$0.505{\scriptstyle\pm.017}$	—	—

Appendix

Table 11: Full per-selector test accuracy at

k=C

(modes

=

labels; mean

\pm

se,

5

seeds), complementing Table 3 , which reports only the gap to the best selector. Class- balanced prototypicality achieves the highest mean accuracy on every dataset; raw per-point scalars (leverage, prototypicality, LOO, Data Shapley) lose to their balanced versions and often to random; greedy coverage is strong but is itself matched (Oxford Pets) or beaten by balanced prototypicality when modes

=

labels. Data Shapley (a TMC estimate) is omitted on the

100

-class datasets for cost.

selector ( $k=C$ )	CIFAR-10	CIFAR-100	Flowers-102
random	$0.491{\scriptstyle\pm.030}$	$0.275{\scriptstyle\pm.005}$	$0.474{\scriptstyle\pm.023}$
greedy facility location	$0.759{\scriptstyle\pm.023}$	$0.395{\scriptstyle\pm.004}$	$0.688{\scriptstyle\pm.013}$
EL2N hard, raw / balanced	$0.187$ / $0.173$	$0.060$ / $0.099$	$0.178$ / $0.502$
EL2N easy, raw / balanced	$0.383$ / $\mathbf{0.848}$	$0.209$ / $\mathbf{0.497}$	$0.050$ / $\mathbf{0.829}$
GraNd hard, raw / balanced	$0.378$ / $0.661$	$0.168$ / $0.287$	$0.236$ / $0.789$
GraNd easy, raw / balanced	$0.278$ / $0.524$	$0.227$ / $0.365$	$0.234$ / $0.696$

Appendix

Table 12: Training-dynamics scores obey the same law (

k=C

3

seeds, test accuracy; pool excludes the

1000

-point validation split, so the reference rows differ slightly from Table 11 . Classes covered: every raw variant covers fewer classes than random — as few as

2.0/10

20.3/100

5.0/102

— while every balanced variant covers all

C

.) Raw top-

k

of every score and orientation trails random (

18/18

); balancing restores coverage in all cases and good accuracy exactly when the within-class ranking is quality-sane (EL2N-easy best of family on all three datasets).

dataset	learner	random	leverage	$k$ -center	greedy-FL	gap (FL $-$ rand)
CIFAR-100	ridge	$0.280{\scriptstyle\pm.014}$	$0.179{\scriptstyle\pm.005}$	$0.243{\scriptstyle\pm.004}$	$\mathbf{0.403}{\scriptstyle\pm.001}$	$+0.123$
	logistic	$0.291{\scriptstyle\pm.016}$	$0.212{\scriptstyle\pm.045}$	$0.175{\scriptstyle\pm.007}$	$\mathbf{0.412}{\scriptstyle\pm.005}$	$+0.121$
	$1$ -NN	$0.263{\scriptstyle\pm.013}$	$0.193{\scriptstyle\pm.012}$	$0.137{\scriptstyle\pm.004}$	$\mathbf{0.399}{\scriptstyle\pm.002}$	$+0.136$
Flowers-102	ridge	$0.486{\scriptstyle\pm.028}$	$0.374{\scriptstyle\pm.011}$	$0.639{\scriptstyle\pm.015}$	$\mathbf{0.683}{\scriptstyle\pm.011}$	$+0.197$
	logistic	$0.481{\scriptstyle\pm.026}$	$0.408{\scriptstyle\pm.004}$	$0.509{\scriptstyle\pm.028}$	$\mathbf{0.702}{\scriptstyle\pm.002}$	$+0.220$
	$1$ -NN	$0.455{\scriptstyle\pm.018}$	$0.377{\scriptstyle\pm.011}$	$0.445{\scriptstyle\pm.007}$	$\mathbf{0.682}{\scriptstyle\pm.004}$	$+0.227$

Appendix

Table 13: The gap is learner-independent (label-free selection,

k=C

3

seeds): the coverage gap (greedy-FL

-

random) and the below-random raw leverage top-

k

persist across ridge, logistic regression, and

1

-NN. Gaps are computed from unrounded means, so they can differ by

0.001

from differences of the displayed values.

(a) label-free selection: accuracy
dataset	$k$	random	leverage	typicality	$k$ -center	greedy-FL
CIFAR-100	$C$	$0.274{\scriptstyle\pm.009}$	$0.175{\scriptstyle\pm.007}$	$0.224{\scriptstyle\pm.006}$	$0.247{\scriptstyle\pm.004}$	$\mathbf{0.405}{\scriptstyle\pm.004}$
	$2C$	$0.387{\scriptstyle\pm.013}$	$0.303{\scriptstyle\pm.010}$	$0.315{\scriptstyle\pm.010}$	$0.355{\scriptstyle\pm.007}$	$\mathbf{0.485}{\scriptstyle\pm.002}$
	$4C$	$0.461{\scriptstyle\pm.012}$	$0.385{\scriptstyle\pm.006}$	$0.385{\scriptstyle\pm.008}$	$0.432{\scriptstyle\pm.005}$	$\mathbf{0.523}{\scriptstyle\pm.004}$
Flowers-102	$C$	$0.490{\scriptstyle\pm.016}$	$0.375{\scriptstyle\pm.006}$	$0.224{\scriptstyle\pm.014}$	$0.642{\scriptstyle\pm.011}$	$\mathbf{0.686}{\scriptstyle\pm.006}$
	$2C$	$0.624{\scriptstyle\pm.011}$	$0.500{\scriptstyle\pm.011}$	$0.382{\scriptstyle\pm.011}$	$0.698{\scriptstyle\pm.005}$	$\mathbf{0.759}{\scriptstyle\pm.009}$

Appendix

Table 14: Complete budget sweeps (

k=C,2C,4C

; mean

\pm

se over seeds; same runs as the main-text tables). Top: label-free selection, test accuracy. Bottom: modes

=

labels, the two headline gaps. Gaps are computed from unrounded means, so they can differ by

0.001

from differences of the displayed values.

	$b{=}0.5$	$\mathbf{b{=}1}$	$b{=}2$	$d{=}32$	$d{=}128$
CIFAR-100 gap	$+0.127$	$\mathbf{+0.123}$	$+0.109$	$+0.062$	$+0.118$
Flowers-102 gap	$+0.210$	$\mathbf{+0.197}$	$+0.194$	$+0.129$	$+0.235$

Appendix

Table 15: Sensitivity of the coverage gap (greedy-FL vs random,

k=C

3

seeds) to the RBF bandwidth multiplier

b

(

\sigma^{2}=b\times

median) and the PCA dimension

d

. Default

b=1

d=64

in bold.

	CIFAR-100		Flowers-102
geometry	block contrast	gap (FL $-$ rand)	block contrast	gap (FL $-$ rand)
noise $\sigma{=}2.0$	$0.037$	$-0.007$	$0.056$	$-0.051$
noise $\sigma{=}1.0$	$0.094$	$+0.044$	$0.146$	$+0.093$
noise $\sigma{=}0.5$	$0.155$	$+0.110$	$0.243$	$+0.187$
real features	$0.197$	$+0.135$	$0.312$	$+0.219$
shrink $\lambda{=}0.3$	$0.299$	$+0.152$	$0.422$	$+0.267$

Appendix

Table 16: The causal law on real features (

k=C

3

seeds). The pool geometry is dialed from diffuse (noise

\sigma\times

feature scale) through the real features to near-block (within-class shrinkage

\lambda

); block contrast is the measured within

-

cross class kernel gap, and the selection gap (greedy-FL

-

random) generally increases with it (monotonically on Flowers-102, one small local reversal on CIFAR-100), vanishing at the diffuse end — effective rank spans

58\!\to\!15

(CIFAR-100). No test-side information is touched.

dataset	$C$	$m_{\star}$ (exact)	random (obs.)	leverage	proto	typicality	greedy-FL
CIFAR-10	$10$	$6.51$	$6.51$	$5.7$	$2.7$	$3.7$	$8.7$
Oxford Pets	$37$	$23.64$	$23.56$	$21.0$	$7.0$	$11.0$	$32.0$
CIFAR-100	$100$	$63.44$	$63.50$	$56.7$	$31.3$	$52.7$	$78.3$
Flowers-102	$102$	$59.28$	$59.04$	$49.0$	$10.3$	$38.7$	$78.7$
Food-101	$101$	$64.02$	$63.93$	$53.0$	$13.3$	$51.0$	$70.7$

Appendix

Table 17: The occupancy formula predicts real coverage with zero parameters (classes covered at

k=C

3

seeds,

300

random draws per seed). Random selection sits on the exact prediction

m_{\star}

computed from each pool’s actual class counts; every raw scalar top-

k

falls below it (concentration); greedy coverage exceeds it.

selector	$k=C$	$k=2C$	$k=4C$
random	$0.275{\scriptstyle\pm.024}$ / $13.0$	$0.298{\scriptstyle\pm.009}$ / $17.3$	$0.339{\scriptstyle\pm.013}$ / $20.0$
leverage top- $k$	$0.156{\scriptstyle\pm.002}$ / $12.0$	$0.179{\scriptstyle\pm.014}$ / $16.0$	$0.208{\scriptstyle\pm.009}$ / $19.3$
typicality top- $k$	$0.112{\scriptstyle\pm.007}$ / $10.7$	$0.134{\scriptstyle\pm.019}$ / $14.3$	$0.166{\scriptstyle\pm.017}$ / $18.0$
$k$ -center	$0.146{\scriptstyle\pm.013}$ / $11.3$	$0.192{\scriptstyle\pm.014}$ / $16.0$	$0.225{\scriptstyle\pm.008}$ / $19.7$
greedy-FL	$0.358{\scriptstyle\pm.016}$ / $14.7$	$0.383{\scriptstyle\pm.016}$ / $18.7$	$0.414{\scriptstyle\pm.009}$ / $20.0$
bal-prototypicality	$\mathbf{0.426}{\scriptstyle\pm.009}$ / $20.0$	$\mathbf{0.439}{\scriptstyle\pm.004}$ / $20.0$	$\mathbf{0.464}{\scriptstyle\pm.002}$ / $20.0$

Appendix

Table 18: The gap on language embeddings (20 Newsgroups, frozen MiniLM,

C=20

3

seeds; accuracy / classes covered). The vision pattern replicates: raw scalar top-

k

below random with under-coverage, coverage above random, balanced prototypicality best.

$k$	random	leverage	$k$ -center	greedy-FL	bal-proto
$C$	$0.053{\scriptstyle\pm.003}$	$0.041{\scriptstyle\pm.004}$	$0.044{\scriptstyle\pm.000}$	$\mathbf{0.066}{\scriptstyle\pm.008}$	$\mathbf{0.066}{\scriptstyle\pm.004}$
$2C$	$0.062{\scriptstyle\pm.004}$	$0.044{\scriptstyle\pm.006}$	$0.054{\scriptstyle\pm.006}$	$0.072{\scriptstyle\pm.002}$	$\mathbf{0.082}{\scriptstyle\pm.003}$
$4C$	$0.094{\scriptstyle\pm.003}$	$0.064{\scriptstyle\pm.004}$	$0.066{\scriptstyle\pm.005}$	$0.109{\scriptstyle\pm.006}$	$\mathbf{0.120}{\scriptstyle\pm.004}$

Appendix

Table 19: End-to-end pilot (CNN trained from scratch on raw CIFAR-100 pixels on the selected subset;

3

seeds). Absolute accuracies are small at these subset sizes, but the selector ordering of the frozen-feature experiments persists at every budget.

$k$	random	leverage	greedy-FL	bal-proto
$C$	$0.253{\scriptstyle\pm.014}$	$0.201{\scriptstyle\pm.011}$	$0.380{\scriptstyle\pm.001}$	$\mathbf{0.488}{\scriptstyle\pm.007}$
$4C$	$0.539{\scriptstyle\pm.001}$	$0.439{\scriptstyle\pm.007}$	$0.614{\scriptstyle\pm.002}$	$\mathbf{0.648}{\scriptstyle\pm.004}$

Appendix

Table 20: LoRA fine-tuning of the selection backbone (rank-

8

adapters on all ViT-B/16 attention layers

+

fresh head, trained on the selected subset; CIFAR-100,

3

seeds, mean

\pm

se). The frozen-feature ordering — leverage

<

random

<

greedy-FL

\leq

balanced prototypicality — holds in every run.

backbone	dataset	random	leverage	$k$ -center	greedy-FL	bal-proto
ViT-B/16	CIFAR-100	$0.280{\scriptstyle\pm.014}$	$0.179{\scriptstyle\pm.005}$	$0.243{\scriptstyle\pm.004}$	$0.403{\scriptstyle\pm.001}$	$\mathbf{0.482}{\scriptstyle\pm.012}$
	Flowers-102	$0.486{\scriptstyle\pm.028}$	$0.374{\scriptstyle\pm.011}$	$0.639{\scriptstyle\pm.015}$	$0.683{\scriptstyle\pm.011}$	$\mathbf{0.843}{\scriptstyle\pm.005}$
ResNet-50	CIFAR-100	$0.182{\scriptstyle\pm.009}$	$0.127{\scriptstyle\pm.003}$	$0.209{\scriptstyle\pm.004}$	$0.240{\scriptstyle\pm.012}$	$\mathbf{0.307}{\scriptstyle\pm.004}$
	Flowers-102	$0.269{\scriptstyle\pm.009}$	$0.188{\scriptstyle\pm.004}$	$0.310{\scriptstyle\pm.005}$	$0.377{\scriptstyle\pm.005}$	$\mathbf{0.492}{\scriptstyle\pm.004}$
CLIP ViT-B/32	CIFAR-100	$0.211{\scriptstyle\pm.007}$	$0.222{\scriptstyle\pm.009}$	$0.300{\scriptstyle\pm.004}$	$0.289{\scriptstyle\pm.005}$	$\mathbf{0.348}{\scriptstyle\pm.007}$
	Flowers-102	$0.303{\scriptstyle\pm.008}$	$0.213{\scriptstyle\pm.005}$	$0.371{\scriptstyle\pm.014}$	$0.397{\scriptstyle\pm.006}$	$\mathbf{0.474}{\scriptstyle\pm.004}$

Appendix

Table 21: The mechanism is encoder-robust (

k=C

3

seeds, accuracy; label-free selectors plus class-balanced prototypicality). Across ImageNet ViT-B/16, ImageNet ResNet-50, and CLIP ViT-B/32: greedy coverage beats random and balanced prototypicality is best in every cell; the raw leverage top-

k

trails random in five of six cells (ties on CLIP/CIFAR-100).

unequal sizes $9{\times}1{+}91$ , $k=9$ : uniform $1.807$ , rare-favoring $9.000$ (Rem 2 )
$k$	$m_{\star}(k)$ (theory)	uniform (sim)	concentrating (sim)	greedy
$2$	$1.918$	$1.918$	$1.000$	$2$
$5$	$4.234$	$4.235$	$1.000$	$5$
$8$	$5.985$	$5.984$	$2.000$	$8$
$10$	$6.894$	$6.899$	$2.000$	$10$

Appendix

Table 22: Direct validation of the occupancy theorem (

r=10

modes, equal size

m=5

unless noted;

2\times 10^{4}

trials). Uniform sampling agrees with the closed form

m_{\star}(k)

within

0.007

modes (Thm 2 ); a concentrating score stays

\leq m_{\star}

(Prop 3 ); unequal modes reverse the bound (Rem 2 ).

selector	CIFAR-100	Flowers-102	Food-101
random	$0.280{\scriptstyle\pm.014}$ (66.7)	$0.486{\scriptstyle\pm.028}$ (60.7)	$0.179{\scriptstyle\pm.005}$ (66.3)
greedy facility location	$0.403{\scriptstyle\pm.001}$ (78.3)	$0.683{\scriptstyle\pm.011}$ (78.7)	$0.254{\scriptstyle\pm.010}$ (70.7)
information gain, greedy (target-agnostic)	$0.282{\scriptstyle\pm.007}$ (66.3)	$0.643{\scriptstyle\pm.007}$ (73.7)	$0.229{\scriptstyle\pm.004}$ (63.3)
EPIG, greedy (target-aware)	$0.411{\scriptstyle\pm.008}$ (76.7)	$0.698{\scriptstyle\pm.017}$ (78.7)	$0.270{\scriptstyle\pm.006}$ (74.7)
EPIG, fixed top- $k$	$0.216{\scriptstyle\pm.013}$ (51.0)	$0.167{\scriptstyle\pm.013}$ (28.3)	$0.069{\scriptstyle\pm.007}$ (48.3)

Appendix

Table 23: Information-theoretic objectives as label-free selectors (protocol of Table 10 :

k=C

, three seeds; accuracy

\pm

se, classes covered in parentheses;

\sigma^{2}=0.1

). Adaptive, target-aware greedy EPIG matches facility location; the same score used as a fixed ranking falls below random.

dataset	$A$ (geometry modes)	$A$ (classes)	slope vs. $U(S)$	$R^{2}$	acc. on uncovered
CIFAR-10	$0.737$	$0.795$	$0.278{\scriptstyle\pm.012}$	$0.84$	$0.046$
Oxford Pets	$0.707$	$0.732$	$0.215{\scriptstyle\pm.005}$	$0.95$	$0.000$
CIFAR-100	$0.494$	$0.503$	$0.047{\scriptstyle\pm.003}$	$0.75$	$0.000$
Flowers-102	$0.720$	$0.740$	$0.164{\scriptstyle\pm.002}$	$0.98$	$0.000$
Food-101	$0.456$	$0.438$	$0.028{\scriptstyle\pm.002}$	$0.73$	$0.000$

Appendix

Table 24: A direct test of the mode-aligned-target assumption (Proposition 1 ).

A

: share of target variance constant within label-free modes (

k

-means,

r=C

) or classes; slope: in-span excess risk per unit of uncovered class mass at

k=C

(hard-mode value

1

Data Selection

Data selection chooses informative, representative, or reliable examples to improve model training and evaluation while reducing data, compute, or labeling costs.

See all 84 papers

Oct 24, 2025

Adaptive Data Selection for Text Classification: Hierarchical Value with a Local Utility Budget

Selecting compact training sets from heterogeneous text sources requires balancing target relevance, label reliability, and coverage. For small classifiers, discarding unfamiliar but useful texts can be as harmful as retaining annotation errors. Gradient-based selection estimates immediate target utility, whereas confidence and coverage scores capture complementary properties. Combining them with fixed weights leaves the utility sacrificed for this evidence uncontrolled across candidate pools. We propose Data Value Contribution (\DVC), which gates lexical, hidden-layer, and prediction-level evidence by neighborhood label support and calibrates its influence under a local utility budget; exact MLP gradient contractions and a monotone scoring path make the budget directly verifiable. At 20% source retention, DVC achieves 75.4 mean macro F1 across eight classification and domain-transfer tasks, exceeding the strongest baseline under shared final training by 0.7 points and random selection by 4.3; an equal-budget tuning study confirms a 0.6-point gain over calibrated quality-only selection.

Xiyang Zhang, Hongzhi Wang, Yuanhe Tian +1

Faculty of Computing, Harbin Institute of Technology

Training Data Selection Data Valuation Text Classification

Ask PDF

Sep 22, 2026

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner

Is the same data equally useful to every learner? The question is whether the value of a training sample belongs to the sample or arises in its interaction with the learner. We test it with a selection experiment. Under a per-class budget, two kinds of samples compete for the slots: easy samples, which a short training run learns first, and coverage samples, which spread over the class in feature space. Easy samples win at small budgets and coverage samples at large ones, and the budget at which the winner flips, the crossover m0, can be measured. We ask whether m0 is set by the data or by the learner. To separate the two, we freeze the selected subsets and change only the learner that trains on them. On low-resolution ImageNet-100, doubling the width of a ResNet-18 moves the crossover from 57 to 85 samples per class on the same subsets. Enlarging the input grid from 32 to 64 pixels at fixed image information moves it from 85 to 57, and a single stride change reproduces or removes this shift. The grid shift shrinks as width grows. Under the native ImageNet-1k protocol, a 4x range of width moves the boundary by at most a few samples per class. On a Vision Transformer, coverage wins at every measured budget, even with easy subsets that work on the convolutional learner. The budget at which one kind of sample stops being the more useful kind therefore belongs to the learner, and the preferred regime of a selection strategy has to be read against the learner that will use the data.

Yangze Liu, Xiao-Long Yin, Zhongyi Han

Shandong University · Nanjing University

Training Data Selection Coreset Selection Data Valuation

Ask PDF

Jul 4, 2026

Validation-Induced Shapley Shifts: How Validation Structure Distorts Data Valuation

Shapley values are widely used to attribute value to training data based on their marginal contribution to performance on a validation set. Existing practice often assumes these values are stable once the training data and model are fixed. In this work, we uncover a systematic vulnerability: even modest changes to the validation set, such as introducing noises, cause directional shifts in Shapley distributions. As noises are added, Shapley values of training samples compress toward zero. We trace this to a noise-induced neighborhood reshuffling effect: perturbations alter the local rank order between validation and training samples, flattening the valuation landscape. Using the KNN-Shapley framework, we show through synthetic and real data that these shifts are consistent and reproducible. Our findings challenge the assumption of Shapley stability and reveal a new axis of fragility in data valuation. We propose normalization and boundary-aware validation strategies to mitigate these distortions and enable more robust, interpretable valuation in machine learning marketplaces.

Yinan Shen, Ziao Yang, Hongfu Liu

Adobe Waltham, MA 02451, USA · Department of Computer Science Brandeis University Waltham, MA 02453-2728, USA

Shapley Value Attribution Data Valuation Training Data Attribution

Ask PDF

Abstract

Data Selection

Explore similar work

Adaptive Data Selection for Text Classification: Hierarchical Value with a Local Utility Budget

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner

Validation-Induced Shapley Shifts: How Validation Structure Distorts Data Valuation