Boundary Mass in Feature Space as a Label-Budget Diagnostic for Representations
Organizations: Shanghai University
Abstract
Low-label transfer is often decided by a practical question: after a representation looks useful, how many labels are still needed to learn the downstream boundary? Transferability and probe-quality scores characterize global alignment, likelihood, margins, and calibration. Boundary-near target mass complements these diagnostics by measuring the ambiguous regions where scarce labels are most costly. We propose effective boundary complexity, a label-budget diagnostic for frozen representations and linear probes. It measures mass in a thin strip around the learned boundary and normalizes it by class balance, so dense ambiguous regions and predicted-class imbalance are both counted. We prove that, under a smooth representation map, density change and tangential surface change cancel at first order, leaving boundary-normal stretch or compression as the governing factor. This yields a cached-feature estimator that fits a probe on one split and counts independent held-out features near its boundary. Experiments support the diagnostic at three levels. Controlled transformations match the predicted law across 74 maps with 0.38% median relative error. On real frozen features, normal-direction changes move both measured boundary mass and label demand, while tangent controls preserve the first-order coefficient. A differentiable surrogate reduces measured complexity by 16-36% at matched top-1 utility. Finally, CIFAR-100 and Tiny-ImageNet experiments support boundary mass as a second-stage diagnostic for near-tied representations. Five-way experiments show strong OvR-interface backbone-ranking agreement (Spearman 0.872), while acquisition experiments show high correlations between initial boundary mass and realized label budgets across three sampling policies (0.779-0.830). Code is available at https://github.com/qqaqq-code/boundary_mass.
Figures & tables
| 5-spc acc. | AULC | budget | ||
|---|---|---|---|---|
| 0.5 | ||||
| 1.0 | ||||
| 2.0 | ||||
| 4.0 |
| Setting | Method | Top-1 | Low-label probe | |
|---|---|---|---|---|
| CIFAR-100 10 spc | standard / -int. | – | ||
| CIFAR-100 50 spc | standard / -int. | – | ||
| Tiny-ImageNet 10 spc | standard / -int. | – | ||
| Tiny-ImageNet 50 spc | standard / -int. | – | ||
| CIFAR-100 R18 | standard | |||
| CIFAR-100 R18 | weight decay |
| Comparison | Spearman | Concordance | Backbone agreement |
|---|---|---|---|
| Hard OvR vs. Hard Interface | |||
| Smooth OvR vs. Smooth Interface |
| Policy | Realized budget | Spearman | Selection regret |
|---|---|---|---|
| Random | |||
| Uncertainty | |||
| Hybrid |
Appendix figures & tables39 assets
Supplementary material from the paper’s appendix.
Appendix
| Check | heads | Pearson | Spearman | error / diff |
|---|---|---|---|---|
| split-half | 160 | |||
| hard-tube vs. kernel | 160 |
| Setting | n | rank stability | median shift |
|---|---|---|---|
| eps0.05/tau0.025/rho1e-06 | 54 | ||
| eps0.05/tau0.05/rho1e-06 | 54 | ||
| eps0.1/tau0.025/rho1e-06 | 54 | ||
| eps0.1/tau0.05/rho1e-06 | 54 | ||
| eps0.1/tau0.1/rho1e-06 | 54 | ||
| eps0.2/tau0.05/rho1e-06 | 54 |
| Setting | heads | rank stability | mean regret | rel. regret | AULC |
|---|---|---|---|---|---|
| fixed | 160 | ||||
| tube | 160 | ||||
| tube | 160 | ||||
| tube | 160 | ||||
| margin-scaled tube | 160 | ||||
| hard tube | 160 |
| Protocol | heads | rank stability | mean regret | rel. regret | AULC |
|---|---|---|---|---|---|
| raw features | 160 | ||||
| standardized | 160 | ||||
| L2-normalized | 160 | ||||
| PCA-whitened | 160 |
| Benchmark | runs | classes | feature dim. |
|---|---|---|---|
| CIFAR-100/R50 | 106 | 100 | 2048 |
| CIFAR-100/ViT-B | 27 | 100 | 768 |
| Tiny-IN/R50 | 45 | 200 | 2048 |
| Asset | Used for | Citation |
|---|---|---|
| CIFAR-100 | binary class-pair experiments, interventions | [ Krizhevsky, 2009 ] |
| Tiny-ImageNet | external replication | [ Le and Yang, 2015 ] |
| ImageNet / ILSVRC | pretraining source for frozen backbones | [ Deng et al., 2009 ; Russakovsky et al., 2015 ] |
| ResNet-50 / ResNet-18 | backbone / end-to-end evaluation | [ He et al., 2016 ] |
| ViT-B/16 | frozen backbone | [ Dosovitskiy et al., 2021 ] |
| DINO-ViT-B/16 | frozen backbone | [ Caron et al., 2021 ] |
| Mode | runs | median dist. | mean dist. | expansion+compression |
|---|---|---|---|---|
| psieff | 118 | |||
| standard | 54 | |||
| weight decay | 78 |
| Full-probe evaluation | Acquisition correlation | ||||
|---|---|---|---|---|---|
| Score | Top-2 regret | Random | Uncertainty | Hybrid | |
| Baseline set | base | + | ||
|---|---|---|---|---|
| NLL + ECE | ||||
| margin + effective rank | ||||
| NLL + ECE + margin + rank | ||||
| all non- proxies |
| Baseline set | base | + | partial corr. | ||
|---|---|---|---|---|---|
| LogME + H-score | |||||
| LogME + H-score + TransRate | |||||
| feature transferability set | |||||
| all non- + transferability |
| Metric | label-budget corr. | selection acc. | mean regret | rel. regret |
|---|---|---|---|---|
| 0.493/0.266 | ||||
| 0.499/0.266 | ||||
| Gaussian-area proxy | 0.240/0.036 | |||
| noise flip proxy | 0.207/0.157 | |||
| median margin | -0.293/-0.239 | |||
| held-out NLL | 0.525/0.404 |
| Metric | label-budget corr. |
|---|---|
| LogME | |
| H-score | |
| TransRate | |
| NCE-style cluster |
| Baseline set | base | + | partial corr. | ||
|---|---|---|---|---|---|
| PACTran-linear | |||||
| feature transferability + PACTran-linear | |||||
| all non- + transferability + PACTran-linear |
| Baseline set | base | + | partial corr. | ||
|---|---|---|---|---|---|
| LEEP + NCE | |||||
| source-logit set | |||||
| source + feature transferability | |||||
| all non- + source/feature |
| Metric | label-budget corr. |
|---|---|
| LEEP | |
| NCE | |
| source-logit LogME |
| Baseline set | base | + | partial corr. | ||
|---|---|---|---|---|---|
| NLL + ECE | |||||
| margin + effective rank | |||||
| feature transferability | |||||
| all non- proxies | |||||
| all non- + transferability |
| Metric | selection acc. | mean regret | rel. regret | mean AULC |
|---|---|---|---|---|
| LogME | ||||
| H-score | ||||
| NLL | ||||
| ECE | ||||
| margin |
| Setting | tasks | backbones | base regret | regret | reduction | AULC | |
|---|---|---|---|---|---|---|---|
| Tiny-ImageNet 5-way OvR | 16 | 4 | |||||
| Tiny-ImageNet 5-way interface | 16 | 4 |
| Policy | regret | reduction | AULC | win rate |
|---|---|---|---|---|
| Non- base | ||||
| OvR top-2 | ||||
| OvR abstain | ||||
| Interface top-2 | ||||
| Interface abstain | ||||
| Non- + OvR reg. |
| Label policy | Realized budget | Spearman (kernel , budget) | Selection regret |
|---|---|---|---|
| Random | |||
| Uncertainty | |||
| Hybrid |
| pilot spc | policy | selection acc. | mean regret | rel. regret | mean AULC |
|---|---|---|---|---|---|
| 2 | choose by | ||||
| 2 | choose by margin | ||||
| 2 | NLL/ECE | ||||
| 2 | LogME/H-score | ||||
| 2 | transferability set | ||||
| 2 | combined non- |
| pilot spc | tasks with lower budget | mean budget reduction | mean relative reduction |
|---|---|---|---|
| 2 | |||
| 5 | |||
| 10 |
| Pilot spc | Refit SD | Backbone-order agreement | Lower- exact | Combined+ exact | Budget agreement |
|---|---|---|---|---|---|
| 2 | |||||
| 5 | |||||
| 10 |
| Policy | selection acc. | mean regret | rel. regret | mean AULC |
|---|---|---|---|---|
| LogME | ||||
| H-score | ||||
| TransRate | ||||
| transferability set | ||||
| transferability set + |
| Full-shot acc. gap | comparisons | lower- win/tie score | mean relative saving |
|---|---|---|---|
| 176 | |||
| 227 |
| Metric | acc.-drop corr. | robust-selection acc. |
|---|---|---|
| Gaussian-area proxy | ||
| noise flip proxy | ||
| median margin | ||
| NLL | ||
| ECE |
| Backbone | Evaluation status |
|---|---|
| CLIP ViT-B/16 | Pending evaluation |
| DINO ViT-B/16 | Completed |
| DINOv2 Base | Pending evaluation |
| MAE ViT-Base | Pending evaluation |
| SigLIP-B/16 | Completed |
| Comparison | Within-task Spearman | Pairwise concordance | Selected-backbone agreement | Mean rank displacement |
|---|---|---|---|---|
| Hard OvR vs. Hard Interface | ||||
| Smooth OvR vs. Hard OvR | ||||
| Smooth Interface vs. Hard Interface | ||||
| Smooth OvR vs. Smooth Interface |
| Predictor | spc-resid. Pearson | spc-resid. Spearman | mixed Pearson | Runs |
|---|---|---|---|---|
| 106 | ||||
| 106 | ||||
| Weight norm | 106 | |||
| Normalized margin | 106 | |||
| Feature effective rank | 106 | |||
| Noise flip rate | 106 |
| Dataset | spc | Standard Top-1 / | -int. Top-1 / | Complexity drop |
|---|---|---|---|---|
| CIFAR-100 | 10 | |||
| CIFAR-100 | 20 | |||
| CIFAR-100 | 50 | |||
| Tiny-ImageNet | 10 | |||
| Tiny-ImageNet | 20 | |||
| Tiny-ImageNet | 50 |
| Method | Test Acc. | ECE | feat. norm | margin | 20/50-spc probe | |||
|---|---|---|---|---|---|---|---|---|
| Standard | ||||||||
| Weight decay | ||||||||
| Margin reg. | ||||||||
| Entropy reg. | ||||||||
| -reg, |
| Test Acc. | ECE | 20/50-spc probe | |||
|---|---|---|---|---|---|
| Method | Test Acc. | ECE | 20/50-spc probe | ||
|---|---|---|---|---|---|
| Standard | |||||
| Weight decay | |||||
| -reg |
Task Transferability Estimation
Task transferability estimation predicts which models, datasets, tasks, or skills will help on a new target before costly adaptation.
See all 42 papers