Less Is More in the Long Tail: Stage-Adaptive Sample Selection for Annotation-Efficient Dense Prediction

Xiaofei Du; Lei Zhang; Shuyu Yan; Manning Wang; Zhijian Song

cs.CVSep 22, 2026

Less Is More in the Long Tail: Stage-Adaptive Sample Selection for Annotation-Efficient Dense Prediction

Authors: Xiaofei Du, Lei Zhang, Shuyu Yan, Manning Wang, Zhijian Song

Organizations: Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China · Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China · Frontier Innovation Center, Department of Systems Biology for Medicine, Qidong-Fudan Innovative Institute of Medical Sciences, School of Basic Medical Sciences, Zhongshan Hospital Clinical Center for Biotherapy, Fudan University, Shanghai 200032, China · Department of Computer Science, University of Exeter, Exeter EX4 4QF, United Kingdom

Abstract

Deep learning performance generally improves with increasing training data, yet this scaling is fundamentally constrained by annotation cost in large-scale dense prediction tasks with long-tailed category distributions, where pixel- or voxel-level annotation is prohibitively expensive. We propose SASS (Stage-Adaptive Sample Selection), a stage-adaptive data-selection framework for pool-based active learning in long-tailed dense prediction. SASS combines three components: label-free self-supervised gradient scoring, prior-guided category rebalancing with validation-driven feedback, and stage-adaptive acquisition aligned with model training dynamics. This design avoids candidate ground-truth masks during gradient scoring while making acquisition responsive to long-tail imbalance and evolving representations. We evaluate SASS on a multimodal 3D medical segmentation testbed comprising over 100,000 samples spanning 108 anatomical structures. SASS recovers 98.3% of full-dataset performance with a 40% training-pool annotation budget, outperforming BADGE by 5.1 percentage points. Moreover, SASS exhibits a statistically supported less-is-more pattern, surpassing full-dataset training at the Hard-group level and, at the structure level, for the pancreas and gallbladder. More broadly, SASS shows that annotation-efficient learning depends not only on which samples are selected, but also on how the annotation budget is distributed across categories and when model-derived scores begin to guide selection.

Figures & tables

Panel of Figure 1 — Fig. 1 : Overview of SASS. The training pool grows progressively from $|\mathcal{S}^{(0)}|\approx 0.10N$ to the target budget through periodic expansion. At each selection round, the Self-Supervised Acquisition Scorer computes label-free gradient scores $G_{i}(t)$ from a teacher-student DINO objective, while the Prior-Guided Scorer produces a Scale-Aware Prior $O_{i}$ and a Coverage-Balance Prior $D_{i}(t)$ , both enhanced by Validation-Driven Feedback $\phi(c,t)$ . The Stage-Adaptive Acquisition Score normalizes and combines these signals into $S_{i}(t)$ , with weights shifting from prior-dominant exploration to gradient-dominant refinement. In the ternary simplex, each stage is placed at its weight vector, the three vertices denoting full weight on $G_{i}$ , $O_{i}$ and $D_{i}$ ; Stage 1 lies on the $O_{i}$ – $D_{i}$ edge, where the gradient weight is zero, whereas Stages 2 and 3 lie inside the triangle, so both priors keep non-zero weight. Selected samples are refined by a Group-Budget Constraint and reordered by a Forgetting-Guided Curriculum before training.

Group	Random	Entropy	Coreset	BADGE	LESS	LESS+O	SASS	Full	Full+CB	$\Delta$ (S–R)
Easy ( $n{=}59$ )	0.651 $\pm$ 0.003	0.665 $\pm$ 0.003	0.661 $\pm$ 0.003	0.674 $\pm$ 0.003	0.674 $\pm$ 0.003	0.691 $\pm$ 0.003	0.715 $\pm$ 0.003	0.733 $\pm$ 0.002	0.732 $\pm$ 0.003	+0.064
Medium ( $n{=}31$ )	0.596 $\pm$ 0.004	0.610 $\pm$ 0.004	0.605 $\pm$ 0.004	0.618 $\pm$ 0.004	0.618 $\pm$ 0.004	0.641 $\pm$ 0.004	0.675 $\pm$ 0.004	0.690 $\pm$ 0.003	0.693 $\pm$ 0.004	+0.079
Hard ( $n{=}18$ )	0.525 $\pm$ 0.006	0.538 $\pm$ 0.006	0.534 $\pm$ 0.006	0.549 $\pm$ 0.005	0.553 $\pm$ 0.005	0.578 $\pm$ 0.005	0.622 $\pm$ 0.005	0.610 $\pm$ 0.005	0.618 $\pm$ 0.005	+0.097
Overall (108)	0.614 $\pm$ 0.003	0.628 $\pm$ 0.003	0.624 $\pm$ 0.003	0.637 $\pm$ 0.003	0.638 $\pm$ 0.003	0.658 $\pm$ 0.003	0.688 $\pm$ 0.003	0.700 $\pm$ 0.002	0.702 $\pm$ 0.003	+0.074

TABLE I : Difficulty-stratified test Dice at epoch 300 under 10-click evaluation (mean

\pm

std, three runs). Subset methods use 40%; Full and Full+CB use 100%. Bold: best 40%-budget result.

Fig. 2 : Qualitative segmentation comparison on four anatomical structures. Red: ground truth; green: prediction; dashed white: ground-truth boundary. LESS+O denotes LESS+Organ.

Structure	Rand	Full	SASS	$\Delta$ (S–R)
Liver	.701 $\pm$ .006	.793 $\pm$ .005	.781 $\pm$ .005	+.080
Kidney (L/R)	.688 $\pm$ .007	.776 $\pm$ .005	.764 $\pm$ .006	+.076
Spleen	.695 $\pm$ .007	.782 $\pm$ .005	.771 $\pm$ .006	+.076
Pancreas	.508 $\pm$ .012	.590 $\pm$ .009	.607 $\pm$ .010 †	+.099
Gallbladder	.518 $\pm$ .011	.598 $\pm$ .009	.614 $\pm$ .010 †	+.096
Aorta	.637 $\pm$ .008	.723 $\pm$ .006	.714 $\pm$ .007	+.077

TABLE II : Dice for 18 high-priority reporting targets under the 10-click protocol at epoch 300. Rows include bilateral and lung-lobe aggregates and report mean

\pm

std across three runs; the final row is the unweighted mean of the 18 displayed row means. Targets were selected by diagnostic relevance [ 28 ] .

\dagger

\ddagger

: see Section IV-A2 .

Fig. 3 : Category-level budget comparison. SASS-40% closely tracks Full-100% across all six categories while improving substantially over Random and SASS-20%.

Structure	$n_{s}$	Rand	Full	SASS	$\Delta$ (S–F)	95% CI	$\phi$
Pancreas	1	0.508 $\pm$ 0.012	0.590 $\pm$ 0.009	0.607 $\pm$ 0.010 †	+0.017	$[0.004,0.030]$	0.72
Adrenal (L/R)	1	0.483 $\pm$ 0.015	0.567 $\pm$ 0.012	0.586 $\pm$ 0.012 ‡	+0.019	$[-0.001,0.039]$	0.81
Esophagus	1	0.497 $\pm$ 0.013	0.578 $\pm$ 0.010	0.593 $\pm$ 0.011 ‡	+0.015	$[-0.003,0.033]$	0.68
Heart RV	1	0.418 $\pm$ 0.017	0.496 $\pm$ 0.014	0.507 $\pm$ 0.014 ‡	+0.011	$[-0.008,0.030]$	0.89
Heart LA	1	0.441 $\pm$ 0.015	0.534 $\pm$ 0.012	0.549 $\pm$ 0.013 ‡	+0.015	$[-0.005,0.035]$	0.83
Lung vessel	1	0.453 $\pm$ 0.014	0.545 $\pm$ 0.011	0.559 $\pm$ 0.012 ‡	+0.014	$[-0.004,0.032]$	0.76

TABLE III : Per-structure Dice and bootstrap uncertainty for seven representative Hard structures (mean

\pm

std, three runs).

n_{s}

: canonical structure categories per row after the mapping in Supplementary Table S1;

\phi

: validation-feedback gap.

\dagger

marks differences whose 95% CI excludes zero; intervals are not adjusted for multiplicity across structures.

\ddagger

: positive mean difference whose interval includes zero.

Category	Random	Full	Full+CB	Entropy	Coreset	BADGE	LESS	LESS+O	SASS	$\Delta$ (S–R)
A: Abdominal	0.628 $\pm$ 0.004	0.714 $\pm$ 0.003	0.716 $\pm$ 0.004	0.643 $\pm$ 0.004	0.639 $\pm$ 0.004	0.651 $\pm$ 0.004	0.654 $\pm$ 0.004	0.675 $\pm$ 0.003	0.705 $\pm$ 0.004	+0.077
B: Thorax	0.589 $\pm$ 0.004	0.681 $\pm$ 0.003	0.684 $\pm$ 0.004	0.603 $\pm$ 0.004	0.599 $\pm$ 0.004	0.611 $\pm$ 0.004	0.612 $\pm$ 0.004	0.635 $\pm$ 0.004	0.668 $\pm$ 0.004	+0.079
C: GI Tract	0.601 $\pm$ 0.005	0.695 $\pm$ 0.004	0.698 $\pm$ 0.004	0.616 $\pm$ 0.005	0.612 $\pm$ 0.005	0.624 $\pm$ 0.004	0.626 $\pm$ 0.004	0.649 $\pm$ 0.004	0.683 $\pm$ 0.004	+0.082
D: Vascular	0.568 $\pm$ 0.005	0.662 $\pm$ 0.004	0.666 $\pm$ 0.005	0.583 $\pm$ 0.005	0.578 $\pm$ 0.005	0.591 $\pm$ 0.005	0.594 $\pm$ 0.005	0.619 $\pm$ 0.005	0.649 $\pm$ 0.005	+0.081
E: Brain & Tumor	0.614 $\pm$ 0.005	0.706 $\pm$ 0.004	0.709 $\pm$ 0.005	0.628 $\pm$ 0.005	0.624 $\pm$ 0.005	0.636 $\pm$ 0.005	0.639 $\pm$ 0.005	0.657 $\pm$ 0.005	0.691 $\pm$ 0.004	+0.077
F: Skeletal	0.641 $\pm$ 0.003	0.727 $\pm$ 0.003	0.726 $\pm$ 0.003	0.652 $\pm$ 0.004	0.649 $\pm$ 0.004	0.659 $\pm$ 0.003	0.663 $\pm$ 0.003	0.677 $\pm$ 0.003	0.704 $\pm$ 0.003	+0.063

TABLE IV : Per-category test Dice at epoch 300 (mean

\pm

std, three runs). Subset methods use 40%; Full and Full+CB use 100%. Bold: best 40%-budget result.

Fig. 4 : Hyperparameter sensitivity. Overall and Hard-structure Dice remain stable around the adopted configuration (dashed lines) for (a) stage boundary

T_{1}

, (b) feedback sharpness

\kappa

, and (c) initial pool ratio

\rho_{0}

$\rho$	Samples	Overall	Easy	Med.	Hard	Recovery
0.20	${\sim}$ 14k	0.660	0.692	0.645	0.582	94.3%
0.40	28k	0.688	0.715	0.675	0.622	98.3%
0.60	${\sim}$ 42k	0.694	0.721	0.682	0.626	99.1%
1.00	70,351	0.700	0.733	0.690	0.610	100%

TABLE V : Annotation-budget scaling on the held-out test set (epoch 300, 10-click). Recovery is relative to Full overall Dice.

Method	Budget	Overall	Medium	Hard
Full Dataset	100%	0.687	0.677	0.596
Random	40%	0.602	0.583	0.512
LESS+Organ	40%	0.647	0.629	0.563
SASS	40%	0.675	0.663	0.604

TABLE VI : Volume-disjoint robustness check. All 40%-budget methods use the same annotation budget.

Fig. 5 : Cumulative ablation at 40% budget. Bars: overall test Dice; red values: incremental gains; purple: Hard-structure Dice. Components grouped into scoring, adaptive acquisition, and stabilization.

Fig. 6 : Component interaction analysis. Each cell: Dice gain over Random for the corresponding pair; diagonal: individual components. Parentheses report the interaction beyond additivity, computed as the paired gain minus the sum of the two individual gains.

Fig. 7 : Selection evolution. (a) Category distribution per query batch; dashed line: Skeletal candidate-pool share (29.5%). (b) Difficulty-group distribution: Hard share rises from 19.8% to 29.4% (

2.1{\times}

candidate-pool proportion). (c) Final cumulative pool by method: SASS reduces Skeletal to 14.8%, shifting budget toward Abdominal (

{\sim}

32%) and GI (

{\sim}

14%).

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 296 papers

Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation
Jun 18, 2026Rémi Hattat, Marine Beaumont, Charline Bertholdt +3Active Learning 3D Medical Image Segmentation
IADI (U1254), Inserm and Université de Lorraine, Nancy, France · CIC-IT 1433, Inserm, Université de Lorraine, and CHRU Nancy, Nancy, France
How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification
Aug 4, 2026Julia Machnio, Mads Nielsen, Mostafa Mehdipour GhaziActive Learning Medical Image Classification
Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark
SemiSAM-O1: How far can we push the boundary of annotation-efficient medical image segmentation?
Apr 27, 2026Yichi Zhang, Le Xue, Bichun Xu +6Image Segmentation Medical Imaging Foundation Models
Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China. · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China. · Department of Radiotherapy, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China. +2

Jun 18, 2026J/K move · Enter open · S save

Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation

Rémi Hattat, Marine Beaumont, Charline Bertholdt, Gabriela Hossu+2

IADI (U1254), Inserm and Université de Lorraine, Nancy, France · CIC-IT 1433, Inserm, Université de Lorraine, and CHRU Nancy, Nancy, France

Active Learning 3D Medical Image Segmentation

Open paper AskPDF arXiv

Deep learning for 3D medical image segmentation requires extensive manual annotations, a major bottleneck in volumetric medical imaging. Active learning aims to reduce this burden by selecting informative samples for annotation, but most methods assume that an initial labeled set is already available. This leaves the cold-start problem largely unresolved: how to select the first volumes from a fully unlabeled pool before any task-specific model is trained. We propose CSCS, a Curriculum-Stratified Cold-Start framework that adapts initial sample selection to the structure of the unlabeled dataset. CSCS combines two self-supervised, label-free signals: local typicality, measuring representativeness in the embedding space, and reconstruction-based uncertainty, used as a proxy for sample difficulty. These signals are combined through a weighted geometric score, where the weighting is determined by a closed-form pacing rule based on the effective annotation budget and the Difficulty-Coverage Ratio, a pool-level statistic measuring the alignment between difficulty and representativeness. We evaluate CSCS on four 3D medical image segmentation benchmarks: BraTS, FeTA, Spleen, and an in-house fetal MRI dataset. Using nnU-Net as downstream segmentation model, CSCS shows consistently competitive performance across datasets and annotation budgets, with the strongest gains in low-to-mid annotation regimes. These results suggest that dataset-aware cold-start initialization can improve the robustness of active learning for 3D medical image segmentation by adapting sample selection to the geometry of the unlabeled pool.

Abstract

Active Learning

Explore similar work

Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation

How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification

SemiSAM-O1: How far can we push the boundary of annotation-efficient medical image segmentation?