OT-CAL: Optimal-Transport-Guided Adaptive Sample Selection for Efficient Visual Learning
Organizations: Department of Computer Science, Purdue University Fort Wayne, Fort Wayne, Indiana, USA
Abstract
Deep neural networks for image recognition are typically trained using mini-batches, with a forward pass to compute predictions and losses followed by a more computationally expensive backward pass to update model parameters. However, not all training samples contribute equally to learning, suggesting that computation can be reduced by performing backward passes only on informative samples. The challenge is that sample informativeness changes throughout training: difficult samples may be valuable early on, improving samples become important later, and class diversity becomes increasingly relevant toward the end. Existing methods often rely on fixed selection rules or computationally expensive per-sample gradient calculations. We propose OT-CAL, an adaptive data selection method that automatically adjusts its sampling preferences during training. OT-CAL represents each sample using five informativeness features, including difficulty, learnability, and uncertainty. A curriculum vector describes the currently preferred sample characteristics, and optimal-transport-based matching assigns selection weights according to how closely samples match this target. The curriculum vector is then updated using the selected samples, forming a feedback loop that adapts without a manually defined schedule. Experiments on three standard visual learning benchmarks show that, when selecting 50% of each mini-batch, OT-CAL improves performance by 1.3--2.1 points over AdaSelection while achieving 1.18--1.24 times faster training than full-data training.
Figures & tables
| Symbol | Meaning |
|---|---|
| , | Mini-batch size (128 for classification); number of classes |
| Profile of sample (5-D; learned 8-D variant ) | |
| Curriculum vector: preferred profile at iteration | |
| Initial curriculum: mean profile of the first mini-batch | |
| , | Perturbed target ; noise std. (0.05) |
| Matching cost of sample , |
| Dataset / model | Full | AdaSel | SB | DM | OT-CAL | Speedup | |
| CIFAR-100 / R18 | 77.5 | 74.2 | 74.5 | 74.4 | 75.8 0.3 | +1.6 | |
| Tiny-IN / R50 | 64.3 | 62.8 | 63.0 | 62.9 | 64.9 0.4 | +2.1 | |
| Tiny-IN / ViT-B | 68.5 | 67.1 | 67.4 | 67.3 | 68.7 0.3 | +1.6 | |
| COCO / DETR | 42.0 | 41.6 | 41.8 | 41.7 | 42.9 0.3 | +1.3 | |
| Mean std over 10 seeds (CIFAR-100) or 5 seeds (others); std shown for OT-CAL. COCO reports box mAP. is relative to AdaSelection (points); all differences significant ( , Holm). SB: selective backpropagation [ 23 ] ; DM: DataMap [ 21 ] . OT-CAL uses learned 8-D profiles. † CIFAR-100 speedup measured with the fixed 5-D profile (Fig. 2 ); the learned 8-D variant adds a small MLP cost and has not been re-timed. | |||||||
| Profile | Selector | Acc. (%) | Hamming | |
|---|---|---|---|---|
| Fixed 5-D | Top- | 75.1 | 0.68 | n/a |
| Fixed 5-D | OT | 75.4 | 0.68 | n/a |
| Learned 8-D | Top- | 75.6 | 0.83 | 0.88 |
| Learned 8-D | OT | 75.8 | 0.83 | 0.94 |
| : linear-regression fit of last-layer per-sample gradient norms from the profile, on a random 10k training subset, averaged over checkpoints taken every 25 epochs. Hamming: selection similarity on a fixed probe set of 1,024 training samples, between the selection masks of consecutive iterations, averaged over training; not logged for the 5-D runs (n/a). Mean over 10 seeds. | ||||
| Feature removed | Acc. (%) | Drop (pp) | Share |
|---|---|---|---|
| None (full profile) | 75.4 | — | — |
| Learnability ( ) | 74.1 | 41% | |
| Variety ( ) | 74.6 | 25% | |
| Uncertainty ( ) | 74.8 | 19% | |
| Gradient proxy ( ) | 75.1 | 9% | |
| Difficulty ( ) | 75.2 | 6% |
Training Data Selection
Training data selection studies how to choose, filter, weight, or schedule examples to improve learning while reducing data and compute costs.
See all 193 papers