Low-Budget Active Learning through Entropic Optimal Transport

Rim Hajal; Mathieu Besançon; Jérôme Malick

cs.LGOct 1, 2026

Low-Budget Active Learning through Entropic Optimal Transport

Authors: Rim Hajal, Mathieu Besançon, Jérôme Malick

Organizations: Univ. Grenoble Alpes, CNRS, Inria, LIG, LJK · Univ. Grenoble Alpes, Inria, CNRS, LIG · Univ. Grenoble Alpes, CNRS, LJK

Abstract

We consider low-budget active learning, which consists of selecting a limited number of points, the coreset, such that a model can be trained to high accuracy on the selection only. This problem is particularly relevant in contexts where labeling requires costly expert intervention, as in medical applications. We leverage features extracted from a pretrained self-supervised model to represent the data, and perform coreset selection directly in this feature space. In this paper, we use entropic optimal transport, specifically the Sinkhorn divergence, as the coreset selection criterion, which first allows us to get dimension-free sample complexity results, and second admits computationally efficient gradient evaluations. This opens the way to using gradient-based algorithms to rapidly compute solution candidates, further improved by a swap-based local search, with guarantees on the solution quality. Experiments on image benchmarks and medical datasets show that our method outperforms state-of-the-art heuristics in low-budget settings.

Figures & tables

Panel of Figure 1 — Figure 1: $\hat{\mu}^{\mathcal{D}}$ for the $\texttt{CCO}(\Delta,m)$ instances. For $B\leq 2$ , we would like to select one point in $-\Delta$ and another in $\Delta$ (the extreme configuration).

Figure 4: Test accuracy (mean

\pm

std) on GasHisSDB.

Method	2	4	6	8	10
FW-Swap	$\mathbf{0.805\pm 0.002}$	$\mathbf{0.714\pm 0.003}$	$\mathbf{0.783\pm 0.003}$	$\mathbf{0.748\pm 0.004}$	$\mathbf{0.783\pm 0.002}$
MaxHerding	$0.487\pm 0.002$	$0.687\pm 0.003$	$0.749\pm 0.002$	$0.705\pm 0.002$	$0.714\pm 0.003$
TypiClust	$0.501\pm 0.002$	$0.642\pm 0.014$	$0.674\pm 0.006$	$0.697\pm 0.027$	$0.732\pm 0.020$
Medoid	$0.588\pm 0.106$	$\mathbf{0.716\pm 0.063}$	$0.735\pm 0.050$	$0.744\pm 0.048$	$0.765\pm 0.038$
k-Center	$0.500\pm 0.001$	$0.547\pm 0.058$	$0.574\pm 0.055$	$0.592\pm 0.057$	$0.629\pm 0.057$
ProbCover	$0.515\pm 0.001$	$0.564\pm 0.002$	$0.645\pm 0.002$	$0.653\pm 0.002$	$0.659\pm 0.002$

Table 1: Test accuracy (mean

\pm

std) on EEGMMIDB dataset.

Method	2	4	6	8	10
FW-Swap	$\mathbf{0.758\pm 0.079}$	$\mathbf{0.773\pm 0.041}$	$\mathbf{0.811\pm 0.029}$	$\mathbf{0.819\pm 0.010}$	$\mathbf{0.829\pm 0.033}$
MaxHerding	$0.499\pm 0.003$	$0.460\pm 0.012$	$0.488\pm 0.023$	$0.538\pm 0.026$	$0.596\pm 0.037$
TypiClust	$0.499\pm 0.008$	$0.507\pm 0.003$	$0.511\pm 0.022$	$0.514\pm 0.012$	$0.556\pm 0.066$
Medoid	$0.550\pm 0.114$	$0.664\pm 0.119$	$0.757\pm 0.061$	$0.740\pm 0.086$	$0.737\pm 0.090$
k-Center	$0.498\pm 0.037$	$0.521\pm 0.025$	$0.524\pm 0.034$	$0.515\pm 0.029$	$0.546\pm 0.050$
ProbCover	$0.503\pm 0.008$	$0.673\pm 0.053$	$0.645\pm 0.060$	$0.656\pm 0.049$	$0.662\pm 0.060$

Table 2: Test accuracy (mean

\pm

std) on the WBCIC-SHU Motor Imagery dataset.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Budget	End Cycle	Start Cycle	Detected Cycle	Detected Std	Cycle Length
10	$11.0$	$12.0$	$14.0$	$1.22474$	$3.0$
20	$22.6$	$23.6$	$25.6$	$2.19089$	$3.0$
30	$33.2$	$34.2$	$36.2$	$1.30384$	$3.0$
40	$44.8$	$45.8$	$47.8$	$2.68328$	$3.0$
50	$54.8$	$55.8$	$57.8$	$2.77489$	$3.0$
60	$64.2$	$65.2$	$67.2$	$4.91935$	$3.0$

Appendix

Table 3: Details of FW-Swap trajectory for different budgets.

Budget	Acquired + Initial	Percentage
10	$(1+8=9)$	$90\%$
20	$(2+16=18)$	$90\%$
30	$(5+24=29)$	$96.67\%$
40	$(8+31=39)$	$97.5\%$
50	$(8+38=46)$	$92\%$
60	$(9+48=57)$	$95\%$

Appendix

Table 4: Number of acquired points and initial points retained in the final selection.

Run	True Minimum	FW-Swap	Gap	Rand-Swap	Random Subset
1	$0.02316$	$0.02549$	$0.00233$	$0.04523$	$0.06378$
2	$0.02522$	$0.02794$	$0.00272$	$0.04341$	$0.14044$
3	$0.02436$	$0.02676$	$0.00239$	$0.03632$	$0.12600$

Appendix

Table 5: Comparison of the objective values obtained by different subset selections over three runs.

Method	10	20	30	40	50	60
FW-Swap	$\mathbf{0.521\pm 0.018}$	$\mathbf{0.569\pm 0.008}$	$0.611\pm 0.008$	$0.613\pm 0.020$	$0.652\pm 0.012$	$0.625\pm 0.006$
MaxHerding	$0.462\pm 0.042$	$0.547\pm 0.027$	$0.605\pm 0.032$	$\mathbf{0.642\pm 0.032}$	$\mathbf{0.674\pm 0.021}$	$0.679\pm 0.021$
TypiClust	$0.483\pm 0.035$	$0.553\pm 0.047$	$0.614\pm 0.020$	$0.637\pm 0.029$	$0.664\pm 0.025$	$\mathbf{0.691\pm 0.016}$
Medoid	$0.440\pm 0.062$	$0.566\pm 0.057$	$\mathbf{0.635\pm 0.021}$	$0.638\pm 0.031$	$0.669\pm 0.026$	$0.683\pm 0.013$
k-Center	$0.444\pm 0.037$	$0.568\pm 0.028$	$0.612\pm 0.037$	$0.631\pm 0.027$	$0.673\pm 0.020$	$0.688\pm 0.015$
ProbCover	$0.432\pm 0.010$	$0.483\pm 0.006$	$0.574\pm 0.003$	$0.598\pm 0.012$	$0.653\pm 0.008$	$0.679\pm 0.005$

Appendix

Table 6: Test accuracy (mean

\pm

std) across active learning methods and budgets on STL-10 with feature space defined by SimCLR.

Method	10	20	30	40	50	60
FW-Swap	$\mathbf{0.537\pm 0.016}$	$0.606\pm 0.018$	$0.639\pm 0.010$	$0.683\pm 0.020$	$\mathbf{0.744\pm 0.009}$	$0.749\pm 0.008$
MaxHerding	$0.468\pm 0.027$	$0.601\pm 0.023$	$0.689\pm 0.008$	$0.704\pm 0.014$	$0.734\pm 0.011$	$\mathbf{0.763\pm 0.007}$
TypiClust	$0.477\pm 0.040$	$0.553\pm 0.035$	$0.639\pm 0.034$	$0.668\pm 0.028$	$0.693\pm 0.041$	$0.732\pm 0.012$
Medoid	$0.518\pm 0.044$	$\mathbf{0.630\pm 0.043}$	$\mathbf{0.714\pm 0.033}$	$\mathbf{0.723\pm 0.020}$	$0.740\pm 0.025$	$0.757\pm 0.022$
k-Center	$0.444\pm 0.059$	$0.610\pm 0.062$	$0.670\pm 0.037$	$0.715\pm 0.037$	$0.723\pm 0.025$	$0.741\pm 0.017$
ProbCover	$0.454\pm 0.014$	$0.561\pm 0.012$	$0.633\pm 0.011$	$0.670\pm 0.026$	$0.721\pm 0.010$	$0.746\pm 0.005$

Appendix

Table 7: Test accuracy (mean

\pm

std) across active learning methods and budgets on CIFAR-10 with feature space defined by SimCLR.

Method	10	20	30	40	50	60
FW-Swap	$\mathbf{0.699\pm 0.008}$	$\mathbf{0.768\pm 0.011}$	$\mathbf{0.829\pm 0.012}$	$0.852\pm 0.016$	$0.879\pm 0.008$	$0.906\pm 0.004$
MaxHerding	$0.612\pm 0.033$	$0.719\pm 0.037$	$0.813\pm 0.023$	$0.846\pm 0.029$	$0.890\pm 0.018$	$0.899\pm 0.018$
TypiClust	$0.617\pm 0.030$	$0.735\pm 0.031$	$0.805\pm 0.038$	$0.851\pm 0.023$	$0.890\pm 0.014$	$\mathbf{0.911\pm 0.014}$
Medoid	$0.606\pm 0.026$	$0.714\pm 0.058$	$0.797\pm 0.035$	$0.841\pm 0.025$	$0.869\pm 0.042$	$0.902\pm 0.022$
k-Center	$0.648\pm 0.019$	$0.749\pm 0.029$	$0.809\pm 0.030$	$\mathbf{0.861\pm 0.018}$	$\mathbf{0.899\pm 0.011}$	$0.910\pm 0.006$
ProbCover	$0.668\pm 0.015$	$0.755\pm 0.018$	$0.750\pm 0.015$	$0.739\pm 0.022$	$0.743\pm 0.023$	$0.724\pm 0.013$

Appendix

Table 8: Test accuracy (mean

\pm

std) across active learning methods and budgets on STL-10.

Method	10	20	30	40	50	60
FW-Swap	$\mathbf{0.307\pm 0.002}$	$\mathbf{0.496\pm 0.004}$	$\mathbf{0.497\pm 0.003}$	$\mathbf{0.576\pm 0.004}$	$0.577\pm 0.002$	$\mathbf{0.648\pm 0.003}$
MaxHerding	$0.110\pm 0.002$	$0.307\pm 0.003$	$0.468\pm 0.003$	$0.516\pm 0.002$	$\mathbf{0.598\pm 0.002}$	$0.606\pm 0.004$
TypiClust	$0.283\pm 0.010$	$0.402\pm 0.026$	$0.447\pm 0.048$	$0.509\pm 0.042$	$0.563\pm 0.029$	$0.599\pm 0.053$
Medoid	$0.245\pm 0.038$	$0.353\pm 0.076$	$0.437\pm 0.055$	$0.563\pm 0.057$	$0.575\pm 0.037$	$0.635\pm 0.047$
k-Center	$0.156\pm 0.043$	$0.226\pm 0.034$	$0.278\pm 0.038$	$0.326\pm 0.034$	$0.376\pm 0.036$	$0.416\pm 0.038$
ProbCover	$0.256\pm 0.004$	$0.309\pm 0.002$	$0.369\pm 0.002$	$0.430\pm 0.005$	$0.477\pm 0.003$	$0.530\pm 0.003$

Appendix

Table 9: Test accuracy (mean

\pm

std) across active learning methods and budgets on SVHN.

Method	80	100
FW-Swap	$\mathbf{0.721\pm 0.002}$	$\mathbf{0.752\pm 0.002}$
MaxHerding	$0.667\pm 0.002$	$0.723\pm 0.003$
TypiClust	$0.666\pm 0.025$	$0.708\pm 0.015$
Medoid	$0.693\pm 0.031$	$0.715\pm 0.028$
k-Center	$0.502\pm 0.037$	$0.581\pm 0.025$
ProbCover	$0.622\pm 0.002$	$0.691\pm 0.003$

Appendix

Table 10: Test accuracy (mean

\pm

std) across active learning methods and budgets on STL-10.

Method	2	4	6	8	10
FW-Swap	$\mathbf{0.665\pm 0.003}$	$0.698\pm 0.004$	$\mathbf{0.820\pm 0.002}$	$\mathbf{0.809\pm 0.002}$	$\mathbf{0.838\pm 0.002}$
MaxHerding	$0.606\pm 0.001$	$0.605\pm 0.002$	$0.609\pm 0.003$	$0.723\pm 0.004$	$0.769\pm 0.002$
TypiClust	$0.632\pm 0.008$	$0.691\pm 0.058$	$0.734\pm 0.032$	$0.730\pm 0.061$	$0.711\pm 0.077$
Medoid	$0.598\pm 0.116$	$0.693\pm 0.067$	$0.754\pm 0.074$	$0.806\pm 0.061$	$0.803\pm 0.047$
k-Center	$0.601\pm 0.070$	$0.668\pm 0.061$	$0.715\pm 0.075$	$0.754\pm 0.044$	$0.770\pm 0.040$
ProbCover	$0.505\pm 0.008$	$\mathbf{0.718\pm 0.005}$	$0.695\pm 0.005$	$0.704\pm 0.007$	$0.688\pm 0.005$

Appendix

Table 11: Test accuracy (mean

\pm

std) across active learning methods and budgets on GasHisSDB.

Explore similar work

One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning
Aug 4, 2026Ning Zhu, Xiaochuan Ma, Juntao Xu +4Pool-Based Active Learning Cold-Start
Glasgow College, University of Electronic Science and Technology of China · School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China · School of Mathematical Sciences, University of Electronic Science and Technology of China
Group-invariant Coresets for Data-efficient Active Learning
Jul 1, 2026L. C. Ayres, J. C. M. Bermudez, S. J. M. de Almeida +1Geometry-Aware Uncertainty Coresets Pool-Based Active Learning
Universidade Federal de Santa Catarina, Florianópolis, SC, Brazil · Universidade Católica de Pelotas, Pelotas, RS, Brazil · Université de Lorraine, CNRS, CRAN, Vandoeuvre-lès-Nancy, France
Active Learning with Low-Rank Structure for Data Selection
Jun 14, 2026Vincent Cohen-Addad, Sasidhar Kunapuli, Vahab Mirrokni +3Geometry-Aware Uncertainty Coresets Low-Rank Structure
Google Research. · University of California, Berkeley. · Institute of Science and Technology Austria (ISTA). +2

Aug 4, 2026, cs.AIJ/K move · Enter open · S save

One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning

Ning Zhu, Xiaochuan Ma, Juntao Xu, Jingze Liang+3

Glasgow College, University of Electronic Science and Technology of China · School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China · School of Mathematical Sciences, University of Electronic Science and Technology of China

Pool-Based Active Learning Cold-Start Optimal Transport Approach Balanced Learning Regularization

Open paper AskPDF arXiv

Cold-Start Active Learning (CSAL) aims to select a valuable subset from an unlabeled pool without any prior knowledge or human assistance. Existing methods take diverse routes based on typicality, coverage, or diversity. Each rests on its own inductive bias and therefore performs well on some tasks yet poorly on others. We argue that the real challenge is not to design yet another selection heuristic, but to make CSAL adapt automatically to the data and task at hand. To this end, we revisit CSAL through the lens of optimal transport. First, we propose a generalized transport selection framework that reveals the shared allocation structure of existing methods and exactly subsumes representative formulations. Second, we introduce a theoretical analysis that characterizes the trade-off controlled by entropic regularization and establishes a task-agnostic minimax bound for cold-start selection. These results provide a principled foundation for adapting the regularization strength to the unlabeled data. Third, we derive a data-adaptive regularization rule and present a novel Sinkhorn-based CSAL algorithm, termed

ε

-Adaptive Selection (

ε

-AS). Extensive experiments on six public datasets and multiple annotation budgets show that

ε

-AS consistently achieves state-of-the-art performance. On ImageNet-1k, it improves the average accuracy over ActiveFT by 1.29% while reducing selection time by 56.2%. Code will be released at https://github.com/Z-yiwei/OT-CSAL

Abstract

Explore similar work

One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning

Group-invariant Coresets for Data-efficient Active Learning

Active Learning with Low-Rank Structure for Data Selection