cs.LGSep 6, 2026

Train Smarter, Not Harder: Switching Signal-Guided Training in Active Learning

Authors: Nagham Omar, Maya Rozenshtein, Evgeny Mishlyakov, Avigdor Gal

Organizations: Faculty of Data and Decision Sciences, Technion

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes. We propose HybridAL, an adaptive training schedule that monitors an online stabilization signal and switches from retraining to fine-tuning after sustained stabilization. Two complementary signals, spectral exponent change ΔαΔα (weight-based) and accuracy change ΔΔAcc (validation-based), span different points on the time-calibration trade-off. Across three encoder backbones and six text-classification tasks (five seeds each), HybridAL keeps endpoint macro-F1 non-inferior to retraining and fine-tuning at a 0.010 margin, saves up to 49% of retraining time, and recovers a substantial fraction of retraining's calibration advantage as measured by negative log-likelihood (NLL). Compared with schedules that switch at a pre-committed round, HybridAL obtains lower NLL at moderate additional cost, showing that trajectory-dependent switching provides a stronger time-calibration trade-off than fixed early switching.

Figures & tables

Figure 1: No single training strategy dominates. Left: speed (bigger = faster). Right: calibration (bigger = lower negative log-likelihood (NLL), i.e. better-calibrated predictions). Axes are min–max normalized per task-family dataset. HybridAL variants use Δα\Delta\alpha (spectral exponent change) and Δ\Delta Acc (accuracy change) as switching signals.
DatasetDomainCSize
Binary
IMDb ( Maas et al., 2011 )sentiment analysis250,000
SST-2 ( Socher et al., 2013 )sentiment analysis268,221
Jigsaw ( Wulczyn et al., 2017 )toxicity detection2159,571
Multi-class
TweetEval ( Barbieri et al., 2020 )sentiment analysis359,899
Table 1: Text classification datasets, showing total size and number of classes (C).
SignalTypeExtra val. pass?Fire rateMean t⋆t^{\star}
Δ\Delta Acc ⋆performanceyes93%9.1
Δ\Delta F1performanceyes90%10.0
Δ\Delta Lossperformanceyes43%13.4
Grad. normperformanceyes40%8.5
Δα⋆\Delta\alpha^{\star}model-basedno97%7.9
ℓ2\ell_{2} distancemodel-basedno0%✗
Table 2: Signal ablation (DistilBERT, 6 datasets ×\times 5 seeds, normalised ε=0.5\varepsilon{=}0.5 , k=3k{=}3 ). Fire rate: fraction that switched; mean t⋆t^{\star} : switch round among firing cells. ⋆ Selected for main results.
Figure 2: Test F1 preservation. (a) Endpoint test F1 distributions per task family (45 cells: 3 backbones ×\times 3 datasets ×\times 5 seeds); median (white dot) and IQR (black bar) overlaid. (b) Per-round validation F1; ±1\pm 1 SEM bands; stars mark each HybridAL variant’s mean switch round t⋆t^{\star} ; insets zoom the convergence region (rounds 15-25). Method colors follow the shared legend below.
Figure 3: Time–calibration trade-off. (a) Mean training time vs. mean test NLL per (method, backbone) (color = method, shape = backbone); dashed line is the all-method Pareto frontier, dotted excludes NewOnly . (b) Per-round mean validation NLL pooled across 6 datasets ×\times 3 backbones ×\times 5 seeds; ±1\pm 1 SEM bands; stars mark each HybridAL variant’s mean switch round t⋆t^{\star} . Method colors follow the shared legend below.
Figure 4: Mean test NLL. Bar color encodes mean switch round. Retrain (dashed) and FineTune (dash-dot) mark the calibration ceiling and floor.
Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

DatasetClassTrain ( nn )Train (%)Val. ( nn )Val. (%)Test ( nn )Test (%)
IMDbnegative19,75050.0025050.005,00050.00
positive19,75050.0025050.005,00050.00
Total39,500—500—10,000—
Jigsawnon-toxic113,97890.421,44390.4128,85690.42
toxic12,0829.581539.593,0599.58
Total126,060—1,596—31,915—
Appendix
Table 3: Per-class distribution of each dataset across train, validation and test splits (seed=42 splits as used in our experiments). Train counts reflect the AL pool after validation holdout; Total rows give the size of each split.
DistilBERT
MethodIMDbAGNewsJigsawSST-2TwtEvYahoo
Retrain0.819 ±\pm 0.0210.900 ±\pm 0.0040.881 ±\pm 0.0030.835 ±\pm 0.0210.634 ±\pm 0.0230.667 ±\pm 0.004 ‡§¶⋆∙∘
FineTune0.823 ±\pm 0.008 ¶0.904 ±\pm 0.0050.883 ±\pm 0.0040.858 ±\pm 0.0180.626 ±\pm 0.0210.655 ±\pm 0.006 §
NewOnly0.823 ±\pm 0.0060.899 ±\pm 0.0010.882 ±\pm 0.0070.854 ±\pm 0.0090.552 ±\pm 0.1170.637 ±\pm 0.009
HybridAL ( Δα\Delta\alpha )0.814 ±\pm 0.0140.902 ±\pm 0.0070.881 ±\pm 0.004 ∘0.844 ±\pm 0.0220.627 ±\pm 0.0250.656 ±\pm 0.004 §
HybridAL ( Δ\Delta Acc)0.827 ±\pm 0.0060.905 ±\pm 0.0050.880 ±\pm 0.0050.857 ±\pm 0.0120.629 ±\pm 0.0180.654 ±\pm 0.005 §
Appendix
Table 4: Full per-(method ×\times backbone ×\times dataset) test F1 (mean ±\pm std across 5 seeds). Cell shading is per column within each backbone (greener == higher == better). Superscripts indicate that the cell is significantly higher than the reference method (paired tt -test, p<0.05p<0.05 , two-sided): †\dagger Retrain , ‡\ddagger FineTune , \lx@sectionsign\lx@sectionsign NewOnly , \lx@paragraphsign\lx@paragraphsign HybridAL ( Δα\Delta\alpha ), ⋆\star HybridAL ( Δ\Delta Acc), ∙\bullet FixedSwitch (3), ∘\circ FixedSwitch (5), ⋄\diamond FixedSwitch (7), △\triangle FixedSwitch (10).
VariantReferenceLower boundδ=0.005\delta{=}0.005δ=0.010\delta{=}0.010
Δα\Delta\alphaRetrain+0.0005+0.0005✓✓
Δα\Delta\alphaFineTune−0.0045-0.0045✓✓
Δ\Delta AccRetrain−0.0001-0.0001✓✓
Δ\Delta AccFineTune−0.0050-0.0050✗✓
Δα\Delta\alphaper-cell best−0.0086-0.0086✗✓
Δ\Delta Accper-cell best−0.0091-0.0091✗✓
Appendix
Table 5: TOST non-inferiority of endpoint F1: one-sided 95%95\% lower confidence bounds on the mean paired difference, n=90n{=}90 (backbone, dataset, seed) cells. ✓: non-inferior at that margin.
MethodIMDbAGNewsJigsawSST-2TwtEvYahooMean
DistilBERT
Retrain0.426 ±\pm 0.039 ‡¶∙∘⋄△0.320 ±\pm 0.008 ‡§¶⋆∙∘⋄△0.133 ±\pm 0.008 ‡⋆∙⋄0.391 ±\pm 0.0290.814 ±\pm 0.066 ‡¶⋆∙∘⋄△1.108 ±\pm 0.018 ‡§¶⋆∙∘⋄△0.532 ±\pm 0.336 ‡§¶⋆∙∘⋄△
FineTune0.614 ±\pm 0.0750.448 ±\pm 0.0420.153 ±\pm 0.0160.460 ±\pm 0.0531.379 ±\pm 0.0751.609 ±\pm 0.0300.777 ±\pm 0.540
NewOnly0.501 ±\pm 0.036 ‡∙∘⋄△0.362 ±\pm 0.013 ‡⋆∙∘⋄△0.133 ±\pm 0.007 ‡⋆∙⋄0.435 ±\pm 0.0410.880 ±\pm 0.105 ‡∙∘⋄△1.265 ±\pm 0.046 ‡⋆∙∘⋄△0.596 ±\pm 0.382 ‡¶⋆∙∘⋄△
HybridAL ( Δα\Delta\alpha )0.641 ±\pm 0.1400.408 ±\pm 0.0590.150 ±\pm 0.023 ∘0.412 ±\pm 0.038 △1.288 ±\pm 0.3211.441 ±\pm 0.1920.724 ±\pm 0.508 ∘
HybridAL ( Δ\Delta Acc)0.457 ±\pm 0.079 ‡∙∘⋄△0.433 ±\pm 0.0230.156 ±\pm 0.0080.393 ±\pm 0.032 △1.163 ±\pm 0.2461.490 ±\pm 0.1750.682 ±\pm 0.498 ‡∙∘⋄△
Appendix
Table 6: Full per-(method ×\times backbone ×\times dataset) test NLL (lower is better). Each cell reports mean ±\pm std across 5 seeds. Cell shading is per-column within each backbone (greener == lower == better). The Mean column averages across the 30 (dataset, seed) cells per (method, backbone). Superscripts indicate that the cell is significantly lower than the reference method (paired tt -test, p<0.05p<0.05 , two-sided): †\dagger Retrain , ‡\ddagger FineTune , \lx@sectionsign\lx@sectionsign NewOnly , \lx@paragraphsign\lx@paragraphsign HybridAL ( Δα\Delta\alpha ), ⋆\star HybridAL ( Δ\Delta Acc), ∙\bullet FixedSwitch (3), ∘\circ FixedSwitch (5), ⋄\diamond FixedSwitch (7), △\triangle FixedSwitch (10).
MethodIMDbAGNewsJigsawSST-2TwtEvYahooMean
DistilBERT
Retrain526 ±\pm 19690 ±\pm 33719 ±\pm 41834 ±\pm 39886 ±\pm 221375 ±\pm 44838 ±\pm 272
FineTune367 ±\pm 11 †¶⋆⋄△407 ±\pm 17 †¶⋆∘⋄△459 ±\pm 12 †⋆⋄△621 ±\pm 14 †¶⋆∘⋄△564 ±\pm 13 †⋆∘⋄△675 ±\pm 19 †¶⋆∘⋄△515 ±\pm 115 †¶⋆∘⋄△
NewOnly153 ±\pm 6 †‡¶⋆∙∘⋄△324 ±\pm 15 †‡¶⋆∙∘⋄△374 ±\pm 13 †‡¶⋆∙∘⋄△122 ±\pm 3 †‡¶⋆∙∘⋄△94 ±\pm 7 †‡¶⋆∙∘⋄△162 ±\pm 12 †‡¶⋆∙∘⋄△205 ±\pm 107 †‡¶⋆∙∘⋄△
HybridAL ( Δα\Delta\alpha )478 ±\pm 84564 ±\pm 102572 ±\pm 115 †780 ±\pm 97687 ±\pm 126 †1061 ±\pm 258 †690 ±\pm 234 †
HybridAL ( Δ\Delta Acc)510 ±\pm 70512 ±\pm 42 †523 ±\pm 21 †727 ±\pm 84 †765 ±\pm 1021000 ±\pm 189 †673 ±\pm 204 †
Appendix
Table 7: Full per-(method ×\times backbone ×\times dataset) training time in seconds (lower is better). Each cell reports mean ±\pm std across 5 seeds. Cell shading is per-column within each backbone (greener == lower == better). The Mean column averages across the 30 (dataset, seed) cells per (method, backbone). Superscripts indicate that the cell is significantly lower than the reference method (paired tt -test, p<0.05p<0.05 , two-sided): †\dagger Retrain , ‡\ddagger FineTune , \lx@sectionsign\lx@sectionsign NewOnly , \lx@paragraphsign\lx@paragraphsign HybridAL ( Δα\Delta\alpha ), ⋆\star HybridAL ( Δ\Delta Acc), ∙\bullet FixedSwitch (3), ∘\circ FixedSwitch (5), ⋄\diamond FixedSwitch (7), △\triangle FixedSwitch (10).
Figure 5: Per-round Δα\Delta\alpha trajectories ( ε=10−4\varepsilon{=}10^{-4} , k=3k{=}3 ) for four representative runs. Dashed line marks ε\varepsilon ; the marker marks t⋆t^{\star} .
Figure 6: Per-round Δ\Delta Acc trajectories ( ε=5×10−3\varepsilon{=}5{\times}10^{-3} , k=2k{=}2 ) for the same four cases as Figure 5 .
Figure 7: Distribution of below- ε\varepsilon stretch lengths, 9090 runs per signal. Stretches shorter than kk are filtered by the patience parameter and do not trigger a switch.
 Δα\Delta\alpha ( ε=10−4\varepsilon{=}10^{-4} )Δ\Delta Acc ( ε=5×10−3\varepsilon{=}5{\times}10^{-3} )
Datasett ⁣⋆t^{\!\star}max post% >ε{>}\varepsilont ⁣⋆t^{\!\star}max post% >ε{>}\varepsilon
IMDb8.44.2e​-​44317.83.8e​-​283
AG News10.74.4e​-​4468.02.5e​-​254
Jigsaw6.92.9e​-​4319.01.6e​-​236
SST-28.44.6e​-​43310.83.9e​-​267
TweetEval6.34.5e​-​44112.56.7e​-​280
Appendix
Table 8: Post-switch signal dynamics (3 backbones ×\times 5 seeds; switched cells only). t ⁣⋆t^{\!\star} : mean switch round averaged unweightedly across the 3 backbones; max post: largest post-switch signal value pooled across all switched cells; % >ε{>}\varepsilon : fraction of post-switch rounds above threshold.
 HybridAL ( Δα\Delta\alpha )HybridAL ( Δ\Delta Acc)
DatasetDistilBERTBERTRoBERTaMeanDistilBERTBERTRoBERTaMean
IMDb12.38.54.58.415.523.014.817.8
AG News11.38.812.010.78.68.07.48.0
Jigsaw11.25.44.06.97.49.610.09.0
SST-27.59.68.08.49.89.013.510.8
TweetEval7.56.64.76.314.0✗11.012.5
Appendix
Table 9: Mean empirical switch round t⋆t^{\star} per (method, backbone, dataset), averaged across the seeds that fired. ✗: no seed switched within T=25T{=}25 rounds; such cells are excluded from the means.
Methodτ\tauNLLNLL ( τ\tau )ECEECE ( τ\tau )
Retrain0.970.5320.5175.4%2.6%
FineTune2.290.7770.53313.7%2.5%
NewOnly1.200.5960.5608.0%3.3%
HybridAL ( Δα\Delta\alpha )1.970.7220.53212.1%2.8%
HybridAL ( Δ\Delta Acc)1.800.6820.52511.8%3.0%
Appendix
Table 10: Temperature scaling on DistilBERT (6 datasets ×\times 5 seeds). τ\tau : fitted temperature; NLL( τ\tau ), ECE( τ\tau ): post-scaling values.
Dataset∣Vsig∣|\mathcal{V}_{\text{sig}}|Fire ratet⋆t^{\star}Test F1
AG News25100%8.08.0 ±\pm 2.62.60.90100.9010 ±\pm 0.00240.0024
50100%10.010.0 ±\pm 8.98.90.90440.9044 ±\pm 0.00100.0010
10067%10.510.5 ±\pm 9.29.20.90220.9022 ±\pm 0.00010.0001
20033%11110.90040.9004 ±\pm 0.00610.0061
full ( 1,2001{,}200 )100%7.77.7 ±\pm 2.52.50.90410.9041 ±\pm 0.00650.0065
IMDb2533%11110.82360.8236 ±\pm 0.00860.0086
Appendix
Table 11: Validation-size sensitivity for HybridAL ( Δ\Delta Acc) on DistilBERT, 3 seeds per cell. t⋆t^{\star} is averaged over firing seeds only; single values indicate one firing seed.
Comparison vs. RetrainPre-switchPost-switch
HybridAL ( Δα\Delta\alpha )1.00001.0000 ±\pm 0.00000.00000.01740.0174 ±\pm 0.04270.0427
HybridAL ( Δ\Delta Acc)0.99920.9992 ±\pm 0.00710.00710.01970.0197 ±\pm 0.05150.0515
Appendix
Table 12: Jaccard overlap of acquired batches with the paired Retrain run, pooled over 6 (dataset, seed) cells.
SignalDatasetPre-switchPost-switch
Δα\Delta\alphaIMDb(47.6,52.4)(47.6,52.4)(51.4,48.6)(51.4,48.6)
Δα\Delta\alphaAG News(29.0,11.9,33.1,26.1)(29.0,11.9,33.1,26.1)(25.0,9.8,31.1,34.1)(25.0,9.8,31.1,34.1)
Δ\Delta AccIMDb(52.5,47.5)(52.5,47.5)(51.5,48.5)(51.5,48.5)
Δ\Delta AccAG News(30.6,13.5,29.2,26.7)(30.6,13.5,29.2,26.7)(24.6,11.0,32.8,31.5)(24.6,11.0,32.8,31.5)
Appendix
Table 13: Class composition of acquired batches, in percent. IMDb: (negative, positive). AG News: (World, Sports, Business, Sci/Tech).
MethodPre-switchPost-switchChange
HybridAL ( Δα\Delta\alpha )0.84550.8455 ±\pm 0.16650.16650.75470.7547 ±\pm 0.09070.0907−0.0909-0.0909
HybridAL ( Δ\Delta Acc)0.83920.8392 ±\pm 0.18350.18350.76240.7624 ±\pm 0.08840.0884−0.0768-0.0768
Retrain0.84190.8419 ±\pm 0.16170.1617——
Appendix
Table 14: Mean predictive entropy of the acquired batch. Retrain never switches, so its trajectory is reported as a single reference value.
RoundRetrainΔα\Delta\alphaΔ\Delta Acc
56.096.09 ±\pm 2.442.445.905.90 ±\pm 2.472.476.586.58 ±\pm 2.852.85
105.285.28 ±\pm 2.372.377.277.27 ±\pm 3.353.357.607.60 ±\pm 4.234.23
154.284.28 ±\pm 1.311.316.436.43 ±\pm 2.912.917.157.15 ±\pm 4.954.95
205.435.43 ±\pm 4.014.017.367.36 ±\pm 3.953.959.879.87 ±\pm 4.274.27
244.704.70 ±\pm 1.031.038.478.47 ±\pm 4.014.0110.0610.06 ±\pm 4.114.11
Last 55.115.11 ±\pm 3.083.088.018.01 ±\pm 3.763.7610.1710.17 ±\pm 3.873.87
Appendix
Table 15: Validation ECE (%, 15-bin) by round, mean over 6 (dataset, seed) cells.
Figure 8: Spearman correlation between the eight candidate switching signals, n=720n{=}720 (run ×\times round) observations from Retrain DistilBERT runs. Performance signals ( Δ\Delta F1, Δ\Delta Acc, Δ\Delta Loss) cluster at ρ≤0.79\rho{\leq}0.79 ; Δα\Delta\alpha is orthogonal to every other signal, motivating its selection as the second HybridAL variant alongside Δ\Delta Acc.
Figure 9: Parallel-coordinates profile of the eight switching signals on five normalised axes (higher = better); Retrain and FineTune (dashed) are references. Δα\Delta\alpha and Δ\Delta Acc dominate on overall rank, motivating their selection as HybridAL variants.
Figure 10: Per-round mean validation F1 averaged across all six datasets (equal weight per dataset); ⋆\star marks each signal’s mean switch round across the dataset/seed firings.
Test F1F_{1} ↑\uparrow
SignalIMDbAGNewsJigsawSST-2TwtEvYahooMean
Retrain0.819 ±\pm 0.0210.900 ±\pm 0.0040.881 ±\pm 0.0030.835 ±\pm 0.0210.634 ±\pm 0.0230.667 ±\pm 0.0040.789 ±\pm 0.105
Δ\Delta Acc0.825 ±\pm 0.0090.904 ±\pm 0.0030.881 ±\pm 0.0070.847 ±\pm 0.0230.633 ±\pm 0.0090.657 ±\pm 0.0060.791 ±\pm 0.109
Δα\Delta\alpha0.826 ±\pm 0.0060.901 ±\pm 0.0050.880 ±\pm 0.0070.861 ±\pm 0.0100.633 ±\pm 0.0120.657 ±\pm 0.0040.793 ±\pm 0.109
Δ\Delta F10.830 ±\pm 0.0110.904 ±\pm 0.0030.880 ±\pm 0.0050.847 ±\pm 0.0230.628 ±\pm 0.0110.656 ±\pm 0.0060.791 ±\pm 0.111
Δ\Delta Loss0.804 ±\pm 0.0220.901 ±\pm 0.0050.883 ±\pm 0.0050.853 ±\pm 0.0200.636 ±\pm 0.0060.662 ±\pm 0.0050.790 ±\pm 0.107
Appendix
Table 16: Per-dataset signal-ablation on DistilBERT (5 seeds per cell; all HybridAL signals at ε=0.5\varepsilon{=}0.5 , k=3k{=}3 ). Four sub-tables stack the per-(signal ×\times dataset) means of test F1, test NLL, total training time, and switching rate (fraction of runs in which the signal triggered a switch within 25 rounds). The final Mean column averages across the six datasets. Cell shading is per column; darker green = better.
Figure 11: Hyperparameter cross-sections through (ε ⁣∗,k ⁣∗)(\varepsilon^{\!*},k^{\!*}) for Δα\Delta\alpha (top row, ε ⁣∗=10−4\varepsilon^{\!*}{=}10^{-4} , k ⁣∗=3k^{\!*}{=}3 ) and Δ\Delta Acc (bottom row, ε ⁣∗=5×10−3\varepsilon^{\!*}{=}5{\times}10^{-3} , k ⁣∗=2k^{\!*}{=}2 ). In each row, the leftmost two panels sweep kk (with ε=ε ⁣∗\varepsilon{=}\varepsilon^{\!*} ) and ε\varepsilon (with k=k ⁣∗k{=}k^{\!*} ) for mean val NLL; the rightmost two panels do the same for mean training time. Marker color encodes switch rate; the chosen cell is marked with ⋆\star .
 mean val F1mean val NLLmean time (s)
ε\varepsilonk=2k{=}2k=3k{=}3k=5k{=}5k=7k{=}7k=2k{=}2k=3k{=}3k=5k{=}5k=7k{=}7k=2k{=}2k=3k{=}3k=5k{=}5k=7k{=}7
Δα\Delta\alpha
5e-50.8580.8570.8550.8550.4470.3850.3700.370290.8318.0331.9322.5
1e-40.8540.8650.8560.8550.4920.4550.3880.370246.3284.8313.7320.8
1.5e-40.8550.8550.8580.8550.4980.5110.4350.370214.3229.4289.3320.9
2e-40.8550.8550.8590.8610.4980.5110.4360.416214.3229.3278.9302.3
Appendix
Table 17: (ε,k)(\varepsilon,k) tuning grid for both signals. Three metrics shown side-by-side: mean val F1, mean val NLL, and mean training time (s); for NLL and time, lower is better. Gray cells mark the chosen (ε ⁣⋆,k ⁣⋆)(\varepsilon^{\!\star},k^{\!\star}) applied to the main results.
ConfigurationMethodAG NewsIMDb
fixed 55 epFineTune0.8920.892 ±\pm 0.0050.0050.8080.808 ±\pm 0.0090.009
fixed 55 epRetrain0.8970.897 ±\pm 0.0040.0040.8220.822 ±\pm 0.0100.010
max 1010 + ESFineTune0.8950.895 ±\pm 0.0040.0040.8180.818 ±\pm 0.0070.007
max 1010 + ESRetrain0.8920.892 ±\pm 0.0070.0070.8210.821 ±\pm 0.0120.012
Appendix
Table 18: Early-stopping spot check, 3 seeds per cell.
Figure 12: Final test F1 vs. entropy acquisition subsample size NN on IMDb (top) and SST-2 (bottom), mean and ±1\pm 1 std across three seeds. Red dotted line: the chosen N=1000N{=}1000 .
Method∣L0∣|\mathcal{L}_{0}|AG NewsIMDbYahoo
Retrain500.8990.899 ±\pm 0.0030.0030.8200.820 ±\pm 0.0070.0070.6580.658 ±\pm 0.0040.004
HybridAL500.9020.902 ±\pm 0.0040.0040.8280.828 ±\pm 0.0080.0080.6540.654 ±\pm 0.0080.008
Retrain1000.8980.898 ±\pm 0.0040.0040.8210.821 ±\pm 0.0150.0150.6580.658 ±\pm 0.0090.009
HybridAL1000.8980.898 ±\pm 0.0040.0040.8260.826 ±\pm 0.0140.0140.6430.643 ±\pm 0.0100.010
Retrain2000.9000.900 ±\pm 0.0040.0040.8190.819 ±\pm 0.0210.0210.6670.667 ±\pm 0.0040.004
HybridAL2000.9020.902 ±\pm 0.0070.0070.8140.814 ±\pm 0.0140.0140.6560.656 ±\pm 0.0040.004 ∗
Appendix
Table 19: Pool size ablation, 5 seeds per cell. ∗ p<0.05p<0.05 vs. Retrain (paired tt -test).
MethodnnAG NewsIMDbYahoo
Retrain160.8960.896 ±\pm 0.0050.0050.8090.809 ±\pm 0.0090.0090.6460.646 ±\pm 0.0130.013
HybridAL160.8950.895 ±\pm 0.0060.0060.8160.816 ±\pm 0.0110.0110.6510.651 ±\pm 0.0080.008
Retrain320.9000.900 ±\pm 0.0040.0040.8190.819 ±\pm 0.0210.0210.6670.667 ±\pm 0.0040.004
HybridAL320.9020.902 ±\pm 0.0070.0070.8140.814 ±\pm 0.0140.0140.6560.656 ±\pm 0.0040.004 ∗
Retrain640.9120.912 ±\pm 0.0030.0030.8360.836 ±\pm 0.0080.0080.6790.679 ±\pm 0.0030.003
HybridAL640.9110.911 ±\pm 0.0030.0030.8340.834 ±\pm 0.0050.0050.6680.668 ±\pm 0.0090.009
Appendix
Table 20: Batch size ablation, 5 seeds per cell. ∗ p<0.05p<0.05 vs. Retrain (paired tt -test).
SamplerMethodAG NewsIMDbYahoo
EntropyRetrain0.9000.900 ±\pm 0.0040.0040.8190.819 ±\pm 0.0210.0210.6670.667 ±\pm 0.0040.004
EntropyHybridAL0.9020.902 ±\pm 0.0070.0070.8140.814 ±\pm 0.0140.0140.6560.656 ±\pm 0.0040.004 ∗
RandomRetrain0.8870.887 ±\pm 0.0060.0060.8250.825 ±\pm 0.0070.0070.6620.662 ±\pm 0.0090.009
RandomHybridAL0.8860.886 ±\pm 0.0070.0070.8200.820 ±\pm 0.0060.0060.6510.651 ±\pm 0.0140.014
BADGERetrain0.8970.897 ±\pm 0.0040.0040.8280.828 ±\pm 0.0070.0070.6680.668 ±\pm 0.0040.004
BADGEHybridAL0.8980.898 ±\pm 0.0040.0040.8330.833 ±\pm 0.0030.0030.6510.651 ±\pm 0.0070.007 ∗
Appendix
Table 21: Sampler ablation, 5 seeds per cell. ∗ p<0.05p<0.05 vs. Retrain (paired tt -test).
More in this topic

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 296 papers

Explore similar work

CardsList

  1. A Mechanism-Driven Theory of Phase Transitions in Active Learning

    Jun 30, 2026Julia Machnio, Mads Nielsen, Mostafa Mehdipour GhaziActive LearningStatistical Learning Theory

    Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark

  2. CAAL: Contextual Bandits based Online Hand-Craft Active Learning Strategy Selection

    Jun 6, 2026Shao-An Yin, Jiacong Li, Tianpei Xie +3Active LearningContextual Bandits

    Department of Electrical and Computer Engineering, University of Minnesota, Twin Cities, USA · Jiacong Li, Tianpei Xie, Cecile Levasseur, and Wojciech Kowalinski are with Amazon.

  3. GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training

    May 25, 2026Yuelin Hu, Wei Liu, Zhenbo Yu +2Supervised Fine-TuningAdaptive Loss Weighting

    Shanghai Jiao Tong University · Shanghai Maritime University