Adaptive Active Learning for Regression via Reinforcement Learning

Simon D. Nguyen; Troy Russo; Kentaro Hoffman; Tyler H. McCormick

stat.MLMar 11, 2026

Adaptive Active Learning for Regression via Reinforcement Learning

Authors: Simon D. Nguyen, Troy Russo, Kentaro Hoffman, Tyler H. McCormick

Organizations: Department of Statistics, University of Washington · Department of Sociology, University of Washington

Abstract

Active learning for regression reduces labeling costs by selecting the most informative samples. Improved Greedy Sampling is a prominent method that balances feature-space diversity and output-space uncertainty using a static, multiplicative rule. We propose Weighted improved Greedy Sampling (WiGS), which replaces this framework with a dynamic, additive criterion. We formulate weight selection as a reinforcement learning problem, enabling an agent to adapt the exploration-investigation balance throughout learning. Experiments on 18 benchmark datasets and a synthetic environment show WiGS outperforms iGS and other baseline methods in both accuracy and labeling efficiency, particularly in domains with irregular data density where the baseline's multiplicative rule ignores high-error samples in dense regions.

Figures & tables

Method	Weight $w_{x}^{(t)}$	Dynamic	Adaptive
Random Sampling	NA	$\times$	$\times$
GSx	1	$\times$	$\times$
GSy	0	$\times$	$\times$
iGS	NA	$\times$	$\times$
WiGS - Static Weight (investigation focus)	$<0.5$	$\times$	$\times$
WiGS - Static Weight (exploration focus)	$>0.5$	$\times$	$\times$

Table 1: Summary of baseline and proposed methods.

w_{x}^{(t)}

denotes the weight assigned to the feature-space distance at iteration

t

in Equation 5 . We distinguish between Dynamic strategies (time-varying based on a fixed schedule) and Adaptive strategies (state-dependent based on learned feedback from the data).

Figure 1: Visualization of the synthetic dataset ( $N=1000$ ). The high-noise region (trap) at $x\approx 0.85$ coincides with high data density.

Panel of Figure 2 — Figure 2: Performance difference relative to iGS (red line). Values below zero indicate superior performance. The adaptive WiGS-SAC agent and exploration-focused static variants (bottom cluster) consistently outperform the baseline, reducing absolute error by up to 0.05.

Figure 3: Global Performance Heatmap. Values represent the ratio of the Area Under the RMSE Curve (AUC) for each method relative to the iGS baseline. Blue cells ( $<1.0$ ) indicate superior performance (lower cumulative error), while red cells ( $>1.0$ ) indicate inferior performance. Notably, the WiGS methods (bottom rows) demonstrate consistent robustness across diverse domains.

Figure 4: Relative Label Efficiency ( $N_{rel}$ ) aggregated across 20 datasets. The distribution represents the labeling budget required to achieve $70\%$ (blue) and $80\%$ (green) of the total possible performance gain. Values to the left ( $<1.0$ ) indicate superior efficiency. The adaptive WiGS agents (SAC, MAB) not only require fewer labels to reach these milestones than the iGS baseline, but also exhibit narrower variance, highlighting their robust generalization across different datasets.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Optimizer	Adam
Learning Rate ( $\alpha_{lr}$ )	$3\cdot 10^{-4}$
Discount Factor ( $\gamma$ )	0.99
Replay Buffer Size ( $\mathcal{B}$ )	10,000
Batch Size	64
Soft Update Rate ( $\tau$ )	0.005

Appendix

Table 2: Hyperparameters for the WiGS-SAC agent.

Panel of Figure 5 — Figure 5: Visualization of the synthetic datasets. Shows $N=1000$ sample data points (dots) and the true underlying function $f(x)$ (solid line). Shaded regions indicate areas of high noise that overlap with dense data regions from the GMM, creating strategic conflicts between exploration and investigation.

Panel of Figure 6 — Figure 6: Spatial visualization of the learned weighting policy. Each point represents a selected sample, colored by the average weight $w_{x}$ assigned to it by the WiGS-SAC agent. Red indicates high exploration focus; Blue indicates high investigation focus. The agent learns to decouple the strategy, applying exploration to complex oscillations (vertices) and investigation to linear regions.

Panel of Figure 7 — Figure 7: Average weight trend for WiGS-SAC on the Two-Regime and Three-Regime dataset. The solid line shows the mean weight across 100 seeds, while the shaded region represents $\pm 1$ standard deviation. The persistent high variance demonstrates that the agent remains reactive throughout the learning process, rather than converging to a static heuristic.

Table 3: Statistical significance of performance differences on synthetic datasets. Tables display p-values from pairwise Wilcoxon Signed-Rank Tests comparing Full-Pool RMSE traces across 100 independent trials. A p-value

<0.05

indicates a statistically significant difference in performance distributions.

no.	Dataset	Source	Size	Features
1	AutoMPG	UCI ML Repository	392	8
2	Beer	Kaggle	365	5
3	Body Fat	Kaggle	252	13
4	Burbidge - Correct	Burbidge et al. [2007]	1000	1
5	Burbidge - Misspecified	Burbidge et al. [2007]	1000	1
6	Burbidge - Low Noise	Burbidge et al. [2007]	1000	1

Appendix

Table 4: Benchmark datasets used in the experiments.

Panel of Figure 8 — Figure 8: Full-pool RMSE trace plots for benchmark datasets (Part 1 of 2).

Panel of Figure 9 — Figure 9: Full-pool RMSE trace plots for benchmark datasets (Part 2 of 2).

	Baselines				Advanced Baselines				WiGS (Static/Decay)				WiGS (Adaptive)
Dataset	Rand.	GSx	GSy	iGS	QBC	Uncert.	EMCM	EGAL	0.25	0.75	Lin.	Exp.	MAB(2)	SAC
Beer	11.1	11.5	11.8	12.1	17.1	11.3	10.9	11.9	12.5	12.5	12.4	12.2	12.0	225.0
Bodyfat	6.8	7.0	7.2	7.3	10.4	6.9	6.9	7.1	7.3	7.3	7.3	7.3	7.2	205.8
Burb. Correct	25.9	27.9	28.9	31.7	39.7	27.0	26.4	37.2	36.6	36.6	36.7	36.7	36.5	234.6
Burb. Low Noise	25.3	27.1	28.1	31.3	39.0	26.2	25.3	34.2	36.2	36.4	36.4	36.3	35.7	448.5
Burb. Misspecified	25.0	26.8	27.7	30.9	39.1	26.2	25.5	34.1	35.8	35.9	35.9	35.7	35.4	461.2

Appendix

Table 5: Median runtime (seconds) across simulation seeds. ‘Uncert.’ denotes Uncertainty Sampling.

Figure 10: Random Forest Results (Heatmap): Values represent the ratio of the Area Under the RMSE Curve (AUC) relative to the iGS baseline using a Random Forest Regressor. Blue cells ( $<1.0$ ) indicate superior performance.

Figure 11: Random Forest Results (Efficiency): Relative Label Efficiency ( $N_{rel}$ ) aggregated across 20 datasets.

Panel of Figure 12 — Figure 12: Full-pool Random Forest RMSE trace plots for benchmark datasets (Part 1 of 2).

Panel of Figure 13 — Figure 13: Full-pool Random Forest RMSE trace plots for benchmark datasets (Part 2 of 2).

Panel of Figure 14 — Figure 14: Full-pool Correlation Coefficients trace plots for benchmark datasets (Part 1 of 2).

Panel of Figure 15 — Figure 15: Full-pool Correlation Coefficients trace plots for benchmark datasets (Part 2 of 2).

Panel of Figure 16 — Figure 16: Variance of the full-pool RMSE trace plots for benchmark datasets (Part 1 of 2).

Panel of Figure 17 — Figure 17: Variance of the full-pool RMSE trace plots for benchmark datasets (Part 2 of 2).

Figure 18: Robust Relative AUC values comparing active learning strategies under robust feature scaling. The results parallel those observed under standard normalization, indicating the method’s robustness to preprocessing techniques.

Figure 19: Relative Label Efficiency ( $N_{rel}$ ) aggregated across 20 datasets with a robust normalization. Again, the results are similar to those under standard normalization

Panel of Figure 20 — Figure 20: Full-pool RMSE trace plots for benchmark datasets with robust normalization (Part 1 of 2).

Panel of Figure 21 — Figure 21: Full-pool RMSE trace plots for benchmark datasets with robust normalization (Part 2 of 2).

Active Learning

Active learning selects the examples, labels, features, or experiments most useful for improving a model under limited budgets.

See all 280 papers

Feature Weighting Improves Pool-Based Sequential Active Learning for Regression
Apr 2, 2026Dongrui WuActive Learning
Ministry of Education Key Laboratory of Image Processing and Intelligent Control, and Hubei Key Laboratory of Brain-inspired Intelligent Systems, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074 China.
Batch Bayesian Active Learning with Partial Batch Label Sampling
Oct 10, 2025Kangping Hu, Stephen MussmannActive Learning
Department of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
Dec 14, 2025Pouya Ahadi, Blair Winograd, Camille Zaug +3Human-in-the-Loop Annotation Active Learning
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA · Ford Motor Company

Apr 2, 2026J/K move · Enter open · S save

Feature Weighting Improves Pool-Based Sequential Active Learning for Regression

Dongrui Wu

Ministry of Education Key Laboratory of Image Processing and Intelligent Control, and Hubei Key Laboratory of Brain-inspired Intelligent Systems, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074 China.

Active Learning

Open paper AskPDF arXiv

Pool-based sequential active learning for regression (ALR) optimally selects a small number of samples sequentially from a large pool of unlabeled samples to label, so that a more accurate regression model can be constructed under a given labeling budget. Representativeness and diversity, which involve computing the distances among different samples, are important considerations in ALR. However, previous ALR approaches do not incorporate the importance of different features in inter-sample distance computation, resulting in inaccurate distances and hence sub-optimal sample selection. This paper proposes four feature weighted single-task ALR approaches and three feature weighted multi-task ALR approaches, where the ridge regression coefficients trained from a small amount of previously labeled samples are used to weight the corresponding features in inter-sample distance computation. Extensive experiments showed that this intuitive and easy-to-implement enhancement almost always improves the performance of five existing ALR approaches, in both single-task and multi-task regression problems. The feature weighting strategy may also be easily extended to stream-based ALR, and classification algorithms.

Abstract

Active Learning

Explore similar work

Feature Weighting Improves Pool-Based Sequential Active Learning for Regression

Batch Bayesian Active Learning with Partial Batch Label Sampling

Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection