Active Feature Acquisition With Incomplete Training Data

Reza Rezvan; Valter Schütz; Han Wu; Linus Aronsson; Morteza Haghir Chehreghani

cs.LGSep 26, 2026

Active Feature Acquisition With Incomplete Training Data

Authors: Reza Rezvan, Valter Schütz, Han Wu, Linus Aronsson, Morteza Haghir Chehreghani

Organizations: Department of Computer Science and Engineering Chalmers University of Technology & University of Gothenburg

Abstract

In many prediction tasks, acquiring all features can be a prohibitively expensive or outright impossible task. Further, in many cases a static subset of features may not be enough to solve the problem sufficiently across various instances. Active Feature Acquisition (AFA) addresses these problems by formalizing the trade-off between feature cost and predictive performance during sequential feature selection. However, prior AFA work largely assumes access to complete training data, an assumption that is often violated in practice. Here we study AFA with Incomplete Training Data (AFA-ITD), showing that under missing completely at random (MCAR) data, one-step acquisition values remain unchanged, whereas multi-step values can decrease. We analyze three approaches to learning from incomplete data: aliasing, filtering, and generative restoration. We show that filtering can require a number of training instances scaling exponentially with the dimension, whereas generative restoration scales exponentially with the acquisition budget. We empirically test our theory on a controlled experiment and across common AFA datasets and find that missingness mainly damages methods that exploit multi-step acquisitions and that generative restoration is able to recover lost performance in many experiments. Code is available at https://github.com/Linusaronsson/AFA-Benchmark/tree/missing-data.

Figures & tables

Panel of Figure 1 — Figure 1: The shortcut problem for a fixed missingness mask $m$ where the branch features $x_{2}$ and $x_{3}$ are missing (a) , and the unchanged one-step and differing two-step $1+Q_{k}$ optimal values for both training and evaluation (b) .

Panel of Figure 2 — Figure 2: The usable training instances for each of the approaches, on Example 1 under MCAR with $p=0.3$ .

Figure 3: Evaluation regret of the myopic

\widehat{Q}_{1}

(gray, dashed) and full planning-horizon

\widehat{Q}_{b}

(solid) methods, across the three learning approaches swept over the dimension

d

(top row) and the acquisition budget

b

(bottom row), under MCAR at

p=0.6

, with 95% confidence bands.

Figure 4: Evaluation regret of the full planning-horizon

\widehat{Q}_{b}

policies of filtering (a) and generative restoration (b) in the dimension sweep at

b=2

, against the effective number of training instances

n(1-p)^{d}

and

n(1-p)^{b}

, respectively, for

d\in\{6,8,10\}

and

p\in\{0.3,0.5,0.7\}

Figure 5: Restricted training vs. generative restoration under MCAR at

p=0.7

. Arrows point from restricted training to generative restoration, and bars are 95% bootstrap intervals over five splits. Methods and datasets are ordered by descending mean missingness damage. Gray rows denote myopic methods ( DIME , GDFS ), hatched rows denote non-myopic methods with intermediate predictive rewards ( OL , ODIN ), and white rows denote non-myopic acquisition with a terminal prediction objective ( AACO , JAFA ).

$V_{\mathrm{static}}$		$\Delta_{\mathrm{route}}$		$\omega_{\mathrm{route}}$
MiniBooNE	$0.894\pm 0.004$	CUBE	$0.245\pm 0.010$	NHANES	$0.669\pm 0.000$
CUBE-NUC	$0.847\pm 0.019$	ACTG175	$0.175\pm 0.012$	CUBE	$0.342\pm 0.000$
ACTG175	$0.846\pm 0.012$	CUBE-NUC	$0.174\pm 0.017$	ACTG175	$0.287\pm 0.001$
CUBE	$0.824\pm 0.010$	Diabetes	$0.166\pm 0.001$	Heart disease	$0.264\pm 0.003$
NHANES	$0.731\pm 0.005$	Heart disease	$0.127\pm 0.015$	CUBE-NUC	$0.221\pm 0.000$
Heart disease	$0.720\pm 0.019$	CUBE-NM	$0.059\pm 0.010$	Diabetes	$0.205\pm 0.000$

Table 1: Fixed-route structure. Each column ranks the datasets by that quantity alone, largest first, so the three rankings are independent. Mean

\pm

standard error over five dataset splits. Static references and route weights are selected on training data;

V_{\mathrm{static}}

and

\Delta_{\mathrm{route}}

are reported on validation data. Each dataset is at its largest evaluation budget

b

, which is 15 for Diabetes and MiniBooNE, 14 for CUBE-NM, 10 for ACTG175, CUBE and NHANES, 7 for CUBE-NUC and Heart disease.

Method	Complete	Margin over $V_{\mathrm{static}}$	Damage
DIME	$0.705$	$-0.027$	$+0.002$
ODIN	$0.667$	$-0.065$	$+0.009$
GDFS	$0.745$	$+0.013$	$+0.059$
AACO	$0.758$	$+0.026$	$+0.062$
JAFA	$0.707$	$-0.026$	$+0.072$
OL	$0.745$	$+0.013$	$+0.092$

Table 2: Complete-data score, its margin over the static route

V_{\mathrm{static}}

( Table 1 ), and missingness damage under MCAR at

p=0.7

, averaged over the eight datasets and, for JAFA , OL and ODIN , over both state representations. Across method–dataset cells, Pearson

r=0.59

and Spearman

\rho=0.34

between margin and damage.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Predictive performance against wall-clock time per trained method. Each arrow points from restricted training to generative restoration for the same dataset and method.

Figure 7: Restricted training against generative restoration at

p=0.7

under all four missingness mechanisms.

Figure 8: Restoration gain against missingness damage, one panel per method and one block per mechanism, over all eight datasets and three rates.

Active Feature Acquisition

Active Feature Acquisition studies how models can selectively obtain costly or missing inputs to improve predictions within limited budgets.

See all 19 papers

BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisition
Aug 3, 2026Jiaorong Feng, Qian Li, Ying LiCost-Aware Inference Active Learning
Curtin Business School, Curtin University, Perth, Western Australia, Australia · †Present affiliation: Independent Researcher. · School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Western Australia, Australia
Explanations-Driven Active Feature Acquisition for Algorithmic Recourse
Sep 10, 2026Vinura Galwaduge, Jagath SamarabanduCounterfactual Explanations Cost-Aware Inference
Dept. of Electrical and Computer Engineering Western University London, ON, Canada
The Lagrangian Structure of Cost-Penalized Active Feature Acquisition
Oct 15, 2026Joseph BinghamActive Feature Acquisition Constrained Optimization
Faculty of Computer Science Faculty of Biology Technion – Israel Institute of Technology Haifa 3200003, Israel

Aug 3, 2026J/K move · Enter open · S save

BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisition

Jiaorong Feng, Qian Li, Ying Li

Curtin Business School, Curtin University, Perth, Western Australia, Australia · †Present affiliation: Independent Researcher. · School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Western Australia, Australia

Cost-Aware Inference Active Learning Active Feature Acquisition Classification

Open paper AskPDF arXiv

Active feature acquisition (AFA) asks which unobserved feature to measure next for each test instance under a budget. Greedy rules are easy to train but can overlook context features whose value is realized only through later acquisitions, while reinforcement-learning and generative approaches introduce difficult optimization or conditional-density estimation. We introduce \method, a deployable, supervised alternative that learns a separate candidate-conditioned risk-to-go function for every remaining budget. Starting from the one-step terminal classification risk, the functions are fitted backward with Bellman targets; inference greedily minimizes the learned terminal risk using only observed values, the mask, candidate identity, and remaining budget. A controlled non-myopic benchmark shows the expected mechanism: at budgets two and three, \method improves accuracy over its one-step ablation by

4.84\pm2.17

and

4.39\pm1.10

percentage points (mean

\pm

standard error over five seeds). On Fashion-MNIST with 20 candidate pixels, it improves accuracy at every nontrivial reported budget on average, including

10.20\pm0.74

points at four acquisitions; its mean paired gain across budgets

\{2,4,8,12,16\}

3.50\pm0.37

points. A three-seed MiniBooNE study is mixed at small budgets but positive at 8 and 16 acquisitions, identifying a current boundary rather than supporting a universal claim. These results establish a reproducible mechanism-level case for direct Bellman risk regression and delimit the experiments still needed for state-of-the-art comparison.

Abstract

Active Feature Acquisition

Explore similar work

BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisition

Explanations-Driven Active Feature Acquisition for Algorithmic Recourse

The Lagrangian Structure of Cost-Penalized Active Feature Acquisition