HALO: Hierarchical Evidence Selection and Reinforcement Learning for Allocating Budgeted Label Optimization from Longitudinal Clinical Records
Authors: Huaiyuan Ying, Feiqing Huang, Ryumei Nakada, Katherine P. Liao, Tianxi Cai
Organizations: Department of Statistics and Data Science Tsinghua University Beijing, China · Department of Biostatistics Harvard T.H. Chan School of Public Health Boston, MA, USA · Brigham and Women’s Hospital Harvard Medical School Boston, MA, USA · Harvard T.H. Chan School of Public Health Harvard Medical School Boston, MA, USA
Learning from extensive longitudinal clinical records--spanning repeated encounters, evolving conditions, and heterogeneous documentation--under a finite labeling budget requires solving two coupled problems: deciding what evidence to keep and deciding which patients deserve higher-fidelity labels. Prior work on active learning and multi-fidelity label acquisition mainly addresses the second problem, while treating the evidence presented to the acquisition policy as fixed. We present HALO, a Hierarchical Evidence Selection and Reinforcement Learning pipeline for Allocating Budgeted Label Optimization from Longitudinal Clinical Records. This framework jointly learns what to read and what label source to acquire under budget. HALO first selects informative evidence from longitudinal records and then uses a budget-aware policy to allocate free-, silver-, and gold-fidelity label sources. For frozen policies, our theory characterizes asymmetric acquired-label channels and gives a budget-constrained finite-sample gold-risk bound for a common downstream logistic model. In a heart-failure case study, HALO attained the highest MedAlign acquired-label F1 at six of eight target budgets. A downstream classifier trained on HALO-acquired labels attained the highest external MGB F1 at budgets 15-60 and remained within 0.01 of the best comparator at budgets 75-120.
Figures & tables
Figure 1: Overview of HALO. Hierarchical evidence selection first retains informative notes using a frozen note scorer and a fixed threshold τ1 . A trainable segment-informativeness model with parameters ξ then samples segment-retention decisions during PPO training; at inference, segments are retained when their predicted retention probabilities exceed τ2=0.5 . The retained evidence is summarized into a compact patient representation. Budgeted label allocation feeds the compact summary to a PPO-updated patient-worthiness scorer, implemented with a T5-scale encoder and ordinal head, which routes each patient to free-, silver-, or gold-fidelity label tiers under a finite labeling budget. The design objective is to reserve expensive labels for patients for whom the policy predicts higher fidelity to be most valuable.
Figure 2: Performance across target label-acquisition budgets. (a) MedAlign acquired-label F1 measures agreement between each strategy’s acquired labels and the GPT-5-derived proxy reference on 106 patients. (b) MGB external downstream F1 evaluates a penalized logistic regression trained on each strategy’s acquired MedAlign labels against independently curated clinician chart-review labels on 299 MGB patients. Each plotted value is the mean F1 across ten repeated runs. Label-source costs are c(f)=0 , c(s)=0.6 , and c(g)=3 . Higher values are better in both panels. Lines connect the eight evaluated target-budget settings.
Active learning (AL) promises to reduce the cost of medical imaging projects by lowering the number of clinical labels required. However, practical deployment requires committing to a sampling strategy before the full annotation budget is spent, and choosing the wrong strategy can increase rather than decrease costs. We propose Active-Learning Deployment Advisor (ALDA), a deployment-oriented framework for AL method selection under clinical performance constraints. Given a short pilot phase, ALDA fits a parametric learning-curve model to each candidate strategy, estimates whether that strategy is expected to reach a required clinical performance target, and predicts the number of expert annotations needed to do so. In addition to absolute annotation cost, ALDA introduces a deployment window that quantifies the sensitivity of this cost estimate to uncertainty in the clinical threshold. The final recommendation follows a risk-aware rule: among strategies with near-optimal predicted cost, ALDA prefers the strategy with the narrowest deployment window, the most robust to threshold revisions. Experiments on four medical imaging classification domains show that ALDA predicts the deployment-optimal method from a pilot of 15-30% of the intended budget and reduces annotation costs by up to 82% compared with a poor strategy choice. Rather than introducing a new sampling heuristic, ALDA provides a practical decision layer that answers a deployment-critical question: how many labels are enough?
Julia Machnio, Mads Nielsen, Mostafa Mehdipour Ghazi
Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark
We study how a limited labeling budget should be allocated to minimize multiclass zero-one classification risk. We consider parametric classification problems in which features are observed for all sampling units while class labels can be acquired selectively. By combining the Fisher information supplied by an acquired label with the local geometry of multiclass excess risk, we derive an acquisition criterion that minimizes the leading asymptotic coefficient of expected multiclass excess risk. The resulting rule values a label according to how strongly its information is aligned with parameter directions that perturb the active Bayes decision boundary, rather than according to posterior uncertainty or global parameter information alone. We characterize the oracle acquisition design, establish its threshold structure, and derive face-specific and cost-sensitive extensions. An analytic example shows that posterior uncertainty and classification value can produce different, and even reversed, acquisition rankings. We further develop a two-stage adaptive procedure that attains the oracle leading-risk criterion under regularity conditions and provide explicit results for Gaussian discriminant analysis. Three-class QDA experiments illustrate the resulting acquisition geometry, while an application to the six-class Statlog Landsat Satellite data shows that classification-risk acquisition can differ materially from both uncertainty-based acquisition and the complete-classification-information comparator. The adaptive classification-risk design attains lower mean error than this Fisher comparator across the labeling budgets considered, although it does not uniformly outperform entropy or margin sampling and differences among the targeted strategies become small as the labeling budget increases.
F. Setoudehtanzangi, Geoffrey J. McLachlan
Dipartimento di Scienze Statistiche, Universit`a di Padova, Padova, Italy · School of Mathematics and Physics, The University of Queensland, Australia
Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently. We quantify this requirement for the area under the generalized risk-coverage curve (AUGRC). A prelabel lower bound rules out insufficient budgets. With all labels known, a covering linear program bounds the minimum number of labels sufficient to fix the winner (the certificate size) within K−1 labels for K candidates. For fixed K, independent uniform orders and identical predictions, the prelabel bound approaches one quarter of the pool. With iid Bernoulli errors independent of the orders, every exact acquisition policy reads almost all labels asymptotically, although a two-candidate certificate needs only half. Across 108 feature-panel comparisons on nine datasets, disagreement labels settle every accuracy choice but no AUGRC choice. A 20% budget is ruled out in 96 conditions; certificates need 56-57% on average. On ten conditions with pretrained image classifiers, confidence-score choice reads 68-91% of 10,000 labels for exact selection and 50-67% with AUGRC tolerance 5×10−4. An exact stopping test works with any acquisition order. Together, these results link confidence ranks to label budgets and certified model comparison.