HALO: Hierarchical Evidence Selection and Reinforcement Learning for Allocating Budgeted Label Optimization from Longitudinal Clinical Records
Authors: Huaiyuan Ying, Feiqing Huang, Ryumei Nakada, Katherine P. Liao, Tianxi Cai
Organizations: Department of Statistics and Data Science Tsinghua University Beijing, China · Department of Biostatistics Harvard T.H. Chan School of Public Health Boston, MA, USA · Brigham and Women’s Hospital Harvard Medical School Boston, MA, USA · Harvard T.H. Chan School of Public Health Harvard Medical School Boston, MA, USA
Learning from extensive longitudinal clinical records--spanning repeated encounters, evolving conditions, and heterogeneous documentation--under a finite labeling budget requires solving two coupled problems: deciding what evidence to keep and deciding which patients deserve higher-fidelity labels. Prior work on active learning and multi-fidelity label acquisition mainly addresses the second problem, while treating the evidence presented to the acquisition policy as fixed. We present HALO, a Hierarchical Evidence Selection and Reinforcement Learning pipeline for Allocating Budgeted Label Optimization from Longitudinal Clinical Records. This framework jointly learns what to read and what label source to acquire under budget. HALO first selects informative evidence from longitudinal records and then uses a budget-aware policy to allocate free-, silver-, and gold-fidelity label sources. For frozen policies, our theory characterizes asymmetric acquired-label channels and gives a budget-constrained finite-sample gold-risk bound for a common downstream logistic model. In a heart-failure case study, HALO attained the highest MedAlign acquired-label F1 at six of eight target budgets. A downstream classifier trained on HALO-acquired labels attained the highest external MGB F1 at budgets 15-60 and remained within 0.01 of the best comparator at budgets 75-120.
Figures & tables
Figure 1: Overview of HALO. Hierarchical evidence selection first retains informative notes using a frozen note scorer and a fixed threshold τ1 . A trainable segment-informativeness model with parameters ξ then samples segment-retention decisions during PPO training; at inference, segments are retained when their predicted retention probabilities exceed τ2=0.5 . The retained evidence is summarized into a compact patient representation. Budgeted label allocation feeds the compact summary to a PPO-updated patient-worthiness scorer, implemented with a T5-scale encoder and ordinal head, which routes each patient to free-, silver-, or gold-fidelity label tiers under a finite labeling budget. The design objective is to reserve expensive labels for patients for whom the policy predicts higher fidelity to be most valuable.
Figure 2: Performance across target label-acquisition budgets. (a) MedAlign acquired-label F1 measures agreement between each strategy’s acquired labels and the GPT-5-derived proxy reference on 106 patients. (b) MGB external downstream F1 evaluates a penalized logistic regression trained on each strategy’s acquired MedAlign labels against independently curated clinician chart-review labels on 299 MGB patients. Each plotted value is the mean F1 across ten repeated runs. Label-source costs are c(f)=0 , c(s)=0.6 , and c(g)=3 . Higher values are better in both panels. Lines connect the eight evaluated target-budget settings.