We study online individualized treatment selection when only the outcome under the assigned treatment is observed. Our objective is to preferentially treat individuals whose potential outcomes under treatment exceed a covariate-specific control reference, while limiting false discoveries among treated individuals. We propose Partial-Feedback Adaptive Individualized Treatment Selection (PAITS), which initializes learning from historical data and combines sequential estimation of local null risks, randomized exploration, and feedback-driven updates of the treatment selection threshold. Under regularity conditions, we establish asymptotic false discovery rate control and sublinear gap-weighted assignment regret relative to a population oracle. We further establish a minimax lower bound on assignment regret under prescribed uniform nuisance-estimation requirements and give conditions under which PAITS attains the corresponding constrained minimax rate up to logarithmic factors. These results quantify the tradeoff between accurate treatment assignment and nuisance learning under partial feedback. Simulation studies and a real-data analysis using the Jester recommendation dataset illustrate the finite-sample performance of PAITS.
Figures & tables
Figure 1: Empirical FDR over online time for Models M1–M3. The top row compares four control-reference estimators at χ=0.5 , and includes the known-nuisance setting. The bottom row varies χ , using weighted B-spline estimation for both nuisance functions. The horizontal line denotes the nominal FDR level α=0.1 .
Figure 2: Log–log plots of the mean cumulative gap-weighted assignment regret for Models M1–M3. The top row compares four control-reference estimators at χ=0.5 . The bottom row varies χ , using weighted B-spline estimation for both nuisance functions.
Figure 3: Sensitivity to the historical sample size for Models M1–M3. The historical sample sizes are nhist∈{1000,2000,3000,4000} . The top row reports the empirical FDR over online time, and the bottom row reports the log–log plots of the mean cumulative gap-weighted assignment regret.
Figure 4: Sensitivity to the step-size exponent ζ for Models M1–M3. The adaptive cutoff uses a polynomial step-size schedule ηt≍t−ζ . The top row reports the empirical FDR over online time, and the bottom row reports the log–log plots of the mean cumulative gap-weighted assignment regret.
Figure 5: Performance of PAITS on the Jester data over five randomized replay orders. Left: cumulative false discovery proportion over online time, where the thick solid line represents the pointwise average over the five orders and the dashed line denotes the target level α=0.10 . Right: log–log trajectories of cumulative gap-weighted assignment regret relative to the finite-population oracle, together with the fitted log–log slopes.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Supplementary log–log fits for the mean cumulative gap-weighted assignment regret in Models M1–M3, corresponding to Figure 2 . The top row compares four control-reference estimators at χ=0.5 , with weighted B-spline estimation of F1 throughout. The bottom row varies χ , using weighted B-spline estimation for both nuisance functions. The lines show the fitted linear trends over the displayed time interval, and the legends report their slopes.
Figure 7: Supplementary log–log fits for the historical-sample-size comparison in Figure 3 . The panels correspond to Models M1–M3, with nhist∈{1000,2000,3000,4000} . Both nuisance functions are estimated by weighted B-spline regression, with χ=0.5 in M1 and M2 and χ=0.4 in M3. The lines show fitted linear trends in the log mean cumulative gap-weighted assignment regret against logt over the displayed time interval. The legends report the fitted slopes.
Figure 8: Supplementary log–log fits for the step-size comparison in Figure 4 . The panels correspond to Models M1–M3, with ζ∈{0.55,0.60,0.65,0.70,0.75} . Both nuisance functions are estimated by weighted B-spline regression, with χ=0.5 in M1 and M2 and χ=0.4 in M3. The lines show fitted linear trends in the log mean cumulative gap-weighted assignment regret against logt over the displayed time interval. The legends report the fitted slopes.