Conformal Selection for Hierarchical Streaming Data
Organizations: School Mathematical Sciences, Peking University · Department of Statistics and Data Science, Southern University of Science and Technology · School of Statistics and Data Science, Zhejiang Gongshang University · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen ·
Abstract
Many real-world tasks involve making sequential decisions about whether to select candidates before their responses of interest are revealed. In this work, we address the online selection problem in the context of hierarchical data, where candidates arrive sequentially from different populations. This extends existing online conformal selection frameworks beyond the standard i.i.d./exchangeable data assumption. The primary challenge lies in constructing valid statistics for each candidate sequentially under the hierarchical data structure. To this end, we first develop Online Hierarchical Selection (OHS), a distribution-free method that enables principled candidate selection for hierarchical data streams. Specifically, we introduce methods to construct rank-based e-values for both between-group and within-group samples, and apply averaged e-values to the online e-BH procedure. We then develop OHS with feedback (OHSF), which boosts the power of OHS by leveraging potential response feedback. Theoretically, the proposed methods achieve false discovery rate control under mild hierarchical exchangeability and data arrival assumptions at any time. We also generalize methods to broader settings, such as data with covariate shift and selection under effect-related responses. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness of the proposed methods.
Figures & tables
Conformal Prediction
Conformal prediction wraps predictive models with calibrated sets or intervals that provide statistical coverage guarantees.
See all 401 papers