cs.LGJun 12, 2026

A Complexity Measure for Active Learning in Multi-group Mean Estimation

Authors: Abdellah AznagRachel CummingsAdam N. Elmachtoub

Organizations: Department of Industrial Engineering and Operations Research & Data Science Institute, Columbia University

Abstract

We study a \emph{max-risk} objective for active learning in a multi-group mean estimation dd-armed bandits: a learner adaptively allocates a budget of TT samples across dd groups to minimize the worst-case uncertainty index maxk[d]σk2/nk\max_{k\in[d]}σ_k^2/n_k, where σkσ_k is the standard deviation of the distribution of arm dd, and nkn_k is the number of times arm dd is sampled. We develop a local minimax framework and prove the first general lower bound for this objective, valid for any finite-variance hypothesis class. The bound separates difficulty into three orthogonal factors: a \emph{budget} term, a \emph{heteroscedasticity} index measuring how unevenly the uncertainty is spread across arms, and a model-dependent complexity measure, the \emph{Variance Local Curvature} (VLC\mathrm{VLC}), which captures how much information a local change of variance creates inside the hypothesis class. For smooth classes, the VLC\mathrm{VLC} is a reparametrization of a variance--Fisher information, with closed-form values for common families. Benchmarking against the strongest available upper bound shows near-optimality up to logarithmic factors in broad regimes, and pinpoints a systematic gap in highly heterogeneous instances. Our proof introduces two key ingredients: a loss-induced 1\ell_1 geometry on the decision space, and a representation-based instance generator that reduces hard-instance construction to an explicit random matrix calculation.

Explore similar work

Jul 7, 2026cs.LG

The Approximation Ratio for the Risk of Myopic Bayesian Active Learning for Linear Regression

Active learning studies the fundamental question: what data should we choose to observe? The greedy algorithm in optimal experiment design is a common heuristic and also equivalent to myopic Bayesian active learning for linear regression, the common framework where long-term planning is replaced with the one-step optimal choice. In this work, we prove a first-of-its-kind approximation ratio for the greedy algorithm's risk that is tight up to an absolute constant. The approximation ratio is linear in the maximum initial leverage score (MILS), a newly identified quantity fundamental to the greedy algorithm's performance. Finally, we illustrate the results with simple numerical simulations.
Stephen Mussmann
May 14, 2026cs.LG

A Mutual Information Lower Bound for Multimodal Regression Active Learning

Active learning for continuous regression has lacked an acquisition function that targets epistemic uncertainty when the predictive distribution is multimodal: variance misses modal disagreement, and information-theoretic targets like BALD are designed for discrete outputs. We introduce a Two-Index framework that makes this separation explicit: one stochastic index selects among competing model hypotheses (epistemic source), while a second governs within-hypothesis randomness (aleatoric source). An entropy decomposition within the framework identifies the mutual information between the output and the epistemic index as a principled acquisition objective, and we prove this quantity vanishes as the model is trained on growing datasets, confirming that it captures exactly the uncertainty data can resolve. Because this mutual information is intractable for continuous outputs, we derive the Mutual Information Lower Bound (MI-LB) acquisition function, a closed-form approximation for Mixture Density Network ensembles. On benchmarks featuring multimodal systems, MI-LB matches or beats every baseline evaluated and is the only method to do so consistently -- geometric and Fisher-based baselines compete only when the input space already encodes the multimodality, and collapse otherwise.
Leonardo Ferreira Guilhoto, Akshat Kaushal, Paris Perdikaris
May 4, 2026cs.LG

Gradient-Discrepancy Acquisition for Pool-Based Active Learning

The effectiveness of active learning hinges on the choice of the acquisition criterion by which a learning algorithm selects potentially informative data points whose label is subsequently queried. This paper proposes a novel gradient-based acquisition criterion, derived from a generalization bound introduced by Luo et al. (2022). This criterion can be applied in lieu of uncertainty measures in uncertainty sampling, or incorporated into diversity-based methods that consider the spread of sampled points in addition to the uncertainty of their labels. We provide a theoretical justification of the proposed acquisition criterion, and demonstrate its effectiveness in an empirical evaluation.
Mohamadsadegh Khosravani, Sandra Zilles