Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models

Jing Wang; HaiYing Wang; Qiang Zhang; Hao Helen Zhang

stat.MLAug 23, 2026

Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models

Authors: Jing Wang, HaiYing Wang, Qiang Zhang, Hao Helen Zhang

Organizations: Department of Statistics, University of Connecticut, Storrs, CT 06269, USA · Vickie and Jack Farber Vision Research Center, Wills Eye Hospital Thomas Jefferson University, 840 Walnut St, Philadelphia, PA 19107, USA · Department of Mathematics, University of Arizona, Tucson, AZ 85721, USA

Abstract

Subsampling is effective in tackling computational challenges for massive data with rare events. Overly aggressive subsampling may adversely affect estimation efficiency, and optimal subsampling is essential to mitigate the information loss. However, existing optimal subsampling probabilities depend on data scales, and some scaling transformations may result in inefficient subsamples. This problem is more significant when there are inactive features, because their influence on the subsampling probabilities can be arbitrarily magnified by inappropriate scaling transformations. We tackle this challenge and introduce a scale-invariant optimal subsampling function in the context of sparse models, where inactive features are commonly assumed. Instead of focusing on estimating model parameters, we define an optimal subsampling function to minimize the prediction error, using adaptive lasso to outline the estimation procedure and study its theoretical guarantee. We first introduce the adaptive lasso estimator for rare-events data and establish its oracle properties, thereby validating the use of subsampling. Then we derive a scale-invariant optimal subsampling function that minimizes the prediction error of the inverse probability weighted (IPW) adaptive lasso. Finally, we present an estimator based on the maximum sampled conditional likelihood (MSCL) to further improve the estimation efficiency. We conduct numerical experiments using both simulated and real-world data sets to demonstrate the performance of the proposed methods.

Figures & tables

Panel of Figure 1 — Figure 1: Prediction errors with different scale transformations of the same model with (a) non-sparse parameter $(-1,-1,-0.01,-0.01,-0.01,-0.01)^{\mathrm{T}}$ and (b) sparse parameter $(-1,0,0,0,0,0)^{\mathrm{T}}$ .

Panel of Figure 2 — Figure 2: Contributions of individual variables to sampling probabilities. The bar charts in the left panel present contributions with the original scale of $x_{(6)}$ ; those in the right panel present contributions after re-scaling $x_{(6)}$ .

Panel of Figure 3 — Figure 3: Bar charts for contributions of individual variables to the P-optimal probabilities. The left panel presents the contributions with the original scale and the right panel presents the contributions after re-scaling.

Panel of Figure 4 — Figure 4: eMSE for different true parameters with different sampling rates.

Panel of Figure 5 — Figure 5: eMPSE of estimated probability with different sampling rates.

$\rho$	first-stage	Uni	A-OS	L-OS	P-OS
0.0025	13.27(0.34)	2.84(0.02)	2.97(0.02)	2.96(0.02)	2.96(0.02)
0.005	12.46(0.32)	2.94(0.02)	3.04(0.03)	3.05(0.03)	3.06(0.03)
0.0075	12.76(0.33)	2.97(0.01)	3.04(0.02)	3.03(0.02)	3.03(0.02)
0.01	12.81(0.34)	2.96(0.01)	3.03(0.02)	3.02(0.01)	3.02(0.01)

Table 1: Mean number of selected variables in Case C.

$\rho$	Uni	A-OS	L-OS	P-OS
0.0025	0.168(0.017)	0.086(0.013)	0.088(0.013)	0.084(0.013)
0.005	0.100(0.013)	0.068(0.011)	0.066(0.011)	0.066(0.011)
0.0075	0.066(0.011)	0.046(0.009)	0.048(0.010)	0.046(0.009)
0.01	0.068(0.011)	0.052(0.010)	0.054(0.010)	0.054(0.010)

Table 2: Rates of excluding active variables (false negative rate) in Case C.

Case	Uni	A-OS	L-OS	P-OS	A-lasso (full)	Lasso (full)
A	0.29	1.09	0.91	1.06	129.62	112.97
B	0.31	1.23	1.20	1.27	129.89	122.40
C	0.31	1.02	0.93	1.00	130.33	121.29

Table 3: Mean computational time (seconds).

$\rho$	Las	SIS-3	SIS-6	SIS-13	SIS-20	Las-mle
0.0025	0.025	0.729	0.023	0.034	0.047	0.030
0.0050	0.017	0.728	0.014	0.023	0.031	0.020
0.0075	0.014	0.726	0.012	0.016	0.025	0.016

Table 4: Table of eMSEs with different pilot estimators for Case C.

Panel of Figure 6 — Figure 6: Empirical median AUCs for two real data sets.

Panel of Figure 7 — Figure 7: Empirical median AUC for the IRIS® data set.

Figure 8: Frequencies of selected variables.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

{\hat{\bm{\theta}}}_{\mathrm{pl}}:=\mathop{\arg\max}_{{\bm{\theta}}}\left\{\sum_{i=1}^{N_{\mathrm{pl}}^{*}}[y_{i}^{\mathrm{pl}}g(\bm{x}_{i}^{\mathrm{pl}};{\bm{\theta}})-\log\{1+e^{g(\bm{x}_{i}^{\mathrm{pl}};{\bm{\theta}})+l}\}]-\lambda_{\mathrm{pl}}\sum_{j=1}^{p}|\beta_{(j)}|\right\},

(25)

Appendix

Algorithm 3 Subsampling adaptive lasso algorithm

	Case A (five active variables)
$\rho$	first-stage	Uni	A-OS	L-OS	P-OS
0.0025	14.87(0.30)	4.99(0.02)	5.02(0.02)	5.03(0.02)	5.01(0.02)
0.005	14.68(0.28)	5.05(0.02)	5.07(0.02)	5.06(0.02)	5.08(0.02)
0.0075	14.20(0.27)	5.09(0.03)	5.08(0.02)	5.08(0.02)	5.09(0.03)
0.01	14.46(0.30)	5.13(0.02)	5.13(0.02)	5.13(0.02)	5.12(0.03)

Appendix

Table 5: Mean number of selected variables in Case A and Case B.

	Case A
$\rho$	Uni	A-OS	L-OS	P-OS
0.0025	0.094(0.013)	0.076(0.012)	0.068(0.011)	0.070(0.012)
0.005	0.052(0.010)	0.048(0.010)	0.050(0.010)	0.044(0.010)
0.0075	0.054(0.010)	0.052(0.010)	0.052(0.010)	0.052(0.010)
0.01	0.040(0.009)	0.036(0.008)	0.036(0.008)	0.036(0.008)

Appendix

Table 6: Rates of excluding active variables (false negative rate) in Case A and Case B.

	Case A
$\rho$	Uni	A-OS	L-OS	P-OS
0.0025	0.856(0.016)	0.870(0.015)	0.868(0.015)	0.878(0.015)
0.005	0.884(0.014)	0.872(0.015)	0.880(0.015)	0.872(0.015)
0.0075	0.858(0.016)	0.868(0.015)	0.868(0.015)	0.868(0.015)
0.01	0.854(0.016)	0.866(0.015)	0.862(0.015)	0.868(0.015)

Appendix

Table 7: Rates of selecting the true model.

Panel of Figure 9 — Figure 9: Empirical median squared error of estimated probability for different parameters with different sampling rates. The same pilot sample size is $N_{\mathrm{pl}}=500$ .

Panel of Figure 10 — Figure 10: Empirical median squared error of estimated probability for different parameters with different sampling rates. The same pilot sample size is $N_{\mathrm{pl}}=500$ .

	A			B			C
$\rho$	sUni	P-OS	sP-OS	sUni	P-OS	sP-OS	sUni	P-OS	sP-OS
0.0025	0.848	0.878	0.862	0.850	0.876	0.862	0.828	0.884	0.880
0.005	0.878	0.872	0.850	0.890	0.902	0.890	0.882	0.882	0.880
0.0075	0.850	0.868	0.846	0.850	0.848	0.842	0.906	0.914	0.910
0.01	0.840	0.868	0.844	0.868	0.872	0.862	0.900	0.910	0.904

Appendix

Table 8: Rates of selecting true models.

	A			B			C
$\rho$	sUni	P-OS	sP-OS	sUni	P-OS	sP-OS	sUni	P-OS	sP-OS
0.0025	0.088	0.070	0.068	0.106	0.070	0.070	0.164	0.084	0.084
0.005	0.056	0.044	0.044	0.046	0.036	0.036	0.100	0.066	0.066
0.0075	0.052	0.052	0.052	0.062	0.044	0.044	0.062	0.046	0.046
0.01	0.040	0.036	0.036	0.056	0.050	0.050	0.068	0.054	0.054

Appendix

Table 9: Rates of excluding active variables (false negative rate).

Rare-Event Estimation

Rare-event estimation measures probabilities of failures, extremes, or transitions that ordinary sampling may miss.

See all 28 papers

Oct 5, 2026

Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures

We study estimating rare-event probabilities

I = \mathbb{P}(g(\mathbf{X}) > γ)

with

\mathbf{X} \sim \mathcal{N}(\boldsymbolμ, \boldsymbolΣ)

and general

g : \mathbb{R}^d \to \mathbb{R}

. We address this problem through importance sampling, and propose a framework that substantially improves efficiency and robustness over baselines such as crude Monte Carlo, adaptive cross-entropy, variational-inference-based methods (including reverse- and forward-KL approaches), as well as Safe-ICE, Subset Simulation, and Sequential Monte Carlo, drawing on ideas from both rare-event estimation and cross-entropy optimization. The key contribution has two parts: first, we separate the problem into coverage, to overcome the cold-start barrier, and fitting, to refine proposals once a meaningful signal is available; second, we constrain the final GMM proposal so that it has finite importance-sampling variance (since coverage alone is not sufficient -- without safeguards, importance sampling may still suffer from infinite variance). Together, these ingredients yield expressive proposals; finite variance does not by itself guarantee practical stability at a fixed sampling budget. Extensive experiments demonstrate substantial variance reduction, strong robustness across diverse benchmarks, and favorable cost--efficiency trade-offs, with the proposed approach often outperforming these baselines, particularly in high-dimensional and multimodal settings where competing methods frequently become unstable or fail. Our code is available at https://github.com/lorek/robust-cfi-is.

Paweł Lorek, Rafał Nowak, Rafał Topolnicki +2

University of Wrocław · Tooploox · TRAILS University of Warsaw +3

Gaussian Mixture Models Importance Sampling Variance Reduction Rare-Event Estimation

Ask PDF

Jun 1, 2026

Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data

Estimating the causal effect of time-varying treatments on survival outcomes in large observational studies is computationally demanding, particularly when outcomes are rare. While g-formula-based methods such as the iterative conditional expectation (ICE) estimator provide a principled framework for longitudinal causal inference, they become computationally expensive, especially when bootstrap-based variance estimation is required. In addition, outcome rarity at each time point induces severe class imbalance, leading to instability and convergence issues in logistic regression and related models. To address these challenges, we propose a principled subsampling and reweighting strategy for longitudinal survival data that can be applied to a range of existing causal effect estimators in this setting, including the ICE estimator. The proposed method substantially reduces computational burden while preserving consistency and improving estimation stability in rare-outcome settings. We evaluate the method through simulations and validate it using a large-scale EHR cohort study on social and behavioral determinants of health (SBDH) and suicide risk, demonstrating its effectiveness for modeling rare outcomes in longitudinal data.

Xiaohui Yin, Avijit Mitra, Ying Zhou +2

University of Massachusetts Amherst Amherst, Massachusetts, USA · University of Massachusetts Lowell Lowell, Massachusetts, USA

Causal Effect Estimation Survival Analysis Longitudinal EHR Modeling Rare-Event Estimation

Ask PDF

Apr 27, 2026

Nearly Optimal Subdata Selection

When, in terms of the number of data points, the size of a dataset exceeds available computing resources, or when labeling is expensive, an attractive solution consists of selecting only some of the data points (subdata) for further consideration. A central question for selecting subdata of size

n

from

N

available data points is which

n

points to select. While an answer to this question depends on the objective, one approach for a parametric model and a focus on parameter estimation is to select subdata that retains maximal information. Identifying such subdata is a classical NP-hard problem due to its inherent discreteness. Based on optimal approximate design theory, we develop a new methodology for information-based subdata selection, resulting in subdata that approaches the optimal solution. To achieve this, we develop a novel algorithm that applies to a general model, accommodates arbitrary choices of

N

and

n

, and supports multiple optimality criteria, and we prove its convergence. Moreover, the new methodology facilitates an assessment of the efficiency of subdata selected by any method by obtaining tight lower and upper bounds for the efficiency. We show that the subdata obtained through the new methodology is highly efficient and outperforms all existing methods.

Min Yang, Wei Zheng, John Stufken +3

Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago · Department of Business Analytics and Statistics, University of Tennessee · Department of Statistics, George Mason University[stat.ME] +3

Data Selection Training Data Selection Coreset Selection Approximation Algorithms

Ask PDF

Abstract

Rare-Event Estimation

Explore similar work

Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures

Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data

Nearly Optimal Subdata Selection