Rarity of Singletons Matters & Supplementary Materials

Pragatheeswaran Vipulanandan; Kamal Premaratne; Dilip Sarkar

cs.LGOct 15, 2026

Rarity of Singletons Matters & Supplementary Materials

Authors: Pragatheeswaran Vipulanandan, Kamal Premaratne, Dilip Sarkar

Organizations: Dept. of Elect. and Comp. Engineering University of Miami Coral Gables, FL 33146, USA · Dept. of Comp. Science University of Miami Coral Gables, FL 33146, USA

Abstract

Estimating missing-mass and unseen-species counts from finite samples relies heavily on rare-event occupancy statistics, particularly singletons. In small-sample regimes, these statistics can be highly variable, making classical estimators sensitive to sampling fluctuations and potentially unreliable. We propose a fragility-aware (FA) framework that explicitly quantifies this sensitivity and adaptively regularizes conventional missing-mass and unseen-species estimators. Using singleton-based fragility measures derived from probability mass and occupancy counts, we formulate an entropy--Kullback--Leibler (KL) optimization that adjusts the reliance on base estimates according to their fragility. The resulting objectives are strictly concave, admit unique closed-form solutions, and require no Poissonization or asymptotic approximations. The framework provides a lightweight correction to classical estimators, including Good--Turing and Good--Toulmin, while reducing reliance on potentially unstable rare-event statistics. Across 360 experimental configurations spanning synthetic distributions and real-world language corpora, our method remains competitive in stable regimes while improving estimation accuracy and stability in fragmented, long-tailed, and small-sample settings. The findings demonstrate that accounting for the fragility of rare observations can improve the reliability of inference about unobserved species and their associated probability mass.

Figures & tables

Fig. 1: Overview of our fragility-aware estimation pipeline.

Panel of Figure 2 — Fig. 2: Per-family UCR win rates for missing-mass (top): GT [ 3 ] , SGT [ 16 ] , BD [ 17 ] , PY [ 18 ] ; and unseen-species (bottom): GTo [ 4 ] , Chao [ 1 ] , ACE [ 19 ] , Orl-SGT [ 5 ] , JK1/2 [ 13 ] . Both include FA (ours). Entries indicate the fraction of experiments where the row method achieves lower UCR than the column method.

Cluster	Count	$p_{s}^{(r)}$	$p_{c}^{(\ell)}$	$\widehat{c}^{(\mathrm{GT})}$	$\widehat{c}^{(\mathrm{SGT})}$	$\widehat{c}^{(\mathrm{BD})}$	$\widehat{c}^{(\mathrm{PY})}$	$\widehat{c}^{(\mathrm{FA})}(Ours)$
1. Russia	1	0.1500	0.1500	0.1519	0.1424	0.1582	0.1519	0.1512
2. S. Arabia	8	0.0275	0.2200	0.2228	0.2089	0.2321	0.2228	0.2217
3. Iran	2	0.0500	0.1000	0.1013	0.0949	0.1055	0.1013	0.1008
4. Kuwait	2	0.0250	0.0500	0.0506	0.0475	0.0527	0.0506	0.0504
5. Qatar	1	0.1400	0.1400	0.1418	0.1329	0.1477	0.1418	0.1411
6. Iraq	1	0.1300	0.1300	0.1317	0.1234	0.1371	0.1317	0.1310

TABLE I: Worked example on the semantic-clustering query (

n=15

, six clusters,

n_{\mathrm{new}}=5

\{\Phi_{1},\,\Phi_{2},\,\Phi_{\mathrm{obs}}\}=\{3,\,2,\,6\}

, and

M_{0}=0.210

). The upper panel compares missing-mass estimators (GT, SGT, BD, PY, and FA), while the lower panel compares unseen-species count estimators (GTo, Chao, ACE, Orl-SGT, and FA). Our FA method achieves the lowest absolute error in both tasks.

Panel of Figure 4 — Fig. 3: Synthetic overview (144 experiments). Top: pairwise UCR win rates; bottom: normalized pairwise gain $g_{ij}$ (see ( 13 )). Blue indicates that the row method lowers UCR relative to the column baseline.

Panel of Figure 5 — Fig. 4: Pairwise UCR win rates across Shakespeare (top) and Google 1-gram (bottom), each with 108 experiments, pooled over $r_{\mathrm{future}}\in[0.03,0.10]$ . Each entry $(i,j)$ denotes the fraction of experiments where row method $i$ outperforms column method $j$ ; values above $0.5$ indicate more frequent wins. Fragility-aware estimation performs best on unseen counts and the heavier-tailed Google corpus.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

family	parameter	$N$	Ours	GT	SGT	BD	PY
zipf	$s=0.5$	25	0.149	0.149	0.149	0.510	0.978
zipf	$s=0.5$	50	0.133	0.132	0.201	0.463	0.914
zipf	$s=0.5$	100	0.123	0.123	0.242	0.389	0.803
zipf	$s=0.5$	200	0.099	0.100	0.194	0.259	0.628
zipf	$s=0.5$	500	0.065	0.064	0.280	0.081	0.356
zipf	$s=0.5$	1000	0.033	0.033	0.255	0.076	0.168

Appendix

TABLE S1 : UCR (lower is better) for missing-mass estimation on Zipf populations across tail exponent

s

and sample size

N

family	parameter	$N$	Ours	GT	SGT	BD	PY
HT	$\rho=0.7$	25	0.145	0.144	0.144	0.520	0.992
HT	$\rho=0.7$	100	0.119	0.118	0.165	0.425	0.862
HT	$\rho=0.7$	1000	0.033	0.032	0.311	0.098	0.149
HT	$\rho=0.85$	25	0.154	0.153	0.153	0.512	0.977
HT	$\rho=0.85$	100	0.117	0.119	0.228	0.378	0.804
HT	$\rho=0.85$	1000	0.036	0.036	0.210	0.095	0.132

Appendix

TABLE S2 : Representative UCR values for missing-mass estimation on head–tail mixtures (

\rho

= head-mass fraction). GT is within 0.001–0.002 of Ours at most settings; both dominate SGT and Bayesian baselines at small

N

Panel of Figure 8 — Fig. S1 : Per-family normalized pairwise gain for (a) missing-mass and (b) unseen-species estimation.

Fig. S2 : Missing-mass win-rate heatmaps on Shakespeare across $r_{\mathrm{future}}\in[0.03,0.10]$ (fraction of experiments where the row beats the column on UCR).

Fig. S3 : Unseen-count win-rate heatmaps on Shakespeare across $r_{\mathrm{future}}\in[0.03,0.10]$ .

Fig. S4 : Missing-mass win-rate heatmaps on Google 1-gram across $r_{\mathrm{future}}\in[0.03,0.10]$ .

Fig. S5 : Unseen-count win-rate heatmaps on Google 1-gram across $r_{\mathrm{future}}\in[0.03,0.10]$ .

Rare-Event Estimation

Rare-event estimation measures probabilities of failures, extremes, or transitions that ordinary sampling may miss.

See all 29 papers

Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models
Aug 23, 2026Jing Wang, HaiYing Wang, Qiang Zhang +1Rare-Event Estimation Adaptive Sampling
Department of Statistics, University of Connecticut, Storrs, CT 06269, USA · Vickie and Jack Farber Vision Research Center, Wills Eye Hospital Thomas Jefferson University, 840 Walnut St, Philadelphia, PA 19107, USA · Department of Mathematics, University of Arizona, Tucson, AZ 85721, USA
Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures
Oct 5, 2026Paweł Lorek, Rafał Nowak, Rafał Topolnicki +2Gaussian Mixture Models Importance Sampling
University of Wrocław · Tooploox · TRAILS University of Warsaw +3
Rare Event Estimation via Iterative Unalignment
Sep 21, 2026Hanming Yang, Daksh Mittal, Jing Dong +1Importance Sampling LLM Safety
Decision, Risk, and Operations Division, Columbia Business School

Aug 23, 2026J/K move · Enter open · S save

Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models

Jing Wang, HaiYing Wang, Qiang Zhang, Hao Helen Zhang

Department of Statistics, University of Connecticut, Storrs, CT 06269, USA · Vickie and Jack Farber Vision Research Center, Wills Eye Hospital Thomas Jefferson University, 840 Walnut St, Philadelphia, PA 19107, USA · Department of Mathematics, University of Arizona, Tucson, AZ 85721, USA

Rare-Event Estimation Adaptive Sampling

Open paper AskPDF arXiv

Subsampling is effective in tackling computational challenges for massive data with rare events. Overly aggressive subsampling may adversely affect estimation efficiency, and optimal subsampling is essential to mitigate the information loss. However, existing optimal subsampling probabilities depend on data scales, and some scaling transformations may result in inefficient subsamples. This problem is more significant when there are inactive features, because their influence on the subsampling probabilities can be arbitrarily magnified by inappropriate scaling transformations. We tackle this challenge and introduce a scale-invariant optimal subsampling function in the context of sparse models, where inactive features are commonly assumed. Instead of focusing on estimating model parameters, we define an optimal subsampling function to minimize the prediction error, using adaptive lasso to outline the estimation procedure and study its theoretical guarantee. We first introduce the adaptive lasso estimator for rare-events data and establish its oracle properties, thereby validating the use of subsampling. Then we derive a scale-invariant optimal subsampling function that minimizes the prediction error of the inverse probability weighted (IPW) adaptive lasso. Finally, we present an estimator based on the maximum sampled conditional likelihood (MSCL) to further improve the estimation efficiency. We conduct numerical experiments using both simulated and real-world data sets to demonstrate the performance of the proposed methods.

Abstract

Rare-Event Estimation

Explore similar work

Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models

Robust Importance Sampling for Rare Events via Constrained Gaussian Mixtures

Rare Event Estimation via Iterative Unalignment