Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints

Yasuhiro Yoshida; Noriyoshi Sukegawa; Jiro Iwanaga

math.OCMar 12, 2026

Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints

Authors: Yasuhiro Yoshida, Noriyoshi Sukegawa, Jiro Iwanaga

Organizations: School of Engineering, Stanford University, Stanford, CA 94305, USA · Faculty of Science and Engineering, Hosei University, Tokyo, Japan · Erdos Inc., Kanagawa, Japan; The University of Electro-Communications, Tokyo, Japan

Abstract

We study the problem of estimating multiple discrete unimodal distributions, motivated by search behavior analysis on a real-world platform. To incorporate prior knowledge of precedence relations among distributions, we impose stochastic order constraints and formulate the estimation task as a mixed-integer convex quadratic optimization problem. Experiments on both synthetic and real datasets show that the proposed method reduces the Jensen-Shannon divergence by 2.2% on average (up to 6.3%) when the sample size is small, while performing comparably to existing methods when sufficient data are available.

Figures & tables

Figure 1. Three search timing distributions for keywords containing body weight on Mamari

Figure 2. Change in the estimation error (JSD) with respect to sample size using synthetic data

Figure 3. Visual comparison of estimated distributions with varying sample sizes

Data set	EMP	GAUSSIAN	KERNEL	UNIMODAL	OURS
10 records	0.1425	0.0932	0.0983	0.0888	0.0869
20 records	0.0859	0.1043	0.0568	0.0571	0.0561
30 records	0.0656	0.1147	0.0450	0.0453	0.0446
40 records	0.0552	0.1218	0.0375	0.0391	0.0386
50 records	0.0487	0.1379	0.0354	0.0350	0.0348
60 records	0.0442	0.1261	0.0310	0.0323	0.0322

Table 1. Average estimation error (JSD) for each model with varying record sizes

Instance		EMP	KERNEL	UNIMODAL	OURS
(F1)	diarrhea	0.238	0.157	0.115	0.113
	body_weight	0.251	0.171	0.131	0.130
	constipation	0.209	0.128	0.097	0.095
	abdominal_pain	0.205	0.135	0.111	0.109
	anemia	0.213	0.139	0.109	0.108
(F2)	toddler_defiance	0.122	0.089	0.090	0.087

Table 2. Estimation error by each model for the 10-record dataset

Instance		EMP	KERNEL	UNIMODAL	OURS
(F1)	diarrhea	0.068	0.041	0.037	0.041
	body_weight	0.075	0.061	0.060	0.061
	constipation	0.044	0.033	0.030	0.030
	abdominal_pain	0.058	0.037	0.037	0.038
	anemia	0.051	0.037	0.036	0.036
(F2)	toddler_defiance	0.039	0.030	0.032	0.031

Table 3. Estimation error by each model for the 80-record data set

Panel of Figure 4 — Figure 4. Estimated distributions for two instances

Density Estimation

Density estimation learns probability distributions from data to measure likelihood, detect anomalies, quantify uncertainty, and generate samples.

See all 66 papers

Estimating Mixture Distributions via Stochastic Mirror Descent
May 24, 2026Mohammadreza Ahmadypour, Tara Javidi, Farinaz KoushanfarStochastic Optimization Statistical Learning Theory
Department of Electrical and Computer Engineering University of California San Diego, San Diego, CA, USA
Symbolic Density Estimation for Discrete Distributions
May 20, 2026Ziwen Liu, Meng LiDensity Estimation
Rice University Houston, TX
Learning Distributions from Samples and Qualities
Oct 13, 2026Marc Jourdan, Nicolas FlammarionParameter Estimation
EPFL, Lausanne, Switzerland

May 24, 2026J/K move · Enter open · S save

Estimating Mixture Distributions via Stochastic Mirror Descent

Mohammadreza Ahmadypour, Tara Javidi, Farinaz Koushanfar

Department of Electrical and Computer Engineering University of California San Diego, San Diego, CA, USA

Stochastic Optimization Statistical Learning Theory Density Estimation

Open paper AskPDF arXiv

We revisit the classical problem of estimating an unknown distribution from its samples by fitting a mixture model that minimizes cross-entropy loss. Framing the task as a stochastic convex optimization problem over the space of

M

-component mixture distributions, we propose a family of estimators derived from the stochastic mirror descent (SMD) algorithm. This optimization-based approach provides a principled and flexible framework that generalizes traditional estimators and proposes a variety of novel estimators through the choice of Bregman divergences. A key advantage of our method is that it scales efficiently with the number of candidate components

f_i

; that is, one can employ a large set of basis distributions in the mixture model without incurring significant computational overhead. This enables richer approximations and improved estimation accuracy. Moreover, in the case of categorical distribution (discrete outcomes) our estimators do not require a strict lower bound, in other words our framework does not require the precise knowledge of the support of the distribution. We demonstrate that, under mild conditions, the proposed

\varphi

-SMD estimators achieve near-optimal convergence rates in both Kullback-Leibler (KL) divergence and

\ell_2

-norm and offer practical benefits when computation is expensive. Our numerical analysis highlights improved performance guaranties over classical estimators, particularly in terms of sample efficiency and scalability.

Abstract

Density Estimation

Explore similar work

Estimating Mixture Distributions via Stochastic Mirror Descent

Symbolic Density Estimation for Discrete Distributions

Learning Distributions from Samples and Qualities