Learning Distributions from Samples and Qualities

Marc Jourdan; Nicolas Flammarion

stat.MLOct 13, 2026

Learning Distributions from Samples and Qualities

Authors: Marc Jourdan, Nicolas Flammarion

Organizations: EPFL, Lausanne, Switzerland

Abstract

We study parameter estimation when each sample from a continuous parametric distribution is accompanied by a binary quality label governed by the same unknown parameter. By imposing hard feasibility constraints, deterministic labels can accelerate estimation from the classical $n^{-1/2}$ rate of smooth quality-aware M-estimators to $n^{-1}$ . Combining PAC guarantees with local error-to-parameter conversions, we establish this rate for both identifiable and finitely ambiguous quality classifiers, with likelihood information resolving the latter ambiguity. When the quality model is known only up to an additive constant, we construct threshold-free pairwise preferences from oppositely labeled samples and retain the accelerated rate despite the unknown threshold. We further show that pointwise qualities and pairwise preferences are incomparable: either can be strictly more informative than the other. Our assumptions hold for Gaussian, Laplace, and exponential families under thresholded log-likelihood rewards. Finally, a minimax lower bound establishes the optimal $n^{-1}$ dependence on sample size, up to logarithmic and dimension-dependent factors, while experiments confirm the benefits of constraint-based estimation.

Figures & tables

Figure 1: Estimation errors of

\mathcal{N}(\theta^{\star},1)

with

\theta^{\star}\sim\mathcal{U}([1,2])

and

N_{\text{runs}}=10^{3}

at (a-b)

\eta(1/2)\approx 1.15

and (c)

n=10^{4}

where

c_{\eta(q)}^{2}:=F^{-1}_{\chi^{2}(1)}(q)

. Error bars are

\pm 2

standard errors.

Figure 2: Estimation errors of

\mathcal{N}(\theta^{\star},I_{d})

with

\theta^{\star}\sim\mathcal{U}([1,2]^{d})

and

N_{\text{runs}}=10^{2}

at (a)

d=20

and (b)

n=10^{4}

where

c_{\eta(d)}^{2}:=F^{-1}_{\chi^{2}(d)}(1/2)

. Error bars are

\pm 2

standard errors.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Estimation errors with $\theta^{\star}\sim\mathcal{U}([1,2])$ of (a-b) Lap $(\theta^{\star},1)$ ( $N_{\text{runs}}=10^{3}$ ) and (c) Rayleigh $(\sqrt{\theta^{\star}})$ ( $N_{\text{runs}}=10^{2}$ ). The thresholds $\eta(\theta^{\star})$ are chosen such that $p_{\theta^{\star}}(r_{\theta^{\star}}>0)=1/2$ . Error bars are $\pm 2$ standard errors.

Figure 4: Estimation errors of (a) Exp

(\theta^{\star})

with

\theta^{\star}\sim\mathcal{U}([1,2])

, (b) Lap

(0,\theta^{\star})

with

\theta^{\star}\sim\mathcal{U}([1,2])

, and (c) Pareto

(\theta^{\star},1)

with

\theta^{\star}\sim\mathcal{U}([2,3])

(

N_{\text{runs}}=10^{3}

). The thresholds

\eta(\theta^{\star})

are chosen such that

p_{\theta^{\star}}(r_{\theta^{\star}}>0)=1/2

. Error bars are

\pm 2

standard errors.

Figure 5: Estimation errors of

\mathcal{N}(\theta^{\star},1)

with

\theta^{\star}\sim\mathcal{U}([1,2])

(

N_{\text{runs}}=10^{3}

) for

\eta(1/2)\approx 1.15

so that

c_{\eta(1/2)}^{2}:=F^{-1}_{\chi^{2}(1)}(1/2)

. Error bars are

\pm 2

standard errors.

Panel of Figure 6 — Figure 6: Estimation errors of $\mathcal{N}(\theta^{\star},I_{d})$ with $\theta^{\star}\sim\mathcal{U}([1,2]^{d})$ ( $N_{\text{runs}}=10^{2}$ ) for (top) $\eta(d)=\frac{1}{2}(1+d\log(2\pi))$ at (a) $d=5$ and (b) $n=10^{4}$ and (bottom) $\eta(d)=d\left(\frac{6}{5}+\frac{1}{2}\log(2\pi)\right)$ at (c) $d=5$ and (d) $n=10^{4}$ . Error bars are $\pm 2$ standard errors.

Figure 7: (a) Estimation errors of

\mathcal{N}(\theta^{\star},1)

with

\theta^{\star}\sim\mathcal{U}([1,2])

(

N_{\text{runs}}=200

) for

\eta(1/2)\approx 1.15

so that

c_{\eta(1/2)}^{2}:=F^{-1}_{\chi^{2}(1)}(1/2)

. Error bars are

\pm 2

standard errors. (b) Notable examples of binary classification loss functions ( Tang et al., 2024 , Figure 2) .

Figure 8: Estimation errors of

\mathcal{N}(\theta^{\star},1)

with

\theta^{\star}\sim\mathcal{U}([1,2])

when (a) normalizing by

\beta

for

\lambda=1

and (b) regularizing by

\lambda

for

\beta=1

(

N_{\text{runs}}=10^{3}

) for

\eta(1/2)\approx 1.15

so that

c_{\eta(1/2)}^{2}:=F^{-1}_{\chi^{2}(1)}(1/2)

. Error bars are

\pm 2

standard errors.

Distribution	Known	$\eta$	$\nabla_{\theta}M_{\mathrm{sto}}(\theta^{\star})$	$V_{\theta^{\star},\mathrm{sto}}$	$\nabla_{\theta}M_{\det}(\theta^{\star})$	$V_{\theta^{\star},\det}$
			$\mathrm{SQ}_{\mathrm{sto}}$		$\mathrm{SQ}_{\det}$
$\mathcal{N}(0,1)$	scale	$1.15$	$(1\pm 14)\cdot 10^{-4}$	$0.85$	$(1\pm 6)\cdot 10^{-4}$	$0.46$
		$1.42$	$(7\pm 13)\cdot 10^{-4}$	$0.84$	$(-3\pm 7)\cdot 10^{-4}$	$0.47$
		$2.12$	$(9\pm 12)\cdot 10^{-4}$	$0.83$	$(-5\pm 7)\cdot 10^{-4}$	$0.60$
Lap $(0,1)$	scale	$1.39$	$(7\pm 12)\cdot 10^{-4}$	$0.82$	$(-5\pm 8)\cdot 10^{-4}$	$0.52$
		$1.69$	$(2\pm 12)\cdot 10^{-4}$	$0.82$	$(7\pm 7)\cdot 10^{-4}$	$0.62$

Appendix

Table 1: Numerical estimation of the consistency of

\mathrm{SQ}_{\mathrm{sto}}

and

\mathrm{SQ}_{\det}

, and asymptotic variance when consistent (

N_{\text{runs}}=10^{6}

). The Student’s

t

-confidence intervals are defined at confidence level

95\%

Figure 9: Numerical estimation of the asymptotic variance

V_{\theta^{\star},\mathrm{sto}}=\alpha_{d}I_{d}

for

\mathrm{SQ}_{\mathrm{sto}}

and

V_{\theta^{\star},\det}=\alpha_{d}I_{d}

for

\mathrm{SQ}_{\det}

with

\mathcal{N}(0_{d},I_{d})

(

N_{\text{runs}}=10^{2}

). The thresholds are

\eta_{1}(d)=\frac{1}{2}(1+d\log(2\pi))

\eta_{2}(d)=d\left(\frac{6}{5}+\frac{1}{2}\log(2\pi)\right)

and

\eta_{3}(d)

such that

c_{\eta_{3}(d)}^{2}=F^{-1}_{\chi^{2}(d)}(1/2)

Parameter Estimation

Parameter estimation studies how to infer unknown quantities in statistical, physical, and machine-learning models from observed data.

See all 144 papers

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions
Sep 22, 2026Yicheng Li, Huifu XuParameter Estimation Minimax Estimation
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong.
Learning Distributions from Multiple Data Providers
Jul 27, 2026Jon Kleinberg, Amin Saberi, Xizhi Tan +1Statistical Learning Theory
Cornell University · Stanford University · Google Research
Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints
Mar 12, 2026Yasuhiro Yoshida, Noriyoshi Sukegawa, Jiro IwanagaDensity Estimation
School of Engineering, Stanford University, Stanford, CA 94305, USA · Faculty of Science and Engineering, Hosei University, Tokyo, Japan · Erdos Inc., Kanagawa, Japan; The University of Electro-Communications, Tokyo, Japan

Sep 22, 2026J/K move · Enter open · S save

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions

Yicheng Li, Huifu Xu

Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong.

Parameter Estimation Minimax Estimation Bradley–Terry Model Preference Elicitation

Open paper AskPDF arXiv

We study preference elicitation under the Bradley-Terry-Luce (BTL) model where the true partworth vector is unknown and has to be estimated as a parameter with elicited preference information. The set of selected pairwise queries is non-uniform, deterministic, and arbitrary over a collection of alternatives, provided that it satisfies a joint identifiability condition. We focus on understanding when the canonical maximum likelihood estimator (MLE) is finite and admits sharp error bounds without explicit compactness constraints on the feasible set or external regularizers. To this end, we derive minimax lower bounds under the standard bounded dynamic range condition, and find that the same Fisher-information geometry in the classic Cramér-Rao lower bounds underpins the finite-sample difficulty of the estimation problem. By combining a non-asymptotic expansion of the likelihood score equation with a fixed-point localization argument, we identify a design-dependent sample size threshold above which the unconstrained canonical MLE exists and is unique with high probability. The same expansion yields a decomposition of the estimation error into a linear stochastic term, an explicit second-order bias, and a higher-order remainder. A refined analysis gives sufficient sample size conditions under which the canonical MLE attains the minimax rates up to logarithmic and constant factors. These results provide a unified non-asymptotic theory for parametric utility elicitation and reveal when the inference is determined by response data alone rather than by external regularization. Preliminary numerical results are consistent with the theoretical findings.

Abstract

Parameter Estimation

Explore similar work

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions

Learning Distributions from Multiple Data Providers

Simultaneous estimation of multiple discrete unimodal distributions under stochastic order constraints