Mixture Proportion Estimation and Weakly-supervised Kernel Test for Conditional Independence

Yushi Hirose; Akito Narahara; Takafumi Kanamori

cs.LGApr 8, 2026

Mixture Proportion Estimation and Weakly-supervised Kernel Test for Conditional Independence

Authors: Yushi Hirose, Akito Narahara, Takafumi Kanamori

Organizations: Institute of Science Tokyo · Institute of Science Tokyo, RIKEN AIP

Abstract

Mixture proportion estimation (MPE) aims to estimate class priors from unlabeled data. This task is a critical component in weakly supervised learning, such as PU learning, learning with label noise, and domain adaptation. Existing MPE methods rely on the irreducibility assumption or its variant for identifiability. In this paper, we propose novel assumptions based on conditional independence (CI) given the class label, which ensure identifiability even when irreducibility does not hold. We develop method of moments estimators under these assumptions and analyze their asymptotic properties. Furthermore, we present weakly-supervised kernel tests to validate the CI assumptions, which are of independent interest in applications such as causal discovery and fairness evaluation. Empirically, we demonstrate the improved performance of our estimators compared with existing methods and that our tests successfully control both type I and type II errors.\label{key}

Figures & tables

	Gaussian	Shuttle	Wine	Dry Bean
DEDPUL	0.043	0.117	0.077	0.074
KM2	0.027	0.152	0.129	0.029
EN	0.063	0.075	0.110	0.107
CI MPE	0.013	0.053	0.031	0.025

Table 1: Mean absolute error for

\hat{\theta}^{\prime}

in the CI MPE experiment. The lowest value in each column is highlighted in bold.

Dataset	Negative Class	# Detected CI pairs	MAE of $\hat{\theta}^{\prime}$
Breast Cancer	B	88	$0.0284\pm 0.0248$
Breast Cancer	M	86	$0.0498\pm 0.0612$
DryBean	SEKER	2	$0.0255\pm 0.0252$
DryBean	HOROZ	4	$0.0179\pm 0.0075$

Table 2: Number of detected CI pairs and mean absolute error of

\hat{\theta}^{\prime}

$(\theta,\theta^{\prime})$	$n=n^{\prime}=100$	$500$	$1000$
(1,0.2)	( – , 0.044 $\pm$ 0.037)	( – , 0.019 $\pm$ 0.012)	( – , 0.015 $\pm$ 0.010)
(0.8,0.2)	(0.048 $\pm$ 0.036, 0.044 $\pm$ 0.030)	(0.016 $\pm$ 0.013, 0.020 $\pm$ 0.014)	(0.015 $\pm$ 0.011, 0.014 $\pm$ 0.010)
(0.5,0.2)	(0.077 $\pm$ 0.079, 0.056 $\pm$ 0.048)	(0.031 $\pm$ 0.024, 0.025 $\pm$ 0.019)	(0.025 $\pm$ 0.017, 0.020 $\pm$ 0.014)

Table 3: Mean and standard deviation of the absolute errors for

(\hat{\theta},\hat{\theta}^{\prime})

in the MCI MPE experiment. The first row corresponds to the PU setting, in which only the results for

\theta^{\prime}

are shown.

$\sigma_{12}$	$n=n^{\prime}=500$	$1000$	$2000$
0	0.051	0.055	0.052
0.2	0.399	0.748	0.996
0.5	1	1	1

Table 4: Rejection rates for the kernel CI test (top) and MCI test (bottom) with true mixture proportions. They should be close to the significance level

0.05

under

H_{0}

(

\sigma_{12}

= 0).

$\sigma_{12}$	$n=n^{\prime}=500$	$1000$	$2000$
0	0.041	0.038	0.042
0.2	0.573	0.915	0.994
0.5	1	1	1

Table 5: Rejection rates for the kernel CI test (top) and MCI test (bottom) without true mixture proportions. They should be close to the significance level

0.05

under

H_{0}

(

\sigma_{12}

= 0).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Positive	Negative
Shuttle	1	other classes
Wine	white	red
Dry Bean	DERMASON	other classes

Appendix

Table 6: Positive and Negative classes used for UCI dataset

$\sigma_{12}^{2}$	MAE of $(\hat{\theta},\hat{\theta}^{\prime})$
0	$(0.026\pm 0.018,0.025\pm 0.019)$
0.1	$(0.074\pm 0.034,0.030\pm 0.021)$
0.2	$(0.132\pm 0.025,0.034\pm 0.021)$

Appendix

Table 7: Mean absolute error of

(\hat{\theta},\hat{\theta}^{\prime})

with weakly-violated CI data

$\sigma_{12}^{2}$	$\operatorname{MAE}$ of $(\hat{\theta},\hat{\theta}^{\prime})$
0	$(0.008\pm 0.005,0.015\pm 0.010)$
0.1	$(0.017\pm 0.016,0.014\pm 0.012)$
0.2	$(0.037\pm 0.011,0.010\pm 0.011)$

Appendix

Table 8: Mean absolute error of

(\hat{\theta},\hat{\theta}^{\prime})

with weakly-violated MCI data

Method	Test power
Test with our null approximation	0.199
Test with true null distribution	0.293

Appendix

Table 9: MCI test power: our null approximation vs. true null distribution

Conditional Independence Testing

Conditional independence testing asks whether two variables remain independent after accounting for other variables, supporting causal discovery and valid inference.

See all 19 papers

Rethinking Learning from Label Proportions via Moment Matching
Oct 15, 2026Tianhao Ma, Wei Wang, Yivan Zhang +4Weakly Supervised Learning
The University of Tokyo · RIKEN AIP
Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence
Jun 6, 2026Takafumi Kanamori, Yushi Hirose, Shohei YamamotoLatent Variable Models Parameter Identifiability
Department of Mathematical and Computing Science, Institute of Science Tokyo · RIKEN Center for Advanced Intelligence Project
A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees
Nov 28, 2025Miao Zhang, Junpeng Li, Changchun Hua +1Weakly Supervised Learning Statistical Learning Theory
Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China

Oct 15, 2026J/K move · Enter open · S save

Rethinking Learning from Label Proportions via Moment Matching

Tianhao Ma, Wei Wang, Yivan Zhang, Dong-Dong Wu+3

The University of Tokyo · RIKEN AIP

Weakly Supervised Learning

Open paper AskPDF arXiv

Learning from label proportions (LLP) is a weakly supervised setting where training data are grouped into bags with only bag-level label proportions observed, and the goal is to learn a classifier that predicts labels for individual instances. Recently, a large body of methods has emerged; however, most of them rely on the assumption that instance-label pairs are sampled i.i.d. and randomly assembled into bags, an assumption often violated in real-world scenarios. Our experimental results indicate that these methods perform suboptimally under non-random settings. In this work, we adopt a more realistic assumption: bags are sampled i.i.d., and instance labels are conditionally independent given the instances. Under this setting, we show that the label counts follow a Poisson multinomial distribution. Motivated by this observation, we propose LLP via moment matching (LLP-MM), a simple yet effective approach which leverages multi-order factorial moments as training objectives, encouraging the classifier predictions to match these moments and thereby more fully exploit the underlying statistical structure of the data. Extensive experiments on benchmark datasets under various bag construction strategies demonstrate the effectiveness of our approach while maintaining high computational efficiency. Our code is available at https://github.com/TianhaoMa5/LLP-MM.

Abstract

Conditional Independence Testing

Explore similar work

Rethinking Learning from Label Proportions via Moment Matching

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence

A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees