Rethinking Learning from Label Proportions via Moment Matching

Tianhao Ma; Wei Wang; Yivan Zhang; Dong-Dong Wu; Takashi Ishida; Gang Niu; Masashi Sugiyama

cs.LGOct 15, 2026

Rethinking Learning from Label Proportions via Moment Matching

Authors: Tianhao Ma, Wei Wang, Yivan Zhang, Dong-Dong Wu, Takashi Ishida, Gang Niu, Masashi Sugiyama

Organizations: The University of Tokyo · RIKEN AIP

Abstract

Learning from label proportions (LLP) is a weakly supervised setting where training data are grouped into bags with only bag-level label proportions observed, and the goal is to learn a classifier that predicts labels for individual instances. Recently, a large body of methods has emerged; however, most of them rely on the assumption that instance-label pairs are sampled i.i.d. and randomly assembled into bags, an assumption often violated in real-world scenarios. Our experimental results indicate that these methods perform suboptimally under non-random settings. In this work, we adopt a more realistic assumption: bags are sampled i.i.d., and instance labels are conditionally independent given the instances. Under this setting, we show that the label counts follow a Poisson multinomial distribution. Motivated by this observation, we propose LLP via moment matching (LLP-MM), a simple yet effective approach which leverages multi-order factorial moments as training objectives, encouraging the classifier predictions to match these moments and thereby more fully exploit the underlying statistical structure of the data. Extensive experiments on benchmark datasets under various bag construction strategies demonstrate the effectiveness of our approach while maintaining high computational efficiency. Our code is available at https://github.com/TianhaoMa5/LLP-MM.

Figures & tables

Figure 1: An illustration of structured bag construction based on instance features. A common example of a structured-bag assumption is to group instances with similar feature patterns into the same bag; for example, as shown in the figure, instances sharing the same shape feature are assigned to one bag.

Dataset	Method	16	32	64	128	Average	Rank
Random Bag
CIFAR-10	PM	93.51 $\pm$ 0.11	90.05 $\pm$ 0.22	75.23 $\pm$ 0.70	60.02 $\pm$ 0.33	79.70	4
	DSQ	93.52 $\pm$ 0.15	91.39 $\pm$ 0.16	79.86 $\pm$ 0.77	61.90 $\pm$ 1.35	81.67	3
	LLP-PVC	91.95 $\pm$ 0.07	90.55 $\pm$ 0.08	83.79 $\pm$ 0.57	64.44 $\pm$ 1.45	82.68	2
	LLP-FC	75.29 $\pm$ 0.72	61.06 $\pm$ 0.87	53.89 $\pm$ 0.61	60.01 $\pm$ 0.24	62.56	7
	ROT	93.21 $\pm$ 0.09	92.04 $\pm$ 0.10	80.52 $\pm$ 0.49	35.03 $\pm$ 1.25	75.20	5

Table 1: Best test accuracy under Random Bag and Cluster Bag constructions (mean

\pm

std over three runs, %). The best result for each bag size and average is bold. Ranks use average accuracy with standard competition ranking.

\dagger

indicates severe overfitting caused by negative loss values, with predictions collapsing to a single class.

Dataset	Method	16	32	64	128	Average	Rank
CIFAR-10	PM	91.00 $\pm$ 0.40	70.37 $\pm$ 0.32	65.38 $\pm$ 1.14	61.35 $\pm$ 0.52	72.03	4
	DSQ	90.98 $\pm$ 0.16	74.24 $\pm$ 0.57	73.78 $\pm$ 1.11	68.83 $\pm$ 1.43	76.96	3
	LLP-PVC	91.85 $\pm$ 0.13	89.18 $\pm$ 0.11	81.44 $\pm$ 0.61	66.78 $\pm$ 0.54	82.31	2
	LLP-FC	58.53 $\pm$ 0.36	49.40 $\pm$ 0.64	49.00 $\pm$ 0.42	64.55 $\pm$ 1.15	55.37	7
	ROT	92.93 $\pm$ 0.07	90.71 $\pm$ 0.11	61.53 $\pm$ 3.70	38.24 $\pm$ 1.22	70.85	5
	EasyLLP	47.72 $\pm$ 1.40	37.40 $\pm$ 0.61	28.82 $\pm$ 2.44	24.09 $\pm$ 1.58	34.51	10

Table 2: Best test accuracy under

{\bm{\alpha}}

-First Bag construction (mean

\pm

std over three runs, %). The best result for each bag size and average is bold. Ranks use average accuracy with standard competition ranking.

Method	Accuracy $\uparrow$	Macro-F1 $\uparrow$	Balanced Acc. $\uparrow$	Weighted-F1 $\uparrow$
PM	$77.35\pm 13.10$	$29.11\pm 10.74$	$32.70\pm 10.01$	$74.46\pm 15.73$
DSQ	$92.39\pm 1.33$	$56.89\pm 7.65$	$60.08\pm 8.30$	$92.70\pm 1.57$
LLP-PVC	$92.82\pm 1.94$	$58.42\pm 4.31$	$61.63\pm 4.27$	$91.58\pm 2.33$
LLP-FC	$75.11\pm 9.57$	$26.26\pm 4.28$	$27.12\pm 4.46$	$72.46\pm 11.60$
ROT	$\mathbf{94.05\pm 0.65}$	$56.11\pm 7.56$	$56.82\pm 7.74$	$93.35\pm 1.17$
EasyLLP	$37.98\pm 8.32$	$11.51\pm 1.77$	$14.50\pm 0.85$	$39.28\pm 3.11$

Table 3: Test performance on KU-Optofil PBC (mean

\pm

std over three runs, %). The best result in each column is bold.

Figure 2: Effect of the order parameter on prediction performance and runtime per epoch across different bag construction strategies and bag sizes. Runtime was measured on a single NVIDIA A100-SXM4-40GB GPU with an AMD EPYC 7742 64-Core Processor.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
${\mathcal{X}}$	Input space.
${\mathcal{Y}}=\{1,\dots,q\}$	Label space.
$q$	Number of classes.
$m$	Bag size.
$\Delta^{q-1}$	The $(q-1)$ -dimensional probability simplex $\left\{(p_{1},\ldots,p_{q})\in\mathbb{R}^{q}\;\middle\|\;p_{k}\geq 0,\ \sum_{k=1}^{q}p_{k}=1\right\}$ .
$x_{j}\in{\mathcal{X}}$	The $j$ -th instance in a bag.

Appendix

Table 4: Notation used in the paper.

Abbreviation	Meaning
WSL	Weakly supervised learning
LLP	Learning from label proportions
PMD	Poisson multinomial distribution
MM	Moment matching
LLP-MM	LLP via Moment Matching

Appendix

Table 5: Abbreviations used in the paper.

Bag	Method	16	32	64	128
Random	LLP-GAN	$92.96\pm 0.09$	$88.96\pm 0.35$	$78.07\pm 1.31$	$58.14\pm 2.94$
Random	MM+GAN	$\mathbf{93.56\pm 0.19}$	$\mathbf{91.18\pm 0.22}$	$\mathbf{80.69\pm 1.17}$	$\mathbf{67.41\pm 1.41}$
Cluster	LLP-GAN	$92.81\pm 0.15$	$90.37\pm 0.16$	$85.16\pm 0.77$	$76.27\pm 0.91$
Cluster	MM+GAN	$\mathbf{93.62\pm 0.05}$	$\mathbf{91.79\pm 0.16}$	$\mathbf{87.17\pm 0.38}$	$\mathbf{76.53\pm 1.93}$
$\alpha$ -First	LLP-GAN	$91.04\pm 0.59$	$74.75\pm 0.55$	$69.52\pm 4.12$	$66.25\pm 2.23$
$\alpha$ -First	MM+GAN	$\mathbf{93.39\pm 0.08}$	$\mathbf{88.12\pm 1.09}$	$\mathbf{77.09\pm 4.17}$	$\mathbf{73.13\pm 5.62}$

Appendix

Table 6: Best test accuracy on CIFAR-10 with GAN-based representation learning (mean

\pm

std over three runs, %). The best result in each paired comparison is bold.

Weakly Supervised Learning

Weakly supervised learning trains models from incomplete, noisy, coarse, or indirect labels instead of dense ground truth.

See all 156 papers

Learning from Label Proportions with Dual-proportion Constraints
Mar 22, 2026Tianhao Ma, Ximing Li, Changchun Li +1Weakly Supervised Learning
The University of Tokyo, Japan · College of Computer Science and Technology, Jilin University, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China +1
Learnability with Partial Labels and Adaptive Nearest Neighbors
Mar 16, 2026Nicolas A. Errandonea, Santiago Mazuelas, Jose A. Lozano +1Statistical Learning Theory k-Nearest Neighbors
Basque Center for Applied Mathematics, Bilbao, Spain · Ikerbasque, Basque Foundation for Science · University of the Basque Country, Spain +1
Leveraging Label Proportion Prior for Class-Imbalanced Semi-Supervised Learning
Mar 3, 2026Kohki Akiba, Shinnosuke Matsuo, Shota Harada +1Semi-Supervised Learning Class-Imbalanced Learning
Kyushu University, Fukuoka, Japan

Mar 22, 2026J/K move · Enter open · S save

Learning from Label Proportions with Dual-proportion Constraints

Tianhao Ma, Ximing Li, Changchun Li, Renchu Guan

The University of Tokyo, Japan · College of Computer Science and Technology, Jilin University, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, China+1

Weakly Supervised Learning

Open paper AskPDF arXiv

Learning from Label Proportions (LLP) is a weakly supervised problem in which the training data comprise bags, that is, groups of instances, each annotated only with bag-level class label proportions, and the objective is to learn a classifier that predicts instance-level labels. This setting is widely applicable when privacy constraints limit access to instance-level annotations or when fine-grained labeling is costly or impractical. In this work, we introduce a method that leverages Dual proportion Constraints (LLP-DC) during training, enforcing them at both the bag and instance levels. Specifically, the bag-level training aligns the mean prediction with the given proportion, and the instance-level training aligns hard pseudo-labels that satisfy the proportion constraint, where a minimum-cost maximum-flow algorithm is used to generate hard pseudo-labels. Extensive experimental results across various benchmark datasets empirically validate that LLP-DC consistently improves over previous LLP methods across datasets and bag sizes. The code is publicly available at https://github.com/TianhaoMa5/CV PR2026_Findings_LLP_DC.

Abstract

Weakly Supervised Learning

Explore similar work

Learning from Label Proportions with Dual-proportion Constraints

Learnability with Partial Labels and Adaptive Nearest Neighbors

Leveraging Label Proportion Prior for Class-Imbalanced Semi-Supervised Learning