We consider learning with possibilistic supervision for multi-class classification. For each training instance, the supervision is a normalized possibility distribution that expresses graded plausibility over the classes. From this possibility distribution, we construct a non-empty closed convex set of admissible probability distributions by combining two requirements: probabilistic compatibility with the possibility and necessity measures induced by the possibility distribution, and linear shape constraints that must be satisfied to preserve the qualitative structure of the possibility distribution. Thus, classes with the same possibility degree receive equal probabilities, and if a class has a strictly larger possibility degree than another class, then it receives a strictly larger probability. Given a strictly positive probability vector output by a model for an instance, we compute its Kullback-Leibler projection onto the admissible set. This projection yields the closest admissible probability distribution in Kullback-Leibler sense. We can then train the model by minimizing the divergence between the prediction and its projection, which quantifies the smallest adjustment needed to satisfy the induced dominance and shape constraints. The projection is computed with Dykstra's algorithm using Bregman projections associated with the negative entropy, and we provide explicit formulas for the projections onto each constraint set. Experiments conducted on synthetic data and on a real-world natural language inference task, based on the ChaosNLI dataset, show that the proposed projection algorithm is efficient enough for practical use, and that the resulting projection-based learning objective can improve predictive performance.
Figures & tables
Figure 1: Training with possibilistic supervision via KL projection onto Fbox .
Tolerance τ
Convergence rate
Mean cycles
90th perc. cycles
Mean final violation
Mean time [s]
1e−02
1.000
1.1
1.0
6.04e-03
0.000
1e−03
1.000
11.9
16.1
9.42e-04
0.004
1e−04
1.000
143.9
215.4
9.93e-05
0.050
1e−06
0.360
881.9
1000.0
6.22e-06
0.309
1e−08
0.080
973.9
1000.0
6.24e-06
0.341
Table 1: KL projection on Fbox for n=100 , with a maximum of Kmax=1000 Dykstra cycles and 100 random runs per tolerance.
Tolerance τ
Convergence rate
Mean cycles
90th perc. cycles
Mean final violation
Mean time [s]
1e−02
1.000
1.1
1.0
6.04e-03
0.000
1e−03
1.000
11.9
16.1
9.42e-04
0.004
1e−04
1.000
143.9
215.4
9.93e-05
0.050
1e−06
1.000
1593.5
2971.6
9.98e-07
0.558
1e−08
0.970
2934.1
5077.5
1.18e-08
1.028
Table 2: KL projection on Fbox for n=100 , with a maximum of Kmax=10000 Dykstra cycles and 100 random runs per tolerance.
Tolerance τ
Convergence rate
Mean cycles
90th perc. cycles
Mean final violation
Mean time [s]
1e−02
1.000
1.1
1.0
6.04e-03
0.000
1e−03
1.000
11.9
16.1
9.42e-04
0.004
1e−04
1.000
143.9
215.4
9.93e-05
0.050
1e−06
1.000
1593.5
2971.6
9.98e-07
0.558
1e−08
1.000
3047.9
5077.5
9.97e-09
1.068
Table 3: KL projection on Fbox for n=100 , with a maximum of Kmax=50000 Dykstra cycles and 100 random runs per tolerance.
Symbol
Meaning and values used in the experiments
Dataset and dimensions
n
Number of classes; fixed to n=20 .
Y
Class set {1,…,n} .
d
Dimension of the input vector x ; d∈{30,80,150} .
Ntr
Training set size; Ntr∈{200,500,1000} .
Nte
Test set size; fixed to Nte=3000 .
Table 4: Notation and values for the synthetic data generation.
d
β
α
Ntr
lrA
lrB
AccAtr
AccBtr
AccAte
AccBte
30
1.5
0.4
200
0.01
0.0008
0.9960±0.0039
0.9865±0.0094
0.9118±0.0184
0.8689±0.0173
30
1.5
0.4
500
0.03
0.002
0.9944±0.0034
0.9838±0.0096
0.9258±0.0161
0.9055±0.0167
30
1.5
0.4
1000
0.006
0.001
0.9890±0.0075
0.9711±0.0115
0.9277±0.0150
0.9287±0.0136
30
1.5
0.6
200
0.02
0.0006
0.9900±0.0094
0.9760±0.0165
0.9286±0.0155
0.8629±0.0172
30
1.5
0.6
500
0.03
0.001
0.9874±0.0065
0.9704±0.0123
0.9353±0.0127
0.9096±0.0159
30
1.5
0.6
1000
0.01
0.0006
0.9772±0.0110
0.9623±0.0120
0.9384±0.0143
0.9291±0.0136
Table 5: Top- 1 accuracies on the synthetic learning task, for Models A (projection target) and B (fixed target). Each row corresponds to one configuration (d,β,α,Ntr) , with all parameters fixed as in Table 4 . For each model, lrA and lrB are the learning rates selected by validation grid search. Acctrain and Acctest denote the training and test top- 1 accuracies, reported as mean ± standard deviation over 10 independent runs. The best test accuracy between Models A and B is shown in bold.
Train section
Val section
Test section
lrA
lrB
lrC
AccA
AccB
AccC
ΔA−B
ΔA−C
train_full
val_full
test_full
0.007
0.007
0.005
0.500±0.011
0.496±0.008
0.505±0.008
+0.004
−0.005
test_S_amb
0.007
0.007
0.005
0.337±0.014
0.343±0.024
0.358±0.027
−0.006
−0.022
test_S_easy
0.007
0.007
0.005
0.616±0.018
0.604±0.026
0.608±0.011
+0.012
+0.008
val_S_amb
test_full
0.5
0.6
0.0001
0.449±0.022
0.447±0.033
0.456±0.012
+0.002
−0.007
test_S_amb
0.5
0.6
0.0001
0.400±0.011
0.391±0.023
0.373±0.052
+0.009
+0.027
test_S_easy
0.5
0.6
0.0001
0.492±0.027
0.503±0.050
0.516±0.028
−0.011
−0.024
Table 6: Top-1 accuracy on ChaosNLI when the training set is restricted to train_full , train_S_amb , or train_S_easy , for Models A (projection target), B (antipignistic target), and C (vote-proportion target). The selected learning rates for each target are reported alongside the accuracies. Results are organized by training section, validation section, and test section. Accuracies are mean ± standard deviation over paired runs, and the best mean accuracy in each row is shown in bold.
This paper proposes a novel loss concept for supervised classification tasks. Rather than enforcing a direct mapping from each input sample to a single assigned label, we define an optimization objective over all classifier outputs as a bimodal Gaussian distribution. This softer target formulation implicitly captures class ambiguity, mitigates overfitting, and encourages the learning of more robust decision boundaries, all without requiring additional label information. Experimental results demonstrate consistent improvements in robustness, with particularly pronounced gains in low-data regimes, while requiring only minimal modifications to standard training pipelines.
Kathleen Anderson, Thomas Martinetz
Institute for Neuro- and Bioinformatics, University of Lübeck, Germany
Imprecise annotations may support several plausible labelling distributions, yet learning methods often resolve this ambiguity into a single predictive distribution. This can obscure what the annotation evidence leaves unresolved. We introduce epistemic learning from credal supervision, a framework that uses convex sets of plausible labelling distributions, called credal sets, as supervision and learns sets of predictive distributions. We instantiate the framework with the pessimistic--optimistic credal classifier (POCC), which combines a shared backbone with two classification heads trained to minimise worst-case and best-case losses over the supervision sets. Their outputs define a predictive credal set whose spread provides an uncertainty score. We also show how credal labels can be obtained through a simple relaxation of existing probabilistic labels, reducing commitment to their precise probability assignments. This construction admits closed-form inner optimisation under cross-entropy loss, enabling efficient training. Assuming the supervision sets contain the true conditional label distributions, and other regularity assumptions, we establish a finite-sample generalisation bound for the averaged predictor with an explicit penalty for supervision imprecision. We evaluate POCC using human annotator disagreement and teacher predictions, alongside label smoothing as a controlled proxy for annotation imprecision. Across these settings, POCC achieves a favourable balance of predictive accuracy, calibration, and uncertainty-based selective classification versus competitive baselines.
Kaizheng Wang, Siu Lun Chau
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore
Learning from label proportions (LLP) is a weakly supervised setting where training data are grouped into bags with only bag-level label proportions observed, and the goal is to learn a classifier that predicts labels for individual instances. Recently, a large body of methods has emerged; however, most of them rely on the assumption that instance-label pairs are sampled i.i.d. and randomly assembled into bags, an assumption often violated in real-world scenarios. Our experimental results indicate that these methods perform suboptimally under non-random settings. In this work, we adopt a more realistic assumption: bags are sampled i.i.d., and instance labels are conditionally independent given the instances. Under this setting, we show that the label counts follow a Poisson multinomial distribution. Motivated by this observation, we propose LLP via moment matching (LLP-MM), a simple yet effective approach which leverages multi-order factorial moments as training objectives, encouraging the classifier predictions to match these moments and thereby more fully exploit the underlying statistical structure of the data. Extensive experiments on benchmark datasets under various bag construction strategies demonstrate the effectiveness of our approach while maintaining high computational efficiency. Our code is available at https://github.com/TianhaoMa5/LLP-MM.