Probabilistic classification from possibilistic data: computing Kullback-Leibler projection with a possibility distribution

Ismaïl Baaj; Pierre Marquis

cs.AIApr 2, 2026

Probabilistic classification from possibilistic data: computing Kullback-Leibler projection with a possibility distribution

Authors: Ismaïl Baaj, Pierre Marquis

Organizations: LEMMA, Paris-Panthéon-Assas University, Paris, France · Univ. Artois, CNRS, CRIL, Institut Universitaire de France, Lens, France

Abstract

We consider learning with possibilistic supervision for multi-class classification. For each training instance, the supervision is a normalized possibility distribution that expresses graded plausibility over the classes. From this possibility distribution, we construct a non-empty closed convex set of admissible probability distributions by combining two requirements: probabilistic compatibility with the possibility and necessity measures induced by the possibility distribution, and linear shape constraints that must be satisfied to preserve the qualitative structure of the possibility distribution. Thus, classes with the same possibility degree receive equal probabilities, and if a class has a strictly larger possibility degree than another class, then it receives a strictly larger probability. Given a strictly positive probability vector output by a model for an instance, we compute its Kullback-Leibler projection onto the admissible set. This projection yields the closest admissible probability distribution in Kullback-Leibler sense. We can then train the model by minimizing the divergence between the prediction and its projection, which quantifies the smallest adjustment needed to satisfy the induced dominance and shape constraints. The projection is computed with Dykstra's algorithm using Bregman projections associated with the negative entropy, and we provide explicit formulas for the projections onto each constraint set. Experiments conducted on synthetic data and on a real-world natural language inference task, based on the ChaosNLI dataset, show that the proposed projection algorithm is efficient enough for practical use, and that the resulting projection-based learning objective can improve predictive performance.

Figures & tables

Panel of Figure 1 — Figure 1: Training with possibilistic supervision via KL projection onto $\mathcal{F}^{\mathrm{box}}$ .

Tolerance $\tau$	Convergence rate	Mean cycles	90th perc. cycles	Mean final violation	Mean time [s]
$1e-02$	1.000	1.1	1.0	6.04e-03	0.000
$1e-03$	1.000	11.9	16.1	9.42e-04	0.004
$1e-04$	1.000	143.9	215.4	9.93e-05	0.050
$1e-06$	0.360	881.9	1000.0	6.22e-06	0.309
$1e-08$	0.080	973.9	1000.0	6.24e-06	0.341

Table 1: KL projection on

\mathcal{F}^{\mathrm{box}}

for

n=100

, with a maximum of

K_{\max}=1000

Dykstra cycles and 100 random runs per tolerance.

Tolerance $\tau$	Convergence rate	Mean cycles	90th perc. cycles	Mean final violation	Mean time [s]
$1e-02$	1.000	1.1	1.0	6.04e-03	0.000
$1e-03$	1.000	11.9	16.1	9.42e-04	0.004
$1e-04$	1.000	143.9	215.4	9.93e-05	0.050
$1e-06$	1.000	1593.5	2971.6	9.98e-07	0.558
$1e-08$	0.970	2934.1	5077.5	1.18e-08	1.028

Table 2: KL projection on

\mathcal{F}^{\mathrm{box}}

for

n=100

, with a maximum of

K_{\max}=10000

Dykstra cycles and 100 random runs per tolerance.

Tolerance $\tau$	Convergence rate	Mean cycles	90th perc. cycles	Mean final violation	Mean time [s]
$1e-02$	1.000	1.1	1.0	6.04e-03	0.000
$1e-03$	1.000	11.9	16.1	9.42e-04	0.004
$1e-04$	1.000	143.9	215.4	9.93e-05	0.050
$1e-06$	1.000	1593.5	2971.6	9.98e-07	0.558
$1e-08$	1.000	3047.9	5077.5	9.97e-09	1.068

Table 3: KL projection on

\mathcal{F}^{\mathrm{box}}

for

n=100

, with a maximum of

K_{\max}=50000

Dykstra cycles and 100 random runs per tolerance.

Symbol	Meaning and values used in the experiments
Dataset and dimensions
$n$	Number of classes; fixed to $n=20$ .
$\mathcal{Y}$	Class set $\{1,\dots,n\}$ .
$d$	Dimension of the input vector $x$ ; $d\in\{30,80,150\}$ .
$N_{\mathrm{tr}}$	Training set size; $N_{\mathrm{tr}}\in\{200,500,1000\}$ .
$N_{\mathrm{te}}$	Test set size; fixed to $N_{\mathrm{te}}=3000$ .

Table 4: Notation and values for the synthetic data generation.

$d$	$\beta$	$\alpha$	$N_{\mathrm{tr}}$	$\mathrm{lr}_{A}$	$\mathrm{lr}_{B}$	$\mathrm{Acc}_{A}^{\mathrm{tr}}$	$\mathrm{Acc}_{B}^{\mathrm{tr}}$	$\mathrm{Acc}_{A}^{\mathrm{te}}$	$\mathrm{Acc}_{B}^{\mathrm{te}}$
30	1.5	0.4	200	0.01	0.0008	$0.9960\pm 0.0039$	$0.9865\pm 0.0094$	$\bm{0.9118\pm 0.0184}$	$0.8689\pm 0.0173$
30	1.5	0.4	500	0.03	0.002	$0.9944\pm 0.0034$	$0.9838\pm 0.0096$	$\bm{0.9258\pm 0.0161}$	$0.9055\pm 0.0167$
30	1.5	0.4	1000	0.006	0.001	$0.9890\pm 0.0075$	$0.9711\pm 0.0115$	$0.9277\pm 0.0150$	$\bm{0.9287\pm 0.0136}$
30	1.5	0.6	200	0.02	0.0006	$0.9900\pm 0.0094$	$0.9760\pm 0.0165$	$\bm{0.9286\pm 0.0155}$	$0.8629\pm 0.0172$
30	1.5	0.6	500	0.03	0.001	$0.9874\pm 0.0065$	$0.9704\pm 0.0123$	$\bm{0.9353\pm 0.0127}$	$0.9096\pm 0.0159$
30	1.5	0.6	1000	0.01	0.0006	$0.9772\pm 0.0110$	$0.9623\pm 0.0120$	$\bm{0.9384\pm 0.0143}$	$0.9291\pm 0.0136$

Table 5: Top-

1

accuracies on the synthetic learning task, for Models A (projection target) and B (fixed target). Each row corresponds to one configuration

(d,\beta,\alpha,N_{\mathrm{tr}})

, with all parameters fixed as in Table 4 . For each model,

\mathrm{lr}_{A}

and

\mathrm{lr}_{B}

are the learning rates selected by validation grid search.

\mathrm{Acc}^{\mathrm{train}}

and

\mathrm{Acc}^{\mathrm{test}}

denote the training and test top-

1

accuracies, reported as mean

\pm

standard deviation over

10

independent runs. The best test accuracy between Models A and B is shown in bold.

Train section	Val section	Test section	$\mathrm{lr}_{A}$	$\mathrm{lr}_{B}$	$\mathrm{lr}_{C}$	$\mathrm{Acc}_{A}$	$\mathrm{Acc}_{B}$	$\mathrm{Acc}_{C}$	$\Delta_{A-B}$	$\Delta_{A-C}$
train_full	val_full	test_full	0.007	0.007	0.005	$0.500\pm 0.011$	$0.496\pm 0.008$	$\mathbf{0.505\pm 0.008}$	$+0.004$	$-0.005$
		test_S_amb	0.007	0.007	0.005	$0.337\pm 0.014$	$0.343\pm 0.024$	$\mathbf{0.358\pm 0.027}$	$-0.006$	$-0.022$
		test_S_easy	0.007	0.007	0.005	$\mathbf{0.616\pm 0.018}$	$0.604\pm 0.026$	$0.608\pm 0.011$	$+0.012$	$+0.008$
	val_S_amb	test_full	0.5	0.6	0.0001	$0.449\pm 0.022$	$0.447\pm 0.033$	$\mathbf{0.456\pm 0.012}$	$+0.002$	$-0.007$
		test_S_amb	0.5	0.6	0.0001	$\mathbf{0.400\pm 0.011}$	$0.391\pm 0.023$	$0.373\pm 0.052$	$+0.009$	$+0.027$
		test_S_easy	0.5	0.6	0.0001	$0.492\pm 0.027$	$0.503\pm 0.050$	$\mathbf{0.516\pm 0.028}$	$-0.011$	$-0.024$

Table 6: Top-1 accuracy on ChaosNLI when the training set is restricted to train_full , train_S_amb , or train_S_easy , for Models A (projection target), B (antipignistic target), and C (vote-proportion target). The selected learning rates for each target are reported alongside the accuracies. Results are organized by training section, validation section, and test section. Accuracies are mean

\pm

standard deviation over paired runs, and the best mean accuracy in each row is shown in bold.

Soft-Label Learning

Soft-label learning trains models on probability distributions or uncertain targets rather than a single class label.

See all 19 papers

Distributional Loss for Robust Classification
Jun 11, 2026Kathleen Anderson, Thomas MartinetzRobust Loss Functions Classification
Institute for Neuro- and Bioinformatics, University of Lübeck, Germany
Epistemic Learning from Imprecise Annotation
Sep 28, 2026Kaizheng Wang, Siu Lun ChauSoft-Label Learning Epistemic Uncertainty
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore
Rethinking Learning from Label Proportions via Moment Matching
Oct 15, 2026Tianhao Ma, Wei Wang, Yivan Zhang +4Weakly Supervised Learning
The University of Tokyo · RIKEN AIP

Jun 11, 2026J/K move · Enter open · S save

Distributional Loss for Robust Classification

Kathleen Anderson, Thomas Martinetz

Institute for Neuro- and Bioinformatics, University of Lübeck, Germany

Robust Loss Functions Classification Robust Classification

Open paper AskPDF arXiv

This paper proposes a novel loss concept for supervised classification tasks. Rather than enforcing a direct mapping from each input sample to a single assigned label, we define an optimization objective over all classifier outputs as a bimodal Gaussian distribution. This softer target formulation implicitly captures class ambiguity, mitigates overfitting, and encourages the learning of more robust decision boundaries, all without requiring additional label information. Experimental results demonstrate consistent improvements in robustness, with particularly pronounced gains in low-data regimes, while requiring only minimal modifications to standard training pipelines.

Abstract

Soft-Label Learning

Explore similar work

Distributional Loss for Robust Classification

Epistemic Learning from Imprecise Annotation

Rethinking Learning from Label Proportions via Moment Matching