MORE-PLR: multi-output regression employed for partial label ranking

Santo M. A. R. Thies; Juan C. Alfaro; Viktor Bengs

cs.LGSep 24, 2026

MORE-PLR: multi-output regression employed for partial label ranking

Authors: Santo M. A. R. Thies, Juan C. Alfaro, Viktor Bengs

Organizations: Chair of Artificial Intelligence and Machine Learning, Ludwig-Maximilians-Universität München, Munich, 80799, Germany. · Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, Albacete, 02071, Spain. · Laboratorio de Sistemas Inteligentes y Minería de Datos, Universidad de Castilla-La Mancha, Albacete, 02071, Spain. · Munich Center for Machine Learning, Munich, Germany.

Abstract

The partial label ranking problem is a supervised learning scenario that aims to fit a preference model that predicts a bucket order defined over a set of labels for a given input instance. This problem generalizes the well-known label ranking problem, which, in practice, is limited to outputting total orders of labels. Existing partial label ranking methods have primarily extended label ranking approaches to handle ties in predictions. This paper proposes using multi-output regression to address the partial label ranking problem, introducing an encoder that, during the learning phase, transforms the (possibly incomplete) rankings with ties of labels to multivariate regression targets, an underexplored perspective in both label ranking and partial label ranking. Moreover, during the inference phase, we introduce several post-hoc layers that convert the multi-output regression results into the output bucket order to effectively implement this approach. This framework provides learning strategies that are competitive with the current state-of-the-art partial label ranking methods, as demonstrated through experimental evaluations.

Figures & tables

Panel of Figure 1 — Figure 1 : Given a prompt, the LLMgenerates multiple responses, which are ranked by a human labeler from worst to best. This ranking is then used to fine-tune the LLM.

Panel of Figure 2 — Figure 2 : In this example, the goal is to predict a bucket order of vacation destinations for a customer based on their demographic information.

Panel of Figure 3 — Figure 3 : Impact of encoding variants on algorithms’ averaged accuracy across all datasets

Figure 4 : Impact of

q

in the accuracy of the algorithms

Figure 5 : Impact of

\epsilon

on the accuracy of the algorithms

LR
Missing percentage: 0%
Friedman p-value: $\bf 5.136\times 10^{-7}$
Holm results
Method	Rank	P-value	Win	Tie	Loss
ST-PI	2.27	-	-	-	-
Native-PI	4.42	$\bf 1.146\times 10^{-1}$	10	0	3

Table 1 : Friedman’s and Holm’s tests for accuracy with varying missing percentage

Problem	Dataset	RPC	ST-RR	ST-PI	ST-EPS	Chain-RR	Chain-PI	Chain-EPS	Native-RR	Native-PI	Native-EPS
LR	authorship	0.637 $\pm$ 0.141	0.825 $\pm$ 0.137	0.872 $\pm$ 0.299	0.915 $\pm$ 0.147	1.420 $\pm$ 0.008	1.396 $\pm$ 0.009	1.416 $\pm$ 0.007	0.537 $\pm$ 0.014	0.532 $\pm$ 0.017	0.534 $\pm$ 0.013
	glass	0.604 $\pm$ 0.019	0.593 $\pm$ 0.015	0.578 $\pm$ 0.015	0.666 $\pm$ 0.018	1.207 $\pm$ 0.015	1.198 $\pm$ 0.010	1.218 $\pm$ 0.009	0.462 $\pm$ 0.014	0.480 $\pm$ 0.013	0.458 $\pm$ 0.016
	iris	0.429 $\pm$ 0.045	0.538 $\pm$ 0.013	0.523 $\pm$ 0.012	0.536 $\pm$ 0.019	0.860 $\pm$ 0.008	0.847 $\pm$ 0.009	0.865 $\pm$ 0.007	0.483 $\pm$ 0.007	0.482 $\pm$ 0.012	0.475 $\pm$ 0.014
	letter	7.808 $\pm$ 0.042	5.364 $\pm$ 0.068	5.203 $\pm$ 0.022	5.490 $\pm$ 0.040	6.503 $\pm$ 0.019	6.552 $\pm$ 0.019	6.625 $\pm$ 0.034	1.413 $\pm$ 0.013	1.262 $\pm$ 0.023	1.282 $\pm$ 0.011
	libras	1.654 $\pm$ 0.021	1.568 $\pm$ 0.336	1.500 $\pm$ 0.009	1.537 $\pm$ 0.012	2.445 $\pm$ 0.012	2.382 $\pm$ 0.017	2.464 $\pm$ 0.013	0.570 $\pm$ 0.011	0.549 $\pm$ 0.009	0.559 $\pm$ 0.010
	movies	1.573 $\pm$ 0.067	0.870 $\pm$ 0.010	0.864 $\pm$ 0.011	0.929 $\pm$ 0.019	2.067 $\pm$ 0.025	2.013 $\pm$ 0.014	2.060 $\pm$ 0.012	0.456 $\pm$ 0.012	0.466 $\pm$ 0.013	0.454 $\pm$ 0.011

Table 2 : Time results (in seconds) for complete rankings

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Problem	Name	Identifier	#of instances	#of features	#of labels	#of rankings	$\overline{\mbox{\#of buckets}}$
LR	authorship	42834	841	70	4	17	4.0
	glass	42847	214	9	6	30	6.0
	iris	42851	150	4	3	5	3.0
	letter	45727	20000	16	26	17806	26.0
	libras	45736	360	90	15	317	15.0
	pendigits	42856	10992	16	10	2081	10.0

Appendix

Table 3 : Datasets used for the experiments. The algae, movies, and political datasets correspond to real-world problems.

Problem	Dataset	RPC	ST-RR	ST-PI	ST-EPS	Chain-RR	Chain-PI	Chain-EPS	Native-RR	Native-PI	Native-EPS
LR	authorship	0.887 $\pm$ 0.019	0.917 $\pm$ 0.018	0.925 $\pm$ 0.019	0.923 $\pm$ 0.019	0.916 $\pm$ 0.020	0.919 $\pm$ 0.019	0.919 $\pm$ 0.020	0.914 $\pm$ 0.019	0.918 $\pm$ 0.020	0.917 $\pm$ 0.020
	glass	0.856 $\pm$ 0.047	0.893 $\pm$ 0.036	0.901 $\pm$ 0.037	0.898 $\pm$ 0.036	0.883 $\pm$ 0.041	0.890 $\pm$ 0.041	0.889 $\pm$ 0.042	0.902 $\pm$ 0.032	0.907 $\pm$ 0.031	0.904 $\pm$ 0.030
	iris	0.948 $\pm$ 0.037	0.962 $\pm$ 0.033	0.965 $\pm$ 0.031	0.965 $\pm$ 0.032	0.963 $\pm$ 0.034	0.962 $\pm$ 0.036	0.962 $\pm$ 0.036	0.968 $\pm$ 0.032	0.969 $\pm$ 0.030	0.969 $\pm$ 0.030
	letter	0.902 $\pm$ 0.002	0.940 $\pm$ 0.001	0.942 $\pm$ 0.001	0.703 $\pm$ 0.005	0.932 $\pm$ 0.001	0.934 $\pm$ 0.001	0.675 $\pm$ 0.005	0.919 $\pm$ 0.001	0.919 $\pm$ 0.002	0.686 $\pm$ 0.005
	libras	0.856 $\pm$ 0.017	0.898 $\pm$ 0.013	0.905 $\pm$ 0.012	0.874 $\pm$ 0.012	0.898 $\pm$ 0.014	0.904 $\pm$ 0.014	0.876 $\pm$ 0.014	0.883 $\pm$ 0.016	0.889 $\pm$ 0.015	0.861 $\pm$ 0.014
	movies	0.218 $\pm$ 0.027	0.359 $\pm$ 0.033	0.350 $\pm$ 0.032	0.354 $\pm$ 0.033	0.364 $\pm$ 0.042	0.359 $\pm$ 0.041	0.359 $\pm$ 0.040	0.370 $\pm$ 0.033	0.358 $\pm$ 0.033	0.366 $\pm$ 0.033

Appendix

Table 4 : Accuracy results for complete rankings

Problem	Dataset	RPC	ST-RR	ST-PI	ST-EPS	Chain-RR	Chain-PI	Chain-EPS	Native-RR	Native-PI	Native-EPS
LR	authorship	0.866 $\pm$ 0.025	0.909 $\pm$ 0.021	0.919 $\pm$ 0.020	0.917 $\pm$ 0.019	0.726 $\pm$ 0.029	0.807 $\pm$ 0.025	0.851 $\pm$ 0.025	0.728 $\pm$ 0.026	0.817 $\pm$ 0.021	0.854 $\pm$ 0.023
	glass	0.823 $\pm$ 0.053	0.885 $\pm$ 0.035	0.893 $\pm$ 0.039	0.891 $\pm$ 0.038	0.602 $\pm$ 0.060	0.617 $\pm$ 0.060	0.643 $\pm$ 0.061	0.625 $\pm$ 0.056	0.635 $\pm$ 0.059	0.668 $\pm$ 0.060
	iris	0.927 $\pm$ 0.042	0.955 $\pm$ 0.042	0.964 $\pm$ 0.038	0.963 $\pm$ 0.038	0.511 $\pm$ 0.113	0.589 $\pm$ 0.112	0.599 $\pm$ 0.117	0.517 $\pm$ 0.099	0.591 $\pm$ 0.115	0.616 $\pm$ 0.110
	letter	0.879 $\pm$ 0.002	0.934 $\pm$ 0.001	0.936 $\pm$ 0.001	0.734 $\pm$ 0.005	0.802 $\pm$ 0.002	0.650 $\pm$ 0.004	0.718 $\pm$ 0.002	0.722 $\pm$ 0.003	0.611 $\pm$ 0.003	0.629 $\pm$ 0.003
	libras	0.784 $\pm$ 0.022	0.878 $\pm$ 0.014	0.885 $\pm$ 0.015	0.855 $\pm$ 0.014	0.690 $\pm$ 0.022	0.637 $\pm$ 0.024	0.670 $\pm$ 0.021	0.690 $\pm$ 0.020	0.634 $\pm$ 0.025	0.671 $\pm$ 0.020
	movies	0.206 $\pm$ 0.031	0.353 $\pm$ 0.037	0.345 $\pm$ 0.039	0.346 $\pm$ 0.038	0.308 $\pm$ 0.040	0.257 $\pm$ 0.039	0.303 $\pm$ 0.041	0.285 $\pm$ 0.034	0.224 $\pm$ 0.035	0.280 $\pm$ 0.033

Appendix

Table 5 : Accuracy results for incomplete rankings with 30% of missing labels

Problem	Dataset	RPC	ST-RR	ST-PI	ST-EPS	Chain-RR	Chain-PI	Chain-EPS	Native-RR	Native-PI	Native-EPS
LR	authorship	0.814 $\pm$ 0.028	0.893 $\pm$ 0.021	0.909 $\pm$ 0.020	0.909 $\pm$ 0.019	0.567 $\pm$ 0.034	0.629 $\pm$ 0.037	0.722 $\pm$ 0.033	0.609 $\pm$ 0.026	0.667 $\pm$ 0.034	0.777 $\pm$ 0.029
	glass	0.743 $\pm$ 0.058	0.851 $\pm$ 0.042	0.863 $\pm$ 0.040	0.861 $\pm$ 0.043	0.394 $\pm$ 0.068	0.389 $\pm$ 0.073	0.436 $\pm$ 0.078	0.424 $\pm$ 0.065	0.416 $\pm$ 0.069	0.475 $\pm$ 0.068
	iris	0.860 $\pm$ 0.056	0.938 $\pm$ 0.046	0.947 $\pm$ 0.048	0.946 $\pm$ 0.049	0.326 $\pm$ 0.139	0.341 $\pm$ 0.134	0.359 $\pm$ 0.139	0.321 $\pm$ 0.142	0.325 $\pm$ 0.135	0.335 $\pm$ 0.148
	letter	0.834 $\pm$ 0.002	0.923 $\pm$ 0.001	0.924 $\pm$ 0.001	0.761 $\pm$ 0.004	0.687 $\pm$ 0.003	0.293 $\pm$ 0.006	0.573 $\pm$ 0.006	0.579 $\pm$ 0.003	0.305 $\pm$ 0.004	0.462 $\pm$ 0.003
	libras	0.632 $\pm$ 0.027	0.846 $\pm$ 0.016	0.850 $\pm$ 0.016	0.824 $\pm$ 0.015	0.504 $\pm$ 0.029	0.358 $\pm$ 0.027	0.478 $\pm$ 0.029	0.529 $\pm$ 0.026	0.335 $\pm$ 0.035	0.505 $\pm$ 0.027
	movies	0.204 $\pm$ 0.035	0.344 $\pm$ 0.032	0.337 $\pm$ 0.030	0.339 $\pm$ 0.029	0.165 $\pm$ 0.040	0.107 $\pm$ 0.030	0.161 $\pm$ 0.041	0.179 $\pm$ 0.035	0.122 $\pm$ 0.028	0.177 $\pm$ 0.036

Appendix

Table 6 : Accuracy results for incomplete rankings with 60% of missing labels

Explore similar work

Stochastic Order Learning: An Approach to Rank Estimation Using Noisy Data
Jul 9, 2026Chaewon Lee, Seon-Ho Lee, Chang-Su KimNoisy Labels Noisy Data
School of Electrical Engineering, Korea University, Seoul, Korea · Amazon AGI, Seattle, USA
Calibrated Preference Learning: The Case of Label Ranking
May 28, 2026Santo M. A. R. Thies, Viktor Bengs, Timo Kaufmann +2Ranking Preference Learning
Data Science and its Applications, Deutsches Forschungsinstitut für Künstliche Intelligenz (DFKI), Kaiserslautern, Germany · Munich Center for Machine Learning, Munich (MCML), Germany · Ludwig Maximilian Universität München, Munich, Germany
MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment
Aug 25, 2026Dongyue Li, Ziniu Zhang, Lu Wang +1Preference Alignment Mixture Models
Northeastern University, Boston, MA · University of Michigan, Ann Arbor, MI

Jul 9, 2026, cs.LGJ/K move · Enter open · S save

Stochastic Order Learning: An Approach to Rank Estimation Using Noisy Data

Chaewon Lee, Seon-Ho Lee, Chang-Su Kim

School of Electrical Engineering, Korea University, Seoul, Korea · Amazon AGI, Seattle, USA

Noisy Labels Noisy Data Annotations

Open paper AskPDF arXiv

Rank estimation under label noise poses a fundamental challenge, as ordinal annotations often exhibit structured uncertainty rather than simple label corruption. In this paper, we reformulate rank estimation with noisy ordinal labels as a stochastic ordering problem, in which each instance is inherently associated with multiple plausible ranks instead of a single deterministic label. Based on this view, we propose stochastic order learning (SOL), a learning framework that captures ordinal label uncertainty and learns an embedding space through two complementary objectives: a discriminative loss that structures instance--centroid interactions and a stochastic order loss that enforces probabilistic ordering relations between instances. Extensive experiments across diverse datasets demonstrate that SOL enables reliable rank estimation under various types and levels of label noise. The source code is available at https://github.com/cwlee00/SOL.

Abstract

Explore similar work

Stochastic Order Learning: An Approach to Rank Estimation Using Noisy Data

Calibrated Preference Learning: The Case of Label Ranking

MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment