Bayesian Deck-of-cards-based Ordinal Regression with Sequential Preference Elicitation

Marco Grillo; Silvano Zappalà

stat.MLSep 19, 2026

Bayesian Deck-of-cards-based Ordinal Regression with Sequential Preference Elicitation

Authors: Marco Grillo, Silvano Zappalà

Organizations: Institute of Computing Science, Poznań University of Technology, Piotrowo 2, 60-965, Poznań, Poland · Department of Economics and Business, University of Catania, Corso Italia 55, 95129, Catania, Italy

Abstract

The Deck-of-cards-based Ordinal Regression (DOR) infers a value function from a ranking of reference alternatives in which the Decision Maker (DM) inserts blank cards between consecutive levels to express preference intensity. DOR, and its stochastic extension (SMAA-DOR), treat these answers as hard constraints defining a set of compatible value functions. We propose B-DOR, a probabilistic reformulation of DOR in which each pair of adjacent levels yields an ordinal observation, the declared direction and the number of cards, modelled through a cumulative-link likelihood that relates the number of blank cards to the latent value difference between alternatives. Two Bayesian inference algorithms are proposed: BAYES-DOR samples the whole posterior distribution by Hamiltonian Monte Carlo; FTRL-DOR tracks the maximum a posteriori estimate by constrained convex optimization. Moreover, through a multi-step elicitation process, elicitation can be spread over several short sessions reducing the cognitive burden on the DM. Both algorithms enjoy logarithmic regret bounds for prediction that hold for any sequence of DM responses and that guide the choice of the prior hyperparameters. A Monte Carlo study over 768 configurations shows that accuracy grows with the number of sessions, that blank cards add significant information over preference directions alone, that both algorithms maintain good performance under inconsistent answers, and that both outperform DOR and SMAA-DOR. An illustrative application to Italian regional healthcare performance demonstrates the practical applicability of the approach for building composite indicators.

Figures & tables

Panel of Figure 1 — Figure 1: Latent preference scale $z$ partitioned into card bands by thresholds $\theta_{m}$ .

Nominal $F_{\text{inc}}$	$0\%$	$15\%$	$35\%$	$50\%$
Flipped comparisons (all $\binom{k}{2}$ deduced pairs)	$0.0\%$	$7.7\%$	$15.4\%$	$21.5\%$
Flipped comparisons (adjacent pairs only)	$0.0\%$	$13.6\%$	$23.6\%$	$29.7\%$

Table 1: Realized share of flipped pairwise comparisons (preferences deduced from ranking, contradicting the true order) for each nominal inconsistency level

F_{\text{inc}}

Factor	Symbol	Levels
Number of alternatives	$\|A\|$	$10,\ 20,\ 35,\ 50$
Number of criteria	$m$	$3,\ 5,\ 7,\ 9$
Number of elicitation steps	$T$	$1,\ 3,\ 5,\ 10$
Number of alternatives ranked at each step (subset size)	$k$	$3,\ 4,\ 5$
Share of inconsistent data	$F_{\text{inc}}$	$0\%,\ 15\%,\ 35\%,\ 50\%$

Table 2: Factors of the full factorial simulation design and their levels.

	ASR	ASP	AIO
FTRL-DOR	$0.212{\scriptstyle\pm 0.003}$	$0.846{\scriptstyle\pm 0.001}$	$0.502{\scriptstyle\pm 0.008}$
FTRL ${}_{\text{dir}}$	$0.189{\scriptstyle\pm 0.003}$	$0.835{\scriptstyle\pm 0.001}$	$0.469{\scriptstyle\pm 0.008}$
BAYES-DOR	$0.175{\scriptstyle\pm 0.004}$	$0.805{\scriptstyle\pm 0.002}$	$0.439{\scriptstyle\pm 0.011}$
BAYES ${}_{\text{dir}}$	$0.159{\scriptstyle\pm 0.004}$	$0.793{\scriptstyle\pm 0.002}$	$0.412{\scriptstyle\pm 0.010}$
FTRL-DOR $>$ FTRL ${}_{\text{dir}}$ (Wilcoxon $p$ )	$<0.001$	$<0.001$	$<0.001$
BAYES-DOR $>$ BAYES ${}_{\text{dir}}$ (Wilcoxon $p$ )	$<0.001$	$<0.001$	$<0.001$

Table 3: Upper block: mean of each metric over the whole simulation grid, with

95\%

confidence intervals. Lower block: the same means after the first elicitation session only (

T=1

), for the proposed estimators and the DOR and SMAA-DOR. Both blocks report the

p

-value of a one-sided paired Wilcoxon signed-rank test of the hypothesis that the first method scores higher than the second.

Figure 2: Marginal effect of each experimental factor on the three metrics for the two B-DOR algorithms and their direction-only version. Within each plot the bars are sorted from best to worst; error bars are $95\%$ confidence intervals.

Figure 3: Marginal effect of each experimental factor on the three metrics, for the two B-DOR algorithms, SMAA-DOR and the original DOR. Within each level group the bars are sorted from best to worst; error bars are $95\%$ confidence intervals. Single-session comparison ( $T=1$ ).

Figure 4: Ranking metric scores as a function of the merge threshold $\tau$ , for $|A|=10$ , $m=3$ , $k=3$ , pooled over the inconsistency levels. Top axis reports the resulting mean number of levels per session, $\bar{l}=\displaystyle\frac{1}{T}\displaystyle\sum_{t}l_{t}$ . Shaded bands are $95\%$ confidence intervals.

Figure 5: Regret of the two algorithms against the number of gap observations $n_{T}$ on a logarithmic axis, for the configuration $|A|=10$ , $m=3$ , $k=5$ , $T=10$ (mean over five replications; shaded bands are $95\%$ confidence intervals).

ID	Region	Prevention care - $g_{1}$ ( $\uparrow$ )	Primary care - $g_{2}$ ( $\uparrow$ )	Hospital care - $g_{3}$ ( $\uparrow$ )
$R_{1}$	Piedmont	93	90	87
$R_{2}$	Aosta Valley	77	35	53
$R_{3}$	Lombardy	95	76	86
$R_{4}$	Autonomous Province of Bolzano	58	82	62
$R_{5}$	Autonomous Province of Trento	98	83	97
$R_{6}$	Veneto	98	96	94

Table 4: Italian regions’ performance on the LEA criteria in 2024 ( source ).

Panel of Figure 6 — Figure 6: DM preference information for each of three sessions. Each alternative populates its own level and is randomly drawn from $A^{R}$ in Table 4 . Alternatives are ranked, and, blank cards are placed.

Figure 7: Corner plot of the posterior distribution of the normalised marginal value functions $\left[\hat{u}_{1}(100),\hat{u}_{2}(100),\hat{u}_{3}(100)\right]$ after each of the $T=3$ sessions ( $T=1$ in blue, $T=2$ in orange, $T=3$ in green). Diagonal panels show the marginal distributions with the FTRL-DOR estimates (dashed vertical lines); off-diagonal panels show the $90\%$ credible contours of the bivariate distributions with the FTRL-DOR estimates (crosses).

Rank	$T=1$		$T=2$		$T=3$
Rank	Region	$\hat{U}\left(a\right)$	Region	$\hat{U}\left(a\right)$	Region	$\hat{U}\left(a\right)$
1	Veneto	96.09	Veneto	96.46	Veneto	96.94
2	Tuscany	95.31	Tuscany	95.22	Tuscany	95.12
3	Emilia Romagna	92.80	Emilia Romagna	93.22	Emilia Romagna	94.10
4	A. P. of Trento	92.74	A. P. of Trento	92.72	A. P. of Trento	93.58
5	Piedmont	90.13	Piedmont	90.69	Piedmont	91.41

Table 5: The five highest-scoring regions after each of the

T=3

sessions according to FTRL-DOR, with their normalised inferred evaluations. Alternatives included in the DM preferences are written in bold.

Region	Sessions	Rank
Region	Sessions	1	2	3	4	5	6	7	8	$>8$
Veneto ( $R_{6}$ )	$T=1$	67.7	22.4	9.9	0	0	0	0	0	0
	$T=2$	79.3	15.4	5.2	0	0	0	0	0	0
	$T=3$	88.5	10.2	1.2	0	0	0	0	0	0
Tuscany ( $R_{10}$ )	$T=1$	16.8	61.2	11.9	10.1	0	0	0	0	0
	$T=2$	9.8	64.3	12.7	13.2	0	0	0	0	0

Table 6: Evolution across the

T=3

sessions of the Rank Acceptability Indices (%) of the four top-ranked regions, in the case of linear marginal value functions

	$T=1$				$T=2$				$T=3$
Region	$R_{6}$	$R_{10}$	$R_{9}$	$R_{5}$	$R_{6}$	$R_{10}$	$R_{9}$	$R_{5}$	$R_{6}$	$R_{10}$	$R_{9}$	$R_{5}$
Veneto ( $R_{6}$ )	–	76.3	100	81.5	–	86.4	100	87.7	–	95.8	100	91.5
Tuscany ( $R_{10}$ )	23.7	–	89.7	71.3	13.6	–	85.9	71.2	4.2	–	76.5	67.5
Emilia Romagna ( $R_{9}$ )	0	10.3	–	47.6	0	14.1	–	52.3	0	23.4	–	56.5
A. P. of Trento ( $R_{5}$ )	18.6	28.7	52.3	–	12.3	28.8	47.6	–	8.5	32.6	43.5	–

Table 7: Evolution across the

T=3

sessions of the Pairwise Winning Indices (%) among the four top-ranked regions, in the case of linear marginal value functions

Figure 8: Posterior distribution of the normalised marginal value functions in the case of piecewise linear marginal value functions ( $\gamma_{j}=3$ ): rows correspond to criteria, columns to sessions. Each panel shows the BAYES-DOR posterior mean with its $90\%$ credible band and the FTRL-DOR estimate.

Rank	$T=1$		$T=2$		$T=3$
Rank	Region	$U\left(a\right)$	Region	$U\left(a\right)$	Region	$U\left(a\right)$
1	Veneto	94.48	Veneto	93.59	Veneto	93.98
2	Tuscany	92.57	Tuscany	91.05	Tuscany	91.90
3	A. P. of Trento	91.92	A. P. of Trento	88.30	A. P. of Trento	90.81
4	Emilia Romagna	90.67	Emilia Romagna	88.28	Emilia Romagna	89.68
5	Piedmont	85.42	Piedmont	82.89	Piedmont	84.12

Table 8: The five highest-scoring regions after each of the

T=3

sessions according to FTRL-DOR, with their normalised inferred piecewise linear marginal value functions (

\gamma_{j}=3

). Alternatives included in the DM preferences are written in bold.

Region	Sessions	Rank
Region	Sessions	1	2	3	4	5	6	7	8	$>8$
Veneto ( $R_{6}$ )	$T=1$	65.9	23.8	10.3	0	0	0	0	0	0
	$T=2$	84.2	12.8	2.9	0	0	0	0	0	0
	$T=3$	83.0	15.8	1.2	0	0	0	0	0	0
Tuscany ( $R_{10}$ )	$T=1$	13.3	51.9	19.8	15.0	0	0	0	0	0
	$T=2$	2.9	65.9	19.7	11.6	0	0	0	0	0

Table 9: Evolution across the

T=3

sessions of the Rank Acceptability Indices (%) of the four top-ranked regions, in the case of piecewise linear marginal value functions (

\gamma_{j}=3

)

	$T=1$				$T=2$				$T=3$
Region	$R_{6}$	$R_{10}$	$R_{5}$	$R_{9}$	$R_{6}$	$R_{10}$	$R_{5}$	$R_{9}$	$R_{6}$	$R_{10}$	$R_{5}$	$R_{9}$
Veneto ( $R_{6}$ )	–	79.3	76.2	100	–	94.8	86.6	100	–	98.2	83.5	100
Tuscany ( $R_{10}$ )	20.6	–	58.7	84.2	5.2	–	67.2	87.6	1.8	–	60.1	90.6
A. P. of Trento ( $R_{5}$ )	23.8	41.3	–	66.0	13.5	32.9	–	53.6	16.5	40.0	–	63.6
Emilia Romagna ( $R_{9}$ )	0	15.8	34.1	–	0	12.3	46.4	–	0	9.3	36.4	–

Table 10: Evolution across the

T=3

sessions of the Pairwise Winning Indices (%) among the four top-ranked regions, in the case of piecewise linear marginal value functions (

\gamma_{j}=3

)

Ordinal Regression

Ordinal regression predicts ordered categories while accounting for the fact that some mistakes are more severe than others.

See all 38 papers

Heterogeneous Ordinal Structure Learning with Bayesian Nonparametric Complexity Discovery
May 5, 2026Amir Rafe, Subasish DasGraph Structure Learning Bayesian Inference
Texas State University San Marcos, USA
On the Complexity of Preference-Based Bandits
Sep 30, 2026Ahmed Ben Yahmed, Marc Abeille, Clément CalauzènesPairwise Preference Learning Multi-Armed Bandits
CREST, ENSAE Paris, FAIRPLAY · FAIRPLAY
DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression
May 29, 2026Hongxu Ma, Lin Wang, Chenghou Jin +6Ordinal Regression
Fudan University Shanghai, China · Kuaishou Technology Beijing, China · Shanghai University of Finance and Economics Shanghai, China +1

May 5, 2026J/K move · Enter open · S save

Heterogeneous Ordinal Structure Learning with Bayesian Nonparametric Complexity Discovery

Amir Rafe, Subasish Das

Texas State University San Marcos, USA

Graph Structure Learning Bayesian Inference

Open paper AskPDF arXiv

Public attitudes toward artificial intelligence are heterogeneous, ordinally measured, and poorly captured by any single dependency graph. Existing ordinal structure learners assume a shared directed acyclic graph (DAG) across all respondents; recent heterogeneous ordinal graphical-model approaches focus on subgroup discovery rather than confirmatory cluster-specific DAG estimation; and latent profile analyses discard dependency structure entirely. We introduce a heterogeneous ordinal structure-learning framework combining monotone Gaussian score embedding, Bayesian nonparametric (BNP) complexity discovery via a truncated stick-breaking prior, and confirmatory fixed-K estimation with cluster-specific sparse DAG learning. The key methodological insight is a discovery-to-confirmation workflow: the nonparametric stage calibrates plausible archetype complexity, while inner-validated confirmatory refitting yields stable, interpretable structural estimates. On the 2024 Pew American Trends Panel AI attitudes survey, Wave 152 (W152) survey, (N = 4,788, 8 ordinal items), the confirmatory K*=5 model reduces holdout transformed-score mean squared error (MSE) by 25.8% over a single-graph baseline and by 4.6% over mixture-only clustering. A controlled tiered semi-synthetic benchmark calibrated to W152 structure validates recovery across difficulty regimes and transparently reveals failure modes under stress conditions.

Abstract

Ordinal Regression

Explore similar work

Heterogeneous Ordinal Structure Learning with Bayesian Nonparametric Complexity Discovery

On the Complexity of Preference-Based Bandits

DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression