Compote: Collapse-Resistant Multiple Hypothesis Learning For Sequential Recommendation

Louis Castres Saint Martin; Clement Calauzenes; Alain Rakotomamonjy; Patrick Gallinari

cs.LGOct 27, 2026

Compote: Collapse-Resistant Multiple Hypothesis Learning For Sequential Recommendation

Authors: Louis Castres Saint Martin, Clement Calauzenes, Alain Rakotomamonjy, Patrick Gallinari

Organizations: Criteo AI Lab, Paris, France · Sorbonne Université, CNRS, ISIR, 75005 Paris, France

Abstract

Next-item prediction is inherently ambiguous: a single interaction history often admits multiple plausible continuations across distinct catalogue regions. Standard sequential recommenders compress these alternatives into a single prediction, conflating user intents. We introduce COllapse-resistant Multiple hyPOThesis lEarning (COMPOTE), a multi-hypothesis transformer trained with an annealed objective that transitions from uniform supervision to a winner-takes-all assignment. This mechanism forces hypotheses to specialize into distinct catalogue regions, while inference relies on an efficient, normalizer-free max-similarity rule. Theoretically, we show that in the population case, the optimal solutions partition the conditional next-item distribution according to a max-entropy quantization principle, while pooled scores preserve the Bayes-optimal ranking. We also prove a generalization bound scaling logarithmically with the number of hypotheses. Across three Amazon Reviews 2023 benchmarks, COMPOTE outperforms strong ID-based, generative, and multi-interest baselines using item IDs alone. Matched-capacity ablations confirm that performance gains stem directly from the annealed objective, preventing the head collapse observed in conventional multi-head architectures, rather than added capacity, ensembling, or routing.

Figures & tables

Panel of Figure 1 — Figure 1: Multiple-choice transformer. An item-id history ${\mathbf{x}}$ is embedded, summed with a learned positional embedding, and encoded by a stack of $L$ pre-norm causal transformer blocks shared by all heads; the representation ${\mathbf{z}}$ at the last (most recent) position is read off. Two modules consume ${\mathbf{z}}$ : the $M$ hypothesis heads $h_{1},\dots,h_{M}$ , each producing ${\mathbf{h}}_{j}=h_{j}({\mathbf{z}})\in\mathbb{R}^{d}$ , and the confidence score head $\hat{\gamma}$ , whose $j$ -th output is the predicted probability that head $j$ is the best expert for ${\mathbf{x}}$ . Each hypothesis is turned into a distribution over the catalogue by $f_{j,i}({\mathbf{x}})=\softmax_{i}\!\big(\cos({\mathbf{h}}_{j},{\mathbf{e}}_{i})/\beta\big)$ , yielding one score vector $f_{j,\cdot}({\mathbf{x}})\in\Delta^{N\mhyphen 1}$ per head. Everything left of ${\mathbf{z}}$ is a single shared trunk; right of it the $M$ branches are independent, which is what makes head specialisation possible.

	Instrument				Scientific				Game
Method	H@5	H@10	N@5	N@10	H@5	H@10	N@5	N@10	H@5	H@10	N@5	N@10
Caser ( Tang and Wang, 2018 )	2.41	3.86	1.51	1.97	1.59	2.57	1.01	1.32	3.30	5.53	2.09	2.81
HGN ( Ma et al., 2019 )	3.21	5.17	2.02	2.65	2.12	3.51	1.31	1.76	4.24	6.87	2.71	3.56
GRU4Rec ( Hidasi et al., 2016 )	3.24	5.01	2.09	2.66	2.02	3.38	1.29	1.73	4.99	7.99	3.20	4.16
BERT4Rec ( Sun et al., 2019 )	3.07	4.85	1.95	2.52	1.86	2.96	1.19	1.55	4.60	7.35	2.98	3.86
SASRec ( Kang and McAuley, 2018 )	3.33	5.23	2.13	2.74	2.59	4.12	1.50	1.99	5.35	8.47	3.31	4.38

Table 1: Performance (%) on three Amazon 2023 subsets (H@

K

: Hit Rate@

K

, N@

K

: NDCG@

K

). First-block numbers are from Zheng et al. (2025) ; multi-interest models are trained by us under the same protocol ( Appendix F ). COMPOTE uses max-similarity aggregation ( Equation 10 ), equivalent to max-softmax pooling ( Section 4.1 ). Bold : best, underline : second best.

Aggregator	Normalizer	H@5	H@10	N@5	N@10
$g$ – ”max softmax”	required	6.04	9.27	4.01	5.05
$\tilde{g}$ – ”max cosine similarity”	not required	6.06	9.26	4.01	5.04

Table 2: Effect of the inference aggregator on COMPOTE performance (%) on Game, for the same trained model with

M=3

heads and the same checkpoint. “Normalizer” indicates whether ranking requires the per-head catalogue partition function

Z_{m}

. Bold : best.

	Instrument				Scientific				Game
Method	H@5	H@10	N@5	N@10	H@5	H@10	N@5	N@10	H@5	H@10	N@5	N@10
MCT-SASRec	3.63	5.70	2.34	2.97	2.55	3.87	1.71	2.13	5.73	8.84	3.78	4.78
MCT-EnsCE	3.54	5.53	2.28	2.96	2.42	3.80	1.60	2.03	5.74	8.81	3.78	4.77
MCT-Router	3.57	5.48	2.28	2.89	2.40	3.60	1.64	2.00	5.74	8.81	3.78	4.77
COMPOTE (ours)	3.75	5.83	2.45	3.10	2.84	4.20	1.92	2.35	6.06	9.26	4.01	5.04

Table 3: Ablation over the prediction head and the training objective (%). Variants share the same encoder backbone and parameter budget, and rank items with max-similarity aggregation ( Equation 10 ): MCT-SASRec is a single-hypothesis iso-parameter head, MCT-EnsCE uses an ensemble cross-entropy objective, and MCT-Router routes queries to a single expert. Bold : best.

$M$	In-cell mass	Win frequency (%)	Sector share (%)	$\hat{\mathcal{L}}^{\textsc{wta}}$	$\hat{\mathcal{L}}^{\textsc{wta}}+\log M$	Sector agreement
3	$0.929$	$[32.0,\,34.4]$	$[27.9,\,36.3]$	$7.32$	$8.41$	$98.5\%$
4	$0.915$	$[24.5,\,25.6]$	$[23.4,\,27.1]$	$7.02$	$8.41$	$98.7\%$
5	$0.937$	$[18.9,\,21.1]$	$[15.5,\,25.3]$	$6.71$	$8.32$	$99.4\%$

Table 4: Signatures of Theorem 7 on Game as the number of hypotheses

M

varies. All columns are computed on the full test set of

94{,}762

query–target pairs. Win frequencies and sector shares are reported as the range over heads.

Panel of Figure 2 — Figure 2: Catalogue tiling on Game ( $M=3$ ). Normalized items (dots) and head outputs (stars) are projected onto the plane spanned by the static head directions $\bar{{\mathbf{h}}}_{m}$ , the average normalized output of each head over the test set. Items are coloured by the direction they are most aligned with, and the green cross marks the target. Each panel is a test query won by a different head, and in each the target lies in the sector of its winning head.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Distribution of the per-head log-normalizer

\log Z_{m}({\mathbf{x}})

. Histogram of

\log Z_{m}({\mathbf{x}})=\log\sum_{i\leq N}\exp\big(s_{m,i}({\mathbf{x}})\big)

over the

94{,}762

test queries of Game , one colour per head, for the COMPOTE checkpoint with

M=3

of Table 2 (

\beta=0.07

N=25{,}612

). The three heads have almost superimposed distributions, concentrated between

12.2

and

15.0

on a scale where

\log Z_{m}

could a priori take any value in

[\log N-1/\beta,\ \log N+1/\beta]=[-4.1,24.4]

. For a fixed query, the standard deviation of

\log Z_{m}({\mathbf{x}})

across the three heads is

0.13

on average, against a mean value of

13.7

Dataset	#Users	#Items	#Interactions	Sparsity	Avg. len
Instrument	57,439	24,587	511,836	99.964%	8.91
Scientific	50,985	25,848	412,947	99.969%	8.10
Game	94,762	25,612	814,586	99.966%	8.60

Appendix

Table 5: Statistics of the preprocessed datasets [ Zheng et al., 2025 ] . Avg. len is the average length of the user interaction sequences.

Figure 4: Effect of the score-loss weight

\alpha

. N@5 (%) on Scientific with

M=3

under max-similarity aggregation, as a function of the weight

\alpha

of the score loss in Equation 33 . Ticks are evenly spaced and do not follow the scale of

\alpha

. Without the score loss (

\alpha=0

), COMPOTE already reaches

1.82

, above every single-hypothesis variant of Table 3 ; the best weight

\alpha=0.1

adds

0.09

points.

Figure 5: Effect of the number of hypotheses. N@5 on Scientific under max-similarity aggregation, as a function of

M

. Performance improves sharply from

M=1

M=3

, then plateaus: increasing

M

from 3 to 12 does not degrade ranking quality, in line with the sublinear dependence on

M

in Theorem 4 .

Method	Instrument	Scientific	Game
MCT-EnsCE	1.00	1.00	1.00
MCT-Router	1.00	1.00	1.00
COMPOTE (ours)	0.64	0.61	0.58

Appendix

Table 6: Mean pairwise cosine similarity between hypothesis heads on the test set. A value of

1

means the heads are indistinguishable, lower is more specialized.

	Instrument		Scientific		Game
Method	Aggregated	Oracle	Aggregated	Oracle	Aggregated	Oracle
MCT-EnsCE	2.28	2.28	1.60	1.60	3.78	3.78
MCT-Router	2.28	2.28	1.64	1.64	3.78	3.78
COMPOTE (ours)	2.45	3.75	1.92	3.52	4.01	7.78

Appendix

Table 7: NDCG@5 (%) under max-similarity aggregation ( Equation 10 ) and under oracle head selection ( Equation 11 ). Equal columns indicate that all heads induce the same ranking, i.e. complete collapse.

$M$	Mean off-diag. $\cos(\widetilde{{\mathbf{h}}})$	Cross-head top- $10$ Jaccard	Region loyalty
3	$-0.09$	$0.008$	$98.9\%$
4	$-0.17$	$0.003$	$99.4\%$
5	$-0.04$	$0.001$	$99.8\%$

Appendix

Table 8: Latent-space tiling on Game as the number of hypotheses

M

varies. Mean off-diagonal cosine similarity is computed between the static head directions

\widetilde{{\mathbf{h}}}_{m}

(the dataset-wide average of normalized head outputs). The top-

10

diagnostics use

4{,}000

test queries sampled uniformly without replacement. Cross-head Jaccard is the mean pairwise overlap of these lists across heads. Region loyalty is the fraction of a head’s top-

10

items that fall in its catalogue sector.

Panel of Figure 6 — Figure 6: Latent-space tiling on Game . Cosine similarity between the static head directions $\bar{{\mathbf{h}}}_{m}$ (the dataset-wide average of normalized head outputs) for $M\in\{3,4,5\}$ . Off-diagonal values remain near zero or negative, indicating that the heads occupy separated directions rather than collapsing onto one another.

Panel of Figure 7 — Figure 7: Catalogue partition on Scientific ( $M=3$ ). PCA projections on three test sequences (queries 4934, 23025, and 26314) under COMPOTE. Each panel shows the L2-normalized item catalogue in two dimensions (grey), the target next item (green cross), and the three head query embeddings (stars). The heads occupy distinct regions of the catalogue rather than collapsing onto a single direction.

Sequential Recommendation

Sequential recommendation predicts what a user will choose next from ordered interactions, adapting to changing interests and context.

See all 138 papers

RecRec: Recursive Refinement for Sequential Recommendation
Jul 12, 2026Pervez Shaik, Prosenjit Biswas, Abhinav Thorat +2Sequential Recommendation Iterative Refinement
Sony Research India India
Think Harder Where It Matters: Item-Level Adaptive Recursion for Sequential Recommendation
Oct 21, 2026Zhanshuo Cao, Jiakai Tang, Yuqi Zhou +2Sequential Recommendation Adaptive Computation
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment
Aug 14, 2026Haokai Ma, Aoqi Hu, Yueao Xing +5Recommender Systems Generative Recommendation
National University of Singapore Singapore, Singapore · Beijing University of Posts and Telecommunications Beijing, China · Tencent Beijing, China +1

Jul 12, 2026J/K move · Enter open · S save

RecRec: Recursive Refinement for Sequential Recommendation

Pervez Shaik, Prosenjit Biswas, Abhinav Thorat, Ravi Kolla+1

Sony Research India India

Sequential Recommendation Iterative Refinement User Preference Modeling Sequence Modeling

Open paper AskPDF arXiv

Sequential recommender systems typically infer user preferences through single-pass encoding of interaction histories without iterative refinement, relying on increasingly deep architectures to capture complex patterns. In this work, we revisit sequential recommendation from a recursive inference perspective: can user preferences be modeled as a persistent latent state that is recursively refined? We propose RecRec (Recursive Recommendation), a lightweight model that maintains a compact latent state and updates it through a shared recursive module conditioned on interaction evidence. Unlike prior recursive models, RecRec introduces an evidence-anchored correction mechanism that stabilizes refinement by grounding each update in the original interaction context, preventing semantic drift during deep recursive reasoning. Experiments on three benchmark datasets under standard evaluation protocols show that RecRec matches or outperforms state-of-the-art sequential, graph-based, and reasoning-enhanced recommenders while using only 3.9M to 14M parameters. Ablation studies demonstrate that both recursive refinement and the evidence-anchored correction gate contribute significantly to performance, highlighting the effectiveness of recursive latent inference as a scalable alternative to deeper or language-based architectures. Code is available at https://anonymous.4open.science/r/RecRec-6B67/README.md.

Abstract

Sequential Recommendation

Explore similar work

RecRec: Recursive Refinement for Sequential Recommendation

Think Harder Where It Matters: Item-Level Adaptive Recursion for Sequential Recommendation

EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment