History-Conditioned Joint-Prefix Alignment for Generative Recommendation

Hongliang Sun; Lianjie Li; Bolin Zhang; Dianbo Sui; Dianhui Chu; Zhiying Tu

cs.IRAug 29, 2026

History-Conditioned Joint-Prefix Alignment for Generative Recommendation

Authors: Hongliang Sun, Lianjie Li, Bolin Zhang, Dianbo Sui, Dianhui Chu, Zhiying Tu

Organizations: Department of Computer Science, Weihai & Qingdao Research Institute Harbin Institute of Technology Weihai, China

Abstract

Generative recommendation retrieves items by autoregressively generating semantic identifiers, but beam search may discard a target before its complete identifier is generated. Our preliminary analysis across three benchmarks shows that most missed targets are pruned within the first two decoding steps, highlighting the importance of early prefix retention. However, retaining a target's first-token branch alone is insufficient if its continuation is pruned at the next step: aligning only the first-token distribution improves first-token survival but yields little gain in complete-path retention. This observation motivates joint supervision of early branches and their continuations. We propose Prefix Alignment with Temporal History (PATH), which aggregates transition statistics from the training corpus over recent interactions with exponential decay to construct history-conditioned two-token prefix targets. PATH aligns the model's joint predictions with these targets through forward KL divergence, using a chain-rule decomposition that enables unbiased Monte Carlo estimation. At inference time, PATH reuses the transition statistics for pointwise mutual information (PMI) calibration, reranking completed candidates relative to global prefix frequency. Experiments on Beauty, Instruments, and Yelp demonstrate the effectiveness of PATH, showing consistent improvements in recommendation performance and higher full-SID survival rates.

Figures & tables

Panel of Figure 1 — Figure 1: Early beam pruning and ground-truth path survival. (a) A target can survive at $c_{1}^{*}$ yet be irreversibly lost when $c_{2}^{*}$ is removed at the next beam step. Through joint-prefix alignment, PATH aims to retain relevant target paths that would otherwise be pruned, while allowing irrelevant branches to be discarded. (b) Ground-truth prefix survival after $c_{1}$ and $c_{2}$ , full-SID survival, and Hit@10 for the NTP baseline with $B=20$ on Beauty, Instruments, and Yelp. Even when the first-token prefix survives, many target paths are lost at the second decoding step. The subsequent decline from two-token prefix survival to full-SID survival is smaller in absolute terms. This pattern motivates joint supervision of $(c_{1},c_{2})$ to preserve both early branches and their continuations. All four plots report percentages on the same vertical scale.

Figure 2: Overview of PATH. Joint-prefix transitions estimated from training interactions are aggregated over recent history to construct a temporal soft target. Joint forward-KL alignment supervises early decoding paths, while PMI-based calibration adjusts completed candidate scores relative to global prefix frequency.

		Beauty		Instruments		Yelp
Category	Method	H@10	N@10	H@10	N@10	H@10	N@10
Sequential/ collaborative	MF ( Koren et al., 2009 )	.0474	.0191	.0735	.0412	.0381	.0190
	Caser ( Tang and Wang, 2018 )	.0347	.0176	.0710	.0409	.0263	.0134
	HGN ( Ma et al., 2019 )	.0512	.0266	.1048	.0774	.0326	.0159
	LightGCN ( He et al., 2020 )	.0511	.0260	.1000	.0728	.0407	.0207
	SASRec ( Kang and McAuley, 2018 )	.0588	.0313	.0947	.0690	.0296	.0152

Table 1: Recommendation performance on three benchmarks. H@

K

and N@

K

denote Hit@

K

and NDCG@

K

, respectively. Bold indicates the best result in each column.

Dataset	Training objective	$\alpha$	$S_{c_{1}}$	$S_{c_{2}}$	$S_{\mathrm{full}}$
Beauty	NTP	0	43.57	14.26	10.07
	$c_{1}$ -only alignment	0.4	46.39	14.13	10.10
	Joint alignment	0.2	45.24	15.76	11.75
Instruments	NTP	0	47.02	16.83	13.84
	$c_{1}$ -only alignment	0.2	50.77	16.79	13.80
	Joint alignment	0.2	49.27	17.75	14.37

Table 2: Prefix and full-SID survival (%) at

B=20

. Bold indicates the best result within each dataset.

Training target	$S_{c_{1}}$	$S_{c_{2}}$	$S_{\mathrm{full}}$	H@10	N@10
None (NTP)	43.57	14.26	10.07	.0671	.0347
Global marginal	40.95	13.56	9.44	.0667	.0368
History shuffled	40.88	13.37	9.39	.0669	.0360
History aligned (PATH)	45.24	15.76	11.75	.0844	.0477

Table 3: History-conditioned supervision controls on Beauty. Auxiliary losses use

\alpha=0.2

, with inference calibration disabled. Survival rates are percentages. Bold indicates the best result in each column.

		Beauty			Instruments			Yelp
Training	Calibration	H@1	H@10	N@10	H@1	H@10	N@10	H@1	H@10	N@10
NTP	None	.0118	.0671	.0347	.0623	.1107	.0825	.0081	.0429	.0228
PATH	None	.0204	.0844	.0477	.0658	.1167	.0874	.0117	.0515	.0286
	Conditional prior	.0206	.0850	.0482	.0664	.1163	.0875	.0123	.0524	.0292
	PMI	.0208	.0856	.0484	.0664	.1176	.0881	.0118	.0525	.0293

Table 4: Training and calibration ablations. PATH calibration variants share the same checkpoint and candidate set within each dataset. Bold indicates the best result in each column.

Figure 3: Beam-width analysis on Beauty using fixed NTP and PATH checkpoints. (a) Full-SID survival at

B\in\{5,10,20,50\}

. (b) Relative survival gain of PATH over NTP. PATH improves survival at every tested width, with the largest relative gain at

B=5

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Users	Items	Interactions	Avg. length	Density
Beauty	22,363	12,101	198,502	8.88	0.0734%
Instruments	24,772	9,922	206,153	8.32	0.0839%
Yelp	30,431	20,033	316,354	10.40	0.0519%

Appendix

Table 5: Dataset statistics before validation and test holdout. Average length denotes the mean number of interactions per user.

Parameter	Main setting
History window $K_{\mathrm{hist}}$	5
History decay $\gamma$	0.7
Transition smoothing $\varepsilon$	$10^{-4}$
Monte Carlo samples $K_{\mathrm{mc}}$	2
Alignment weight $\alpha$	0.2 / 0.2 / 0.3
Calibration weight $\beta$	0.02

Appendix

Table 6: PATH hyperparameters for the main experiments. Dataset-specific alignment weights are listed in the order Beauty, Instruments, and Yelp.

Dataset	Configuration	$S_{c_{1}}$	$S_{c_{2}}$	$S_{\mathrm{full}}$	Rank	Hit@10	NDCG@10
Beauty	NTP ( $\alpha=0$ )	43.57	14.26	10.07	8.13	.0671	.0347
	$c_{1}$ -only ( $\alpha=0.4$ )	46.39	14.13	10.10	7.81	.0690	.0377
	Joint alignment ( $\alpha=0.2$ )	45.24	15.76	11.75	7.21	.0844	.0477
	Joint alignment + PMI	45.24	15.76	11.75	7.11	.0856	.0484
Instruments	NTP ( $\alpha=0$ )	47.02	16.83	13.84	5.31	.1107	.0825
	$c_{1}$ -only ( $\alpha=0.2$ )	50.77	16.79	13.80	5.40	.1101	.0818

Appendix

Table 7: Complete survival and ranking diagnostics at

B=20

. Survival rates are percentages. Rank denotes the average target rank conditioned on full-SID survival, for which lower is better. The PMI row calibrates the joint-alignment checkpoint immediately above it. Bold indicates the best value within each dataset.

Target / configuration	$\alpha$	$S_{c_{1}}$	$S_{c_{2}}$	$S_{\mathrm{full}}$	Hit@10	NDCG@10	$\Delta$ N@10
NTP	0	43.57	14.26	10.07	.0671	.0347	0.0%
Global marginal	0.1	42.80	14.30	10.22	.0715	.0382	+10.2%
	0.2	40.95	13.56	9.44	.0667	.0368	+6.1%
	0.4	37.28	11.75	7.63	.0567	.0296	-14.7%
History shuffled	0.1	42.62	14.40	10.20	.0724	.0386	+11.3%
	0.2	40.88	13.37	9.39	.0669	.0360	+3.8%

Appendix

Table 8: Control-weight sweeps on Beauty with

B=20

and seed 42.

\Delta

N@10 denotes the relative NDCG@10 change from the NTP baseline. All rows except the final row disable inference calibration. Survival rates are percentages. Bold indicates the best result in each metric column.

Beam $B$	Model	$S_{c_{1}}$	$S_{c_{2}}$	$S_{\mathrm{full}}$	Relative gain
5	NTP	27.38	7.12	3.86	–
5	PATH	28.95	8.49	5.38	+39.4%
10	NTP	35.02	10.19	6.44	–
10	PATH	36.48	11.76	8.23	+27.7%
20	NTP	43.57	14.26	10.07	–
20	PATH	45.24	15.76	11.75	+16.6%

Appendix

Table 9: Prefix survival at different beam widths on Beauty. Survival rates are percentages. Relative gain refers to PATH’s full-SID survival relative to NTP at the same beam width. Bold indicates higher survival within each pair.

Figure 4: Sensitivity to alignment and calibration weights on three datasets. (a) Uncalibrated NDCG@10 versus

\alpha

. (b) PMI-calibrated NDCG@10 versus

\beta

. Outlined markers indicate the main settings,

\alpha=0.2/0.2/0.3

and

\beta=0.02

, rather than the maxima of the test curves.

$K_{\mathrm{mc}}$	H@10	N@10	$S_{\mathrm{full}}$
2	.1176	.0881	14.37
3	.1171	.0880	14.34
4	.1163	.0878	14.46
5	.1194	.0891	14.66
6	.1156	.0871	14.31

Appendix

Table 10: Monte Carlo sample-count sensitivity on Instruments. All configurations use

\alpha=0.2

K_{\mathrm{hist}}=5

\gamma=0.7

, and seed 42, with PMI parameters

\beta=0.02

and

\tau=30

. Full-SID survival is reported as a percentage. Bold indicates the best result in each column.

Figure 5: History-window and temporal-decay sensitivity on Beauty. Every configuration is retrained with

\alpha=0.3

K_{\mathrm{mc}}=4

, and seed 42, then evaluated with

\beta=0.02

and

\tau=30

. The left column varies

K_{\mathrm{hist}}

with

\gamma=0.7

; the right column varies

\gamma

with

K_{\mathrm{hist}}=5

. The rows report Hit@10, NDCG@10, and full-SID survival, respectively. Stars indicate the best value in each sweep. Gray dashed lines and open circles mark the shared reference configuration

(K_{\mathrm{hist}},\gamma)=(5,0.7)

. Full-SID survival is reported as a percentage.

Dataset	Training objective	H@10	N@10	$S_{\mathrm{full}}$
Beauty	NTP	.0557	.0302	8.36
Beauty	Joint alignment	.0605	.0341	8.71
Instruments	NTP	.1004	.0759	12.32
Instruments	Joint alignment	.1052	.0754	13.02
Yelp	NTP	.0317	.0165	5.02
Yelp	Joint alignment	.0351	.0188	5.48

Appendix

Table 11: Decoder-only evaluation with

B=20

and seed 42. Joint alignment uses

\alpha=0.4

and

K_{\mathrm{mc}}=2

across all datasets. PMI calibration is disabled. Full-SID survival is reported as a percentage. Bold indicates the better result within each dataset.

Generative Recommendation

Generative recommendation models user histories and item catalogs to generate personalized items, lists, or queries rather than only scoring candidates.

See all 103 papers

EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment
Aug 14, 2026Haokai Ma, Aoqi Hu, Yueao Xing +5Recommender Systems Generative Recommendation
National University of Singapore Singapore, Singapore · Beijing University of Posts and Telecommunications Beijing, China · Tencent Beijing, China +1
Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation
Aug 12, 2026Kangning Zhang, Haotian Fang, Xukun Luo +6Sequential Recommendation RL for Recommendation
Shanghai Jiao Tong University Shanghai, China · Mei Tuan Beijing, China
AlignPoison: Conditional Targeted Poisoning of Generative Recommenders via Decoding-Aligned Prefixes
Oct 10, 2026Zhuoxuan Zhang, Jinhao Duan, Xinyu Zhao +7Data Poisoning Attacks Generative Recommendation
University of North Carolina at Chapel Hill · University of Notre Dame · Meta

Aug 14, 2026J/K move · Enter open · S save

EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment

Haokai Ma, Aoqi Hu, Yueao Xing, Ruobing Xie+4

National University of Singapore Singapore, Singapore · Beijing University of Posts and Telecommunications Beijing, China · Tencent Beijing, China+1

Recommender Systems Generative Recommendation User Preference Modeling Sequential Recommendation Preference Alignment Multi-Token Prediction

Open paper AskPDF arXiv

Generative recommendation autoregressively generates the semantic IDs of the target item, unifying preference modeling and index retrieval within the shared token space. Recent attempts have introduced Multi-Token Prediction (MTP) into this field, yet they primarily inherit its efficiency merit, leaving its potential as dense supervision unexplored. Unlocking this potential hinges on whether future behaviors qualify as informative supervision. Our analysis reveals that future behaviors carry a semantic echo of the current one far above that of random pairs, which nevertheless decays along horizons under intent transitions, making them informative yet order-dependent signals. Motivated by this, we propose EchoRec, which empowers MTP with cycle-consistent holistic preference alignment across multi-horizon for generative recommendation. It comprises two synergistic modules. Horizon-aware Preference Generation (HPG) sequentially chains lightweight auxiliary branches upon the base recommender, where each branch conditions on its predecessor to respect preference evolution. Verifiable Holistic-Preference Alignment (VHA) further consolidates them into the holistic preference and echoes it back through cycle-consistent projectors to suppress spurious alignment, with theoretical guarantees that exclude the rank-collapse form of spurious alignment under an invertible transport, enabling the holistic preference to be retained in the decoding representation. All auxiliary components serve as disposable scaffolding discarded at inference, introducing negligible online serving overhead. Extensive experiments on three datasets demonstrate the superiority of our EchoRec, together with its naturally acquired multi-item generation ability. Our code and datasets will be available upon acceptance.

Abstract

Generative Recommendation

Explore similar work

EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment

Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation

AlignPoison: Conditional Targeted Poisoning of Generative Recommenders via Decoding-Aligned Prefixes