HASTE: Hierarchical and Attribute-aware Supervision for Generative Recommendation

Tripti Shukla; Shwetha Somasundaram; Akshat Gupta; Rohit Garg; Paul Duff; Ian Anderson

cs.IROct 12, 2026

HASTE: Hierarchical and Attribute-aware Supervision for Generative Recommendation

Authors: Tripti Shukla, Shwetha Somasundaram, Akshat Gupta, Rohit Garg, Paul Duff, Ian Anderson

Organizations: Glance Bengaluru, India · Glance Vilnius, Lithuania · Glance London, United Kingdom

Abstract

Generative recommendation models have shown promising results by representing items as semantic IDs and generating these IDs autoregressively. We introduce an approach showing that explicitly incorporating hierarchical and attribute-level supervision into generative recommendation can simultaneously improve training efficiency and long-tail recommendation performance. Existing approaches typically optimize semantic ID prediction with a standard causal cross-entropy objective, which does not explicitly exploit the hierarchical structure of semantic IDs and can require substantial training to learn effective item representations. Moreover, data-sparse tail items often suffer from poor representations due to limited supervision, resulting in weaker long-tail recommendation performance. We address these challenges with HASTE (Hierarchical and Attribute-aware Supervision for generaTive rEcommendation), a unified training framework that introduces hierarchical and attribute-specific supervision. HASTE's Hierarchical Loss explicitly exploits the multi-level structure of semantic IDs, providing supervision at each level of the item hierarchy and substantially accelerating training convergence. Its Attribute-Specific Losses encourage the model to preserve fine-grained item attributes, leveraging the language model's inherent semantic knowledge to improve representations and recommendations for tail items. We extensively evaluate HASTE on a fashion recommendation benchmark and across multiple domains of the Amazon dataset. HASTE improves recommendation performance across datasets and item-frequency regimes. Notably, the Hierarchical Loss achieves approximately 2x faster convergence than the standard causal cross-entropy objective, substantially improving training efficiency while maintaining strong recommendation quality.

Figures & tables

Figure 1. Overview of the proposed HASTE training framework. The LLM autoregressively predicts hierarchical SIDs

L_{0},L_{1},L_{2},L_{3}

with confidence-gated loss

\mathcal{L}_{H}

, where

w_{i}

and

g_{i}

denote level weights and detached confidence gates. The LLM hidden state

h_{L2}\in\mathbb{R}^{d}

feeds a selected attribute-specific MLP, with

\mathcal{L}_{\mathrm{attr}}

weighted by

\lambda_{\mathrm{attr}}

. The SID tuple

(l_{0},l_{1},l_{2},l_{3})

indexes its corresponding attribute vector.

Figure 2. Hierarchical structure of semantic-ID representations under different fine-tuning objectives. (a) Within-level cohesion, (b) inter-level coupling, and (c) embedding drift from the shared initialization. Hierarchical Loss better preserves the structure inherited from residual quantization than standard causal cross-entropy. bar charts comparing std CE \& hierarchical loss. — Figure 2. Hierarchical structure of semantic-ID representations under different fine-tuning objectives. (a) Within-level cohesion, (b) inter-level coupling, and (c) embedding drift from the shared initialization. Hierarchical Loss better preserves the structure inherited from residual quantization than standard causal cross-entropy. bar charts comparing std CE & hierarchical loss.

Figure 3. Motivation for attribute-specific supervision. (a) Color purity increases across SID levels, revealing fine-grained attribute structure. (b) Nonlinear probes decode color better than linear probes, indicating latent but unorganized attribute information. (c) Attribute supervision mainly reduces color prediction error for torso and tail items with sparse interaction data. bar charts comparing std CE \& hierarchical loss. — Figure 3. Motivation for attribute-specific supervision. (a) Color purity increases across SID levels, revealing fine-grained attribute structure. (b) Nonlinear probes decode color better than linear probes, indicating latent but unorganized attribute information. (c) Attribute supervision mainly reduces color prediction error for torso and tail items with sparse interaction data. bar charts comparing std CE & hierarchical loss.

Dataset	Metric	Sequential		Generative		LLM-Based		Ours (CE loss)	Ours (Hier loss)
		SASRec ( Kang and McAuley, 2018 )	BERT4Rec ( Sun et al., 2019 )	TIGER ( Rajput et al., 2023 )	HSTU ( Zhai et al., 2024 )	IDGenRec ( Tan et al., 2024 )	OneRec-Think ( Liu et al., 2025 )
Beauty	Recall@5	0.0402	0.0232	0.0405	0.0424	0.0484	0.0563	0.0325	0.0588 (+4.4%)
	Recall@10	0.0607	0.0396	0.0623	0.0652	0.0693	0.0791	0.0672	0.0823 (+4.0%)
	NDCG@5	0.0254	0.0146	0.0267	0.0280	0.0337	0.0398	0.0326	0.0387 (-2.8%)
	NDCG@10	0.0320	0.0199	0.0337	0.0353	0.0404	0.0471	0.0358	0.0503 (+6.8%)
Sports	Recall@5	0.0199	0.0102	0.0215	0.0268	0.0270	0.0288	0.0249	0.0322 (+11.8%)

Table 1. Performance comparison across different recommendation methods. The best result is shown in bold and the second-best result is underlined .

Method	Split	Amazon Fashion-140k				Internal-fashion
		Recall		NDCG		Recall		NDCG
		@5	@10	@5	@10	@5	@10	@5	@10
CE	Overall	0.0243	0.0324	0.0236	0.0452	0.0255	0.0319	0.0376	0.0425
	Tail	0.0134	0.0169	0.0245	0.0298	0.0267	0.0342	0.0387	0.0393
HL	Overall	0.0297 (+22.2%)	0.0345 (+6.5%)	0.0316 (+33.9%)	0.0486 (+7.5%)	0.0284 (+11.4%)	0.0351 (+10.0%)	0.0412 (+9.6%)	0.0468 (+10.1%)
	Tail	0.0168 (+25.4%)	0.0215 (+27.2%)	0.0297 (+21.2%)	0.0354 (+18.8%)	0.0301 (+12.7%)	0.0387 (+13.2%)	0.0435 (+12.4%)	0.0459 (+16.8%)

Table 2. Overall and tail-item recommendation performance under hierarchical and attribute-specific supervision. Values in blue denote relative improvement over the corresponding baseline: CE for HL and CE-based attribute variants, and HL for HL-based attribute variants. Bold values indicate the best performance, while underlined values indicate the second-best performance.

Dataset	CE Steps	HL Steps	Speedup
Toys	92000	42000	2.19 $~\times$
Sports	100000	53000	1.89 $~\times$
Beauty	85000	44000	1.93 $~\times$
Average	–	–	$1.99~\times$

Table 3. Training efficiency of standard causal cross-entropy (CE) and Hierarchical Loss (HL). HL substantially reduces the number of training steps required to reach convergence.

SID Supervision	NDCG@10	$\Delta$ vs. CE
Causal CE	0.0328	–
$(s_{a})$	0.0355	+0.0027
$(s_{a},s_{b})$	0.0410	+0.0082
$(s_{a},s_{b},s_{c})$	0.0435	+0.0107
$(s_{a},s_{b},s_{c},s_{d})$	0.0492	+0.0164

Table 4. Ablation of hierarchical supervision across semantic-ID levels on Toys dataset. Using conditional gating, we progressively incorporate higher-level semantic codes into the Hierarchical Loss.

Supervision	Recall@10	NDCG@10	$\Delta$ vs. CE (%)
CE	0.0571	0.0328	–
HL w/ SW	0.0624	0.0386	+9.3 / +17.7
HL w/ CG	0.0678	0.0402	+18.7 / +22.6
HL w/ SS	0.0598	0.0357	+4.7 / +8.8
HL w/ SW, CG, SS	0.0824	0.0492	+44.3 / +50.0

Table 5. Ablation of individual components of the Hierarchical Loss on the Toys dataset. Here, HL denotes Hierarchical Loss, SW static level weighting, CG conditional gating, and SS scheduled sampling.

Statistic	Toys	Beauty	Sports	Fashion-140k	Internal-fashion
Items (catalog)	11,924	12,101	18,357	141,300	163,724
User sequences	19,412	22,363	35,598	21,326	103,047
Total events	167,597	198,502	296,337	71,059	1,640,953
Seq. length (min/med/mean/max)	5/6/8.63/550	5/6/8.88/204	5/6/8.32/296	3/3/3.33/15	3/6/15.92/100
Total conversations	1,390,005	1,720,660	2,508,966	1,407,009	103,047
Train / Validation / Test	1,278,805/69,500/19,412	1,583,007/86,033/22,363	2,308,249/125,448/35,598	1,294,448/70,350/21,326	6,255,087/339,950/203,971

Table 6. Dataset statistics. We report catalog size, user sequences, interaction events, conversation statistics, and metadata attributes. “–” indicates that the corresponding statistic is not applicable.

	Sports	Beauty	Toys
Items	18,357	12,101	11,924
Users (sequences)	35,598	22,363	19,412
Title coverage	99.5%	99.9%	99.5%
Description coverage	90.5%	98.3%	98.5%
Brand coverage	67.2%	82.8%	85.6%
Price coverage	80.4%	95.2%	92.9%

Table 7. Item and metadata coverage for Sports, Beauty, and Toys.

Attribute	Count	Coverage
Items	141,300	—
Sequences	21,326	—
product_type	141,220	99.9%
gender	138,332	97.9%
brand	130,177	92.1%
occasion	130,066	92.0%

Table 8. Structured-attribute coverage in Fashion-140k.

Statistic	Toys	Beauty	Sports	Fashion-140k	Internal-fashion
Representation	1024-d text			1024-d text	1024-d multimodal
Unique 3-tuple SIDs	11,615	11,649	17,361	127,844	153,046
Collision rate	2.6%	3.7%	5.4%	9.5%	6.5%

Table 9. Semantic-ID construction and representation statistics across datasets. Each dataset uses three residual quantization levels with 256 codes per level; a deduplication token is appended when required.

Hyperparameter	Setting
Base weights $(w_{0},w_{1},w_{2},w_{3})$	$(4.0,3.0,2.0,1.0)$
Gating floor	0.2
Gating sharpness $\gamma$	1.0
Max scheduled-sampling rate	0.3
Scheduled-sampling anneal steps	200,000
Blend anneal steps	300,000

Table 10. Hyperparameter settings for the hierarchical loss. Base weights are ordered from the coarsest to the finest SID level. The same configuration is used across Sports, Beauty, and Toys.

Group	Sub-type ( $n$ )	User turn	Assistant turn
SID $\to$ text (decoding)	sid_to_title (11,865)	Product {sid_58,401,722,768} has title:	Doctor Who – Master Laser Screwdriver
	sid_to_description (10,736)	Item {sid_30,341,570,768} is described as:	Only Playhut structures provide children with hours of interactive adventure. EZ Twist technology allows for instant setup and quick fold-down…
	sid_to_context (11,865)	Tell me about product {sid_107,383,713,768}:	Title: Crayola Tub of Fun; Description: Crayola Tub of Fun; Category: Toys & Games
	sid_to_category (11,924)	Product {sid_122,381,749,768} belongs to category:	Toys & Games $>$ Games $>$ Board Games
	sid_to_brand (10,199)	Product {sid_77,477,593,768} is made by:	Parker Brothers
Text $\to$ SID (encoding)	title_to_sid (11,865)	The product “Bandai Tamashii Stage Act.5 for Mechanics Clear” has SemanticID:	{sid_56,289,628,768}

Table 11. One example per conversation sub-type in the Toys dataset, grouped into the five families of Section 9.1.5 . Semantic IDs are abbreviated as {sid_W,X,Y,Z} in this table only, where

W

X

Y

, and

Z

denote the four semantic-ID levels.

Variant	Base weights $(w_{1},w_{2},w_{3},w_{4})$	$\alpha$ schedule	Gate floor / $\gamma$	$\mathtt{ss\_max\_prob}$
CE	$(1.0,1.0,1.0,1.0)$	$\alpha\equiv 1.0$	– / –	$0$
HL w/ SW	$(4.0,3.0,2.0,1.0)$	$\alpha\equiv 1.0$	– / –	$0$
HL w/ CG	$(1.0,1.0,1.0,1.0)$	$1.0\rightarrow 0.0$ over $300{,}000$ steps	$0.2$ / $1.0$	$0$
HL w/ SS	$(1.0,1.0,1.0,1.0)$	$\alpha\equiv 1.0$	– / –	$0.3$
HL w/ SW, CG, SS (full)	$(4.0,3.0,2.0,1.0)$	$1.0\rightarrow 0.0$ over $300{,}000$ steps	$0.2$ / $1.0$	$0.3$

Table 12. Hyperparameter settings for the hierarchical-loss component ablation. Base weights are ordered from the coarsest to the finest SID level. All variants use three SID levels.

Long-Tail Recommendation

See all 8 papers

Aug 12, 2026

Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation

Semantic-ID generative recommenders represent each item as a short sequence of discrete semantic tokens and predict the next item by autoregressively generating this token sequence. This paradigm enables a unified generation interface for item IDs, histories, and item text, but it also creates a structured optimization bottleneck during reward-based post-training: when an early semantic token enters the wrong branch of the item-token space, finite rollout groups rarely reach the ground-truth item, so group-relative optimization receives identical zero rewards and produces no useful advantage. We propose Hint-Conditioned Generative Recommendation (HCGRec), a semantic-ID generative recommendation framework that recovers learning signal for such hard training instances. HCGRec diagnoses each instance with checkpoint rollouts and supplies a minimal target-prefix hint only when the current generator cannot reach the correct item. The model then generates the unhinted suffix under the hinted semantic branch, turning zero-reward groups into informative comparisons over item-token completions. Hinting also changes token identity: hinted prefix tokens are oracle-provided item context, while unhinted suffix tokens are sampled generation actions. We therefore introduce hint-aware credit decomposition, using supervised learning to preserve item-semantic and prefix-structure alignment for hinted tokens and GRPO to optimize the sampled suffix. Experiments on sequential recommendation benchmarks show that HCGRec substantially improves over supervised fine-tuning and vanilla reward-based post-training, while reducing zero-advantage training samples from over 70% to below 20%. The code is accessible at https://github.com/WncFht/GRec.

Kangning Zhang, Haotian Fang, Xukun Luo +6

Shanghai Jiao Tong University Shanghai, China · Mei Tuan Beijing, China

Sequential Recommendation RL for Recommendation Recommender Systems RL for Generative Models Generative Recommendation

Ask PDF

Jul 28, 2026

VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation

Generative recommendation commonly represents items using fixed-length semantic identifiers (SIDs) constructed through clustering and quantization. However, these artificial codes may overcompress item semantics, remain misaligned with pretrained LLM vocabularies, and require costly autoregressive decoding. In light of this, we propose VaLiDRec, a generative recommendation framework based on variable-length, LLM-aligned semantic identifiers. VaLiDRec constructs SIDs directly from informative native LLM vocabulary tokens via token importance estimation, semantic-quality-aware pruning, and collision-aware refinement, allowing identifier lengths to adapt to item semantic complexity. To model user preferences, VaLiDRec incorporates graph-aware soft prompts and reformulates recommendation as token-set prediction with token-level item scoring, eliminating autoregressive SID generation and beam search. Experiments on four real-world datasets show that VaLiDRec consistently outperforms strong sequential and generative recommendation baselines across all evaluation metrics. It further achieves superior zero-shot item cold-start performance and 87.49

\times

faster inference than LC-Rec. These results demonstrate that LLM-native variable-length semantic identifiers provide a more expressive and efficient paradigm for generative recommendation.

Shutong Qiao, Wei Yuan, Tong Chen +3

University of Queensland Brisbane, Australia · Computer Network Information Center, Chinese Academy of Sciences Beijing, China · Griffith University Gold Coast, Australia

Semantic ID Recommendation Graph Prompt Learning Recommender Systems Large Language Model-Based Recommendation Generative Recommendation

Ask PDF

Feb 25, 2026

Trie-Aware Transformers for Generative Recommendation

Generative recommendation (GR) aligns with advances in generative AI by casting next-item prediction as token-level generation rather than score-based ranking. Most GR methods adopt a two-stage pipeline: (i) item tokenization, which maps each item to a sequence of discrete, hierarchically organized tokens; and (ii) autoregressive generation, which predicts the next item's tokens conditioned on the tokens of user's interaction history. Although hierarchical tokenization induces a prefix tree (trie) over items, standard autoregressive modeling with conventional Transformers often flattens item tokens into a linear stream and overlooks the underlying topology. To address this, we propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases via two positional encodings. First, a trie-aware absolute positional encoding aggregates a token's (node's) local structural context (\eg depth, ancestors, and descendants) into the token representation. Second, a topology-aware relative positional encoding injects pairwise structural relations into self-attention to capture topology-induced semantic relatedness. TrieRec is also model-agnostic, efficient, and hyperparameter-free. In our experiments, we implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83% on average across four real-world datasets.

Zhenxiang Xu, Jiawei Chen, Sirui Chen +5

State Key Laboratory of Blockchain and Data Security, Zhejiang University. · College of Computer Science and Technology, Zhejiang University. · Zhejiang University Hangzhou, China +2

Generative Recommendation Sequential Recommendation Transformer Positional Encoding

Ask PDF

Abstract

Long-Tail Recommendation

Explore similar work

Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation

VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation

Trie-Aware Transformers for Generative Recommendation