cs.IRApr 9, 2026

Efficient Dataset Selection for Continual Adaptation of Generative Recommenders

Authors: Cathy Jiao, Juan Elenter, Praveen Ravichandran, Bernd Huber, Joseph Cauteruccio, Todd Wasson, Timothy Heath, Chenyan Xiong, 

Organizations: Carnegie Mellon University · Spotify

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recommendation systems must continuously adapt to evolving user behavior, yet the volume of data generated in large-scale streaming environments makes frequent full retraining impractical. This work investigates how targeted data selection can mitigate performance degradation caused by temporal distributional drift while maintaining scalability. We evaluate a range of representation choices and sampling strategies for curating small but informative subsets of user interaction data. Our results demonstrate that gradient-based representations, coupled with distribution-matching, improve downstream model performance, achieving training efficiency gains while preserving robustness to drift. These findings highlight data curation as a practical mechanism for scalable monitoring and adaptive model updates in production-scale recommendation systems.

Figures & tables

Figure 1: Relative performance improvement of the HSTU model over time, measured with NDCG@50 and HR@50. The top row reports retraining and evaluation performed monthly, while the bottom row reports results every six months. At each time window, model updates are performed by continuing training from the previous checkpoint and incorporating previously unseen sequences. We compare four strategies: (i) no further training, (ii) training with 100% of new data, (iii) training with a random 20% subset of new data, and (iv) training with a random 50% subset of new data.
Figure 2: Diagram of the main block in HSTU and comparison with SASRec
Figure 3: Our training and evaluation pipeline in a continuous learning setting. Training is conducted in 6-month intervals.
Figure 4: Comparison analysis of GradSim and RepSim representation types. Performance is measured as improvement relative to the no-retraining baseline.
Figure 5: Impact of the reference set size used for RepSim selection. The figure reports relative performance trends for NDCG@10, NDCG@50, HR@10, and HR@50 when using reference sets of size 100 and 1000. All metrics are shown over time and normalized to highlight relative improvements.
Figure 6: Comparison analysis of ranking-based sampling: top-k, bottom-k, both. Performance is measured as improvement relative to the no-retraining baseline.
Figure 7: Comparison analysis of TopBottomk versus weighted sampling. Performance is measured as improvement relative to the no-retraining baseline.
Method+1 yr+3 yrs
No Retrain0%0%
Random (20%)38%42%
RepSim (TopBottomK)55%61%
GradSim (Diverse-Weighted)72%78%
Table 1: Fraction of drift-induced NDCG@50 error recovered relative to no retraining. Error reduction is computed with respect to the gap between the no-retraining model and full retraining at each evaluation horizon.
Figure 8: FLOPs versus performance trade-off.
Figure 9: Comparison analysis of different sampling strategies (weighted, knn-weighted, diverse-weighted) against random selection. Performance is measured as improvement relative to the no-retraining baseline.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

 NDCG@10NDCG@50HR@10HR@50MRR
Initial Performance1.0001.0001.0001.0001.000
Test Set (1 week)0.9700.9700.9700.9700.970
Test Set (1 month)0.9190.9210.9430.9220.919
Test Set (1 year)0.8820.8820.9030.8930.869
Test Set (3 years)0.6910.6890.6300.6300.728
Appendix
Table 2: Relative performance of the HSTU model on item prediction for future time periods, normalized by the initial evaluation (baseline = 1.00).
 NDCG@10NDCG@50HR@10HR@50MRR
Full Train Set1.0001.0001.0001.0001.000
Random (50%)0.9220.9280.9500.9600.910
Random (20%)0.8290.8500.9090.9540.796
Random (10%)0.7000.7620.7700.9360.679
BM25 (50%)0.9130.9230.9430.9640.898
BM25 (20%)0.6550.7020.8610.9540.570
Appendix
Table 3: Relative performance of the HSTU model on item prediction up to one week after 01/01/2022, normalized by the Full Train Set (baseline = 1.00).
 NDCG@10NDCG@50HR@10HR@50MRR
Full Train Set1.0001.0001.0001.0001.000
Random (50%)0.9850.9850.9790.9800.991
Random (20%)0.9600.9610.9770.9790.955
Random (10%)0.9070.9330.8900.9650.924
BM25 (50%)0.9830.9780.9960.9800.976
BM25 (20%)0.7670.7920.9350.9760.691
Appendix
Table 4: Relative performance of the HSTU model on item prediction up to one year after 01/01/2022, normalized by the Full Train Set (baseline = 1.00).
MethodRelative FLOPsDominant cost
Random0—
RepSim≈1\approx 1+(n+r)Ffwd+(n+r)F_{\text{fwd}}
GradSim≈3\approx 3+(n+r)(Ffwd+Fbwd)+(n+r)(F_{\text{fwd}}+F_{\text{bwd}})
Appendix
Table 5: Relative FLOPs per period (Random =0=0 ).
More in this topic

Data Selection

Data selection chooses informative, representative, or reliable examples to improve model training and evaluation while reducing data, compute, or labeling costs.

See all 80 papers

Explore similar work

CardsList

  1. Self-Evolving Memory for Generative Recommendation

    Sep 14, 2026Xinyu Lin, Zhuosong Jiang, Zixiao Suo +12Memory-Augmented Neural NetworksRecommender Systems

    National University of Singapore · Meta AI

  2. RAIE: Region-Aware Incremental Preference Editing with LoRA for LLM-based Recommendation

    Feb 28, 2026Jin Zeng, Yupeng Qi, Hui Li +4Recommender SystemsLarge Language Model-Based Recommendation

    School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, Guangdong, China · Department of Computer Science and Technology, Xiamen University, Xiamen, Fujian, China · Artificial Intelligence Research Institute, Shenzhen MSU-BIT University, Shenzhen, Guangdong, China +2

  3. Soft Curriculum Learning for Optimizing Fresh and Generalized Recommendations

    Aug 11, 2026Arnab Bhadury, Siyan Zheng, Anlan Yu +6Popularity BiasLong-Tail Recommendation

    Google Vancouver, BC, Canada · Google Mountain View, CA, USA