Large-scale recommender systems, particularly short-form video platforms, are often bottlenecked by massive popularity feedback loops. In such environments, as models recommend popular items, they generate an overwhelming amount of skewed training data for "head" items. This creates a self-reinforcing cycle where retrieval and ranking models memorize "head" item patterns at the expense of generalizing across the vast "tail" of the catalogue. While Curriculum Learning (CL) offers a powerful mechanism to break this feedback loop by systematically exposing models to progressively more difficult and less frequent examples, its adoption in industrial recommendation has been hampered by hardware utilization inefficiencies or the needs for complicated pre-processing techniques because dynamic data rejection algorithms tend to starve hardware accelearators (TPUs/GPUs) by becoming largely CPU-bound. In this work, we introduce a scalable Soft Curriculum Learning framework designed specifically for continuous training setups within industry-scale retrieval and ranking models. By utilizing loss annealing and in-graph weight adjustments rather than rigid data filtering, we break the popularity feedback and enable dynamic curriculum pacing without sacrificing system throughput. We demonstrate empirical evidence through applications across sequence-based retrieval models (such as SASRec), two-tower retrieval models, and large-scale continuous ranking models. Online A/B tests on our short-video platform demonstrate substantial lifts in both overall user satisfaction and fresh content consumption, all without degrading model throughput.
Figures & tables
Task
Eng-Hard
Eng-Bi
SNGP-Hard
SNGP-Bi
Imp. Pos.
+0.02%
+0.03%
+0.01%
+0.01%
Exp. Neg.
+0.06%
+0.08%
+0.02%
+0.06%
Satisfaction
+0.02%
+0.04%
+0.01%
+0.02%
Imp. Neg.
+0.03%
+0.04%
+0.01%
+0.02%
Table 1. Ranking Offline Performance Evaluations
Video Age 1 1 1 Number of Video Age (in days) from Bucket 1 to 5 in ascending order.
Imp. Pos.
Exp. Neg.
Satisfaction
Imp. Neg
Age Bucket 1
+0.05%
+0.14%
+0.05%
+0.08%
Age Bucket 2
+0.02%
+0.08%
+0.02%
+0.03%
Age Bucket 3
+0.02%
+0.05%
+0.02%
+0.02%
Age Bucket 4
+0.02%
+0.06%
+0.02%
+0.03%
Age Bucket 5
+0.02%
+0.05%
+0.02%
+0.02%
Table 2. Performance Comparison Categorized by Video Age of Eng-Bi
Method
User Satisfaction
User Growth
User Retention
SNGP-bi
+0.42%
+0.15%
+0.15%
Eng-bi
+0.54%
+0.11%
+0.15%
Table 3. Online Ranking Model Performance: User Metrics
Method
User
Cold-start Engagement
Engagement
Age Bucket 1
Age Bucket 2
Age Bucket 3
SNGP-bi
+0.92%
+4.71%
+1.13%
+1.54%
Eng-bi
+1.89%
+5.70%
+2.99%
+2.72%
Table 4. Online Ranking Model Performance: Engagements and Cold-Start Engagement Metrics
Task
Imp. Pos.
Exp. Neg.
Satisfaction
Imp. Neg.
Global-1Day
-0.15%
-0.30%
0.0%
-0.19%
Global-3Day
-0.19%
-0.09%
-0.03%
-0.27%
Table 5. Global Curriculum Learning Offline Performance Evaluations
Task
User Satisfaction
User Growth
User Retention
Global-1Day
-0.71%
-0.19%
-0.12%
Global-3Day
-0.72%
-0.22%
-0.17%
Table 6. Global Curriculum Learning Online Performance Evaluations
Recommendation systems must continuously adapt to evolving user behavior, yet the volume of data generated in large-scale streaming environments makes frequent full retraining impractical. This work investigates how targeted data selection can mitigate performance degradation caused by temporal distributional drift while maintaining scalability. We evaluate a range of representation choices and sampling strategies for curating small but informative subsets of user interaction data. Our results demonstrate that gradient-based representations, coupled with distribution-matching, improve downstream model performance, achieving training efficiency gains while preserving robustness to drift. These findings highlight data curation as a practical mechanism for scalable monitoring and adaptive model updates in production-scale recommendation systems.
Sequential recommender systems typically infer user preferences through single-pass encoding of interaction histories without iterative refinement, relying on increasingly deep architectures to capture complex patterns. In this work, we revisit sequential recommendation from a recursive inference perspective: can user preferences be modeled as a persistent latent state that is recursively refined? We propose RecRec (Recursive Recommendation), a lightweight model that maintains a compact latent state and updates it through a shared recursive module conditioned on interaction evidence. Unlike prior recursive models, RecRec introduces an evidence-anchored correction mechanism that stabilizes refinement by grounding each update in the original interaction context, preventing semantic drift during deep recursive reasoning. Experiments on three benchmark datasets under standard evaluation protocols show that RecRec matches or outperforms state-of-the-art sequential, graph-based, and reasoning-enhanced recommenders while using only 3.9M to 14M parameters. Ablation studies demonstrate that both recursive refinement and the evidence-anchored correction gate contribute significantly to performance, highlighting the effectiveness of recursive latent inference as a scalable alternative to deeper or language-based architectures. Code is available at https://anonymous.4open.science/r/RecRec-6B67/README.md.
Modern Deep Learning Recommendation Models (DLRMs) follow scaling laws with sequence length, driving the frontier toward ultra-long User Interaction History (UIH). However, the industry-standard "Fat Row" paradigm, which pre-materializes these sequences into every training example, creates a storage and I/O wall where data infrastructure usage exceeds GPU training capacity due to data redundancy that is amplified in multi-tenant environments where models with vastly different sequence length requirements share a union dataset. We present a versioned late materialization paradigm that eliminates this redundancy by storing UIH once in a normalized, immutable tier and reconstructing sequences just-in-time during training via lightweight versioned pointers. The system ensures Online-to-Offline (O2O) consistency through a bifurcated protocol that prevents future leakage across both streaming and batch training, while a read-optimized immutable storage layer provides multi-dimensional projection pushdown for heterogeneous model tenants. Disaggregated data preprocessing with pipelined I/O prefetching and data-affinity optimizations masks the latency of training-time sequence reconstruction, keeping training throughput compute-bound by GPUs. Deployed on production DLRMs, the system reduces training data infrastructure resource usage while enabling aggressive sequence length scaling that delivers significant model quality gains, serving as the foundational data infrastructure for modern recommendation model architectures, including HSTU and ULTRA-HSTU.