Training Data Curation

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 104

All topics

Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale
Apr 27, 2026Liang Guo, Ge Song, Litao Deng +9Sequential Recommendation Training Data Curation
Meta Platforms, Inc., USA
GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
Apr 27, 2026Hongxin Li, Yuntao Chen, Zhaoxiang ZhangEfficient VLM Inference Training Data Curation
University of Chinese Academy of Sciences, Beijing, China. · New Laboratory of Pattern Recognition, State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China.
SignDATA: Data Pipeline for Sign Language Translation
Apr 22, 2026Kuanwei Chen, Tingyi LinSign Language Translation Training Data Curation
Computer Science and Information Engineering National Central University, Zhongli, Taiwan · Electrical Engineering National Changhua University Of Education, Changhua, Taiwan
Detoxification for LLM: From Dataset Itself
Apr 21, 2026Wei Shao, Yihang Wang, Gaoyu Zhu +4Training Data Curation LLM Safety
State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
VIDS: A Verified Imaging Dataset Standard for Medical AI
Apr 19, 2026Joan S. Muthu, John ShalenMedical Imaging Training Data Curation
Princeton Medical Systems
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
Apr 19, 2026Wanru Zhao, Yihong Chen, Yuzhi Tang +6Language Model Pretraining LLM Fine-Tuning
University of Cambridge · OATML, University of Oxford · University of Toronto +2
C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment
Apr 17, 2026Pufan Zeng, Yilun Liu, Mingchen Dai +12Multilingual Language Models Synthetic Data Generation
Huawei, China · Huawei, Canada
Not All Invariants Are Equal: Curating Training Data to Accelerate Program Verification with SLMs
Mar 16, 2026Ido Pinto, Yizhak Yisrael Elboher, Haoze Wu +2Fine-Tuning Training Data Curation
Hebrew University of Jerusalem, Israel · Amherst Col · VMware Research by Broadcom, USA.
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Feb 16, 2026Yuchen Yang, Wenze Lin, Enhao Huang +6LLM Fine-Tuning Training Data Curation
1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University +2
Reliable Mislabel Detection for Video Capsule Endoscopy Data
Feb 6, 2026Julia Werner, Julius Oexle, Oliver Bause +5Endoscopy Medical Imaging
Embedded Systems University of Tübingen Tübingen, Germany · Department of Medicine I University Hospital Dresden TU Dresden, Germany
When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning
Jan 12, 2026Chenjie Hao, Weyl Lu, Yuko Ishiwaka +3Adaptive Inference Confidence Estimation in Language Models
University of California, Davis · SoftBank Corp. · Aizip
Removing Noise, not Finding Gold: Quality Filtering for Large-Scale Pretraining
Oct 1, 2025Thiziri Nait Saada, Louis Bethune, Michal Klein +3Language Model Pretraining Training Data Curation
Work done as an intern at Apple · University of Oxford · Apple
Conformal Data Contamination Tests for In-distribution Data Acquisition
Jul 18, 2025Martin V. Vejling, Shashi Raj Pandey, Christophe A. N. Biscio +1Data Contamination Training Data Curation
Department of Mathematical Sciences and Department of Electronic Systems Aalborg University Aalborg, Denmark · Department of Electronic Systems Aalborg University Aalborg, Denmark · Department of Mathematical Sciences Aalborg University Aalborg, Denmark
Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training
Jun 2, 2025Pierre-Carl Langlais, Pavel Chizhov, Catherine Arnett +7Language Model Pretraining Training Data Curation
PleIAs, Paris, France · CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt, Germany · EleutherAI

Apr 27, 2026, cs.IRJ/K move · Enter open · S save

Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale

Liang Guo, Ge Song, Litao Deng, Jianhui Sun+8

Meta Platforms, Inc., USA

Sequential Recommendation Training Data Curation Recommender Systems

Open paper AskPDF arXiv

Modern Deep Learning Recommendation Models (DLRMs) follow scaling laws with sequence length, driving the frontier toward ultra-long User Interaction History (UIH). However, the industry-standard "Fat Row" paradigm, which pre-materializes these sequences into every training example, creates a storage and I/O wall where data infrastructure usage exceeds GPU training capacity due to data redundancy that is amplified in multi-tenant environments where models with vastly different sequence length requirements share a union dataset. We present a \emph{versioned late materialization} paradigm that eliminates this redundancy by storing UIH once in a normalized, immutable tier and reconstructing sequences just-in-time during training via lightweight versioned pointers. The system ensures Online-to-Offline (O2O) consistency through a bifurcated protocol that prevents future leakage across both streaming and batch training, while a read-optimized immutable storage layer provides multi-dimensional projection pushdown for heterogeneous model tenants. Disaggregated data preprocessing with pipelined I/O prefetching and data-affinity optimizations masks the latency of training-time sequence reconstruction, keeping training throughput compute-bound by GPUs. Deployed on production DLRMs, the system reduces training data infrastructure resource usage while enabling aggressive sequence length scaling that delivers significant model quality gains, serving as the foundational data infrastructure for modern recommendation model architectures, including HSTU and ULTRA-HSTU.

Momentum

Related topics

Latest papers 104