Training Data Curation

Follow this topic Ask about this topic

Training data curation studies how to collect, clean, select, label, organize, and mix examples so models learn effectively and generalize to intended settings. Common approaches include quality filtering and deduplication, diversity- or difficulty-aware selection, influence-based scoring, online reweighting, label correction, and targeted synthetic-data generation. Methods are evaluated by downstream performance on tasks such as language-model benchmarks, vision-language reasoning, code generation, and cross-domain medical or speech tests, as well as by data efficiency and robustness. A central tension is that removing noisy or redundant examples can also discard useful rare information, so curation must balance quality with diversity, coverage, and the costs of evaluation and provenance tracking.

Latest papers

156 papers25 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment
Apr 17, 2026Pufan Zeng, Yilun Liu, Mingchen Dai +12Multilingual Language Models Synthetic Data Generation
Huawei, China · Huawei, Canada
Lit2Vec: A Reproducible Workflow for Building a Legally Screened Chemistry Corpus from S2ORC for Downstream Retrieval and Text Mining
Apr 14, 2026Mahmoud Amiri, Jamile Mohammad Jafari, Sara Mostafapour +1Training Data Curation Data Provenance
Leibniz Institute of Photonic Technology, Member of Leibniz Health Technologies, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Albert-Einstein-Strasse 9, 07745 Jena, Germany. · Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena, Member of the Leibniz Centre for Photonics in Infection Research (LPI), Helmholtzweg 4, 07743 Jena, Germany
Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
Apr 12, 2026Yu Li, Xiaoran Shang, Qizhi Pei +11Training Data Curation Data Provenance
Shanghai Artificial Intelligence Laboratory, OpenDataLab · University of Science and Technology of China · Shanghai Jiao Tong University
CuraLight: Debate-Guided Data Curation for LLM-Centered Traffic Signal Control
Apr 7, 2026Qing Guo, Xinhang Li, Junyu Chen +4Traffic Signal Control Language Model-Based Control
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China · Beijing Big Data Center, Beijing, China
Understanding Emergent Misalignment via Feature Superposition Geometry
Apr 7, 2026Gouki Minegishi, Hiroki Furuta, Takeshi Kojima +2Emergent Misalignment in Language Models LLM Alignment
The University of Tokyo · Google DeepMind
MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale
Apr 6, 2026Bin Wang, Tianyao He, Linke Ouyang +40Document Parsing Training Data Curation
ASI-Evolve: AI Accelerates AI
Mar 31, 2026Weixian Xu, Tiantian Mi, Yixiu Liu +6Autonomous Research Agents Automated Algorithm Discovery
SJTU · SII · GAIR
From Big Data to Fast Data: Towards High-Quality Datasets for Machine Learning Applications from Closed-Loop Data Collection
Mar 31, 2026Philipp Reis, Jacqueline Henle, Stefan Otten +1Training Data Curation Training Data Selection
FZI Research Center for Information Technology, Karlsruhe, Germany
A General Model for Deepfake Speech Detection: Diverse Bonafide Resources or Diverse AI-Based Generators
Mar 29, 2026Lam Pham, Khoi Vu, Dat Tran +4Audio Deepfake Detection Training Data Curation
Austrian Institute of Technology, Austria. · FPT University, Vietnam. · Singapore Institute of Technology, Singapore.
LACON: Training Text-to-Image Model from Uncurated Data
Mar 27, 2026Zhiyang Liang, Ziyu Wan, Hongyu Liu +4Geometry-Conditioned Image Generation T2I Generation
Nanjing University · Microsoft Research · HKUST
Fair Data Pre-Processing with Imperfect Attribute Space
Mar 27, 2026Ying Zheng, Yangfan Jiang, Kian-Lee TanAlgorithmic Fairness Training Data Curation
National University of Singapore, Singapore
PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks
Mar 25, 2026Cheng Cui, Yubo Zhang, Ting Sun +11Optical Character Recognition Training Data Curation
PaddlePaddle Team, Baidu Inc.
The Order Is The Message
Mar 24, 2026Jordan LeDouxData-Efficient Learning Training Data Curation
Independent Researcher
Data Curation for Machine Learning Interatomic Potentials by Determinantal Point Processes
Mar 23, 2026Joanna Zou, Youssef MarzoukTraining Data Curation Training Data Selection
Computational Science & Engineering Massachusetts Institute of Technology Cambridge, MA, 02139, USA
Not All Invariants Are Equal: Curating Training Data to Accelerate Program Verification with SLMs
Mar 16, 2026Ido Pinto, Yizhak Yisrael Elboher, Haoze Wu +2Fine-Tuning Training Data Curation
Hebrew University of Jerusalem, Israel · Amherst Col · VMware Research by Broadcom, USA.
Data Darwinism Part II: DataEvolve -- AI can Autonomously Evolve Pretraining Data Curation
Mar 15, 2026Tiantian Mi, Dongming Shan, Zhen Huang +6Training Data Curation Language Model Pretraining
SII · FDU · GAIR +2
FLUX: Data Worth Training On
Mar 14, 2026Gowtham, Sai Rupesh, Sanjay Kumar +2Training Data Curation LLM Training
Quality over Quantity: Demonstration Curation via Influence Functions for Data-Centric Robot Learning
Mar 10, 2026Haeone Lee, Taywon Min, Junsu Kim +4Training Data Curation Data-Efficient Robot Learning
KAIST · University of California, Berkeley · New York University
Divide and Predict: An Architecture for Input Space Partitioning and Enhanced Accuracy
Mar 9, 2026Fenix W. Huang, Henning S. Mortveit, Christian M. ReidysTraining Data Curation
Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
Mar 8, 2026Zongqian Li, Tengchao Lv, Shaohan Huang +8Training Data Curation
Microsoft Research · University of Cambridge
VIVECaption: A Split Approach to Caption Quality Improvement
Mar 8, 2026Varun Ananth, Baqiao Liu, Haoran CaiImage Captioning Evaluation Image Captioning
Adobe Inc.
Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning
Feb 28, 2026Dariush Wahdany, Matthew Jagielski, Adam Dziedzic +1Privacy-Preserving ML Training Data Curation
CISPA Helmholtz Center for Information Security · Anthropic
Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning
Feb 26, 2026Amita Kamath, Jack Hessel, Khyathi Chandu +3VLM Reasoning Reasoning for VLMs
University of Washington · University of California, Los Angeles · Samaya AI +2
Summer-22B: A Systematic Approach to Dataset Engineering and Training at Scale for Video Foundation Model
Feb 26, 2026Simo Ryu, Chunghwan HanVideo Foundation Models Vision Foundation Models
fal
Effects of Training Data Quality on Classifier Performance
Feb 25, 2026Alan F. Karr, Regina RuaneTraining Data Curation Cross-Dataset Generalization
Fraunhofer USA Center Mid-Atlantic, Riverdale MD 20737 · Computational Social Science Laboratory, University of Pennsylvania, Philadelphia, PA 19104
Learning to Solve Complex Problems via Dataset Decomposition
Feb 23, 2026Wanru Zhao, Lucas Caccia, Zhengyan Shi +3Curriculum Learning Training Data Curation
University of Cambridge · Microsoft Research · Mila - Quebec AI Institute
Automated Quality Check of Sensor Data Annotations
Feb 19, 2026Niklas Freund, Zekiye Ilknur-Öz, Tobias Klockau +3Training Data Curation Intelligent Transportation Systems
Digitale Schiene Deutschland, DB InfraGO AG, Berlin, Germany
ÜberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset
Feb 16, 2026DatologyAI, :, Aldo Gael Carranza +32Multilingual Language Models Training Data Curation
DatologyAI
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Feb 16, 2026Yuchen Yang, Wenze Lin, Enhao Huang +6LLM Fine-Tuning Training Data Curation
1The State Key Laboratory of Blockchain and Data Security, Zhejiang University · 2Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · 3Qiuzhen College, Tsinghua University +2
Reliable Mislabel Detection for Video Capsule Endoscopy Data
Feb 6, 2026Julia Werner, Julius Oexle, Oliver Bause +5Endoscopy Medical Imaging
Embedded Systems University of Tübingen Tübingen, Germany · Department of Medicine I University Hospital Dresden TU Dresden, Germany

Apr 17, 2026J/K move · Enter open · S save

C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment

Pufan Zeng, Yilun Liu, Mingchen Dai, Mengyao Piao+11

Huawei, China · Huawei, Canada

Multilingual Language Models Synthetic Data Generation Training Data Curation Cultural Alignment

Open paper AskPDF arXiv

Achieving cultural alignment in Large Language Models (LLMs) increasingly depends on synthetic data generation. For such synthesis, the most vital initial step is seed curation; however, current methods lack quantifiable standards for selecting these seeds. Existing approaches rely on unscalable manual curation or bias-prone LLM extraction, treating cultural specificity as an abstract concept rather than a measurable signal. In this paper, we address this "quantification gap" by proposing C-Mining, an unsupervised framework that transforms the discovery of cultural seeds from a subjective selection process into a computable data mining formulation. Our approach exploits a novel geometric insight, leveraging the cross-lingual misalignment of cultural concepts within pre-trained embedding spaces as a quantifiable discovery signal. By systematically identifying these regions characterized by pronounced linguistic exclusivity and geometric isolation, while actively filtering out noise, C-Mining automatically extracts high-fidelity Culture Points (CPs) from raw multilingual corpora without reliance on human or LLM supervision, reducing preparation costs by more than 150-fold. We further leverage the mined knowledge to steer the synthesis of diverse instruction-tuning datasets. Extensive experiments demonstrate that this seed-centric approach significantly enhances cultural understanding and reasoning capabilities, achieving a +6.03 point improvement on CulturalBench-Hard and surpassing state-of-the-art baselines, providing a scalable, quantifiable solution for high-quality cultural data synthesis.