Training Data Curation

Ask about this topic

Overview

Training data curation studies how to collect, clean, select, label, organize, and mix examples so models learn effectively and generalize to intended settings. Common approaches include quality filtering and deduplication, diversity- or difficulty-aware selection, influence-based scoring, online reweighting, label correction, and targeted synthetic-data generation. Methods are evaluated by downstream performance on tasks such as language-model benchmarks, vision-language reasoning, code generation, and cross-domain medical or speech tests, as well as by data efficiency and robustness. A central tension is that removing noisy or redundant examples can also discard useful rare information, so curation must balance quality with diversity, coverage, and the costs of evaluation and provenance tracking.

Latest papers

CardsList

156 papers25 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

  1. Effectiveness of Automatically Curated Dataset in Thyroid Nodules Classification Algorithms Using Deep Learning

    Feb 1, 2026Jichen Yang, Jikai Zhang, Benjamin Wildman-Tobriner +1Thyroid Nodule DetectionUltrasound Image Classification

    Department of Electrical & Computer Engineering, Duke University, Durham, NC, USA · Department of Radiology, Duke University Medical Center, Durham, NC, USA

  2. When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

    Jan 12, 2026Chenjie Hao, Weyl Lu, Yuko Ishiwaka +3Adaptive InferenceConfidence Estimation in Language Models

    University of California, Davis · SoftBank Corp. · Aizip

  3. Removing Noise, not Finding Gold: Quality Filtering for Large-Scale Pretraining

    Oct 1, 2025Thiziri Nait Saada, Louis Bethune, Michal Klein +3Language Model PretrainingTraining Data Curation

    Work done as an intern at Apple · University of Oxford · Apple

  4. Conformal Data Contamination Tests for In-distribution Data Acquisition

    Jul 18, 2025Martin V. Vejling, Shashi Raj Pandey, Christophe A. N. Biscio +1Data ContaminationTraining Data Curation

    Department of Mathematical Sciences and Department of Electronic Systems Aalborg University Aalborg, Denmark · Department of Electronic Systems Aalborg University Aalborg, Denmark · Department of Mathematical Sciences Aalborg University Aalborg, Denmark

  5. Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training

    Jun 2, 2025Pierre-Carl Langlais, Pavel Chizhov, Catherine Arnett +7Language Model PretrainingTraining Data Curation

    PleIAs, Paris, France · CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt, Germany · EleutherAI

  6. Learning From How Humans Correct

    Jan 30, 2021Tong GuoLearning with Noisy LabelsTraining Data Curation