Training Data Curation

Training data curation studies how to collect, clean, select, label, organize, and mix examples so models learn effectively and generalize to intended settings. Common approaches include quality filtering and deduplication, diversity- or difficulty-aware selection, influence-based scoring, online reweighting, label correction, and targeted synthetic-data generation. Methods are evaluated by downstream performance on tasks such as language-model benchmarks, vision-language reasoning, code generation, and cross-domain medical or speech tests, as well as by data efficiency and robustness. A central tension is that removing noisy or redundant examples can also discard useful rare information, so curation must balance quality with diversity, coverage, and the costs of evaluation and provenance tracking.