Training Data

Training data is crucial for machine learning model development, with current research focusing on improving data quality, efficiency, and mitigating biases. Active areas include generating synthetic data to address scarcity or privacy concerns, developing algorithms to optimize data selection and usage (e.g., self-paced learning, active learning), and mitigating issues like data contamination and imbalance through techniques such as data augmentation, selective parameter merging, and novel loss functions. The quality and characteristics of training data significantly impact model performance, generalization, and robustness, influencing various applications from natural language processing and image recognition to scientific computing and medical diagnosis.

Papers

June 5, 2023

On training locally adaptive CP
Nicolo Colombo
Training Data Conformal Prediction Prediction Interval Gradient Based Algorithm

June 4, 2023

Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation
Yunhe Gao, Zhuowei Li, Di Liu, Mu Zhou, Shaoting Zhang, Dimitris N. Metaxas
Training Data Medical Image Segmentation Image Segmentation Medical Image Datasets Internal Medicine DOCTOR

June 3, 2023

Flew Over Learning Trap: Learn Unlearnable Samples by Progressive Staged Training
Pucheng Dang, Xing Hu, Kaidi Xu, Jinhao Duan, Di Huang, Husheng Han, Rui Zhang, Zidong Du, Qi Guo, Yunji Chen
Training Data Unlearning Framework Unlearning Method Left Leaning Feature Perturbation Memorized Sample

June 2, 2023

June 1, 2023

Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes
Anant Khandelwal, Happy Mittal, Shreyas Sunil Kulkarni, Deepak Gupta
Zero Shot Training Data E Commerce Attribute Value Extraction Unstructured Information NER Model

May 31, 2023

May 30, 2023

Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias
Yu Yang, Eric Gan, Gintare Karolina Dziugaite, Baharan Mirzasoleiman
Training Data Inductive Bias Camera Lens Spurious Correlation Spurious Feature Simplicity Bias Worst Group Accuracy

May 29, 2023

May 28, 2023

May 27, 2023

The Curse of Recursion: Training on Generated Data Makes Models Forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, Ross Anderson
Large Language Model Language Model Generative Model Training Data Full Model Reversal Curse Recursive Algorithm Model Generated Model Collapse

May 26, 2023

Training Data

Papers

On training locally adaptive CP

Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation

Flew Over Learning Trap: Learn Unlearnable Samples by Progressive Staged Training

Fair multilingual vandalism detection system for Wikipedia

VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations

Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes

Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training

A Bayesian Approach To Analysing Training Data Attribution In Deep Learning

FedCSD: A Federated Learning Based Approach for Code-Smell Detection

End-to-end Training of Deep Boltzmann Machines by Unbiased Contrastive Divergence with Local Mode Initialization

Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias

Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning

Membership Inference Attacks against Language Models via Neighbourhood Comparison

Transfer Learning for Power Outage Detection Task with Limited Training Data

Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning

Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers

The Curse of Recursion: Training on Generated Data Makes Models Forget

Double Descent and Overfitting under Noisy Inputs and Distribution Shift for Linear Denoisers

Training Socially Aligned Language Models on Simulated Social Interactions

An Investigation of Noise in Morphological Inflection