Training Data

Training data is crucial for machine learning model development, with current research focusing on improving data quality, efficiency, and mitigating biases. Active areas include generating synthetic data to address scarcity or privacy concerns, developing algorithms to optimize data selection and usage (e.g., self-paced learning, active learning), and mitigating issues like data contamination and imbalance through techniques such as data augmentation, selective parameter merging, and novel loss functions. The quality and characteristics of training data significantly impact model performance, generalization, and robustness, influencing various applications from natural language processing and image recognition to scientific computing and medical diagnosis.

Papers

May 28, 2022

Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead
Niccolò Cavagnero, Fernando Dos Santos, Marco Ciccone, Giuseppe Averta, Tatiana Tommasi, Paolo Rech
Deep Neural Network Training Data Deep Neural Network Training Transient Fault

May 23, 2022

May 20, 2022

SE-MoE: A Scalable and Efficient Mixture-of-Experts Distributed Training and Inference System
Liang Shen, Zhihua Wu, WeiBao Gong, Hongxiang Hao, Yangfan Bai, HuaChao Wu, Xinxuan Wu, Jiang Bian, Haoyi Xiong, Dianhai Yu, Yanjun Ma
Training Data Mixture of Expert Efficient Inference Inference System Scalable Inference DeepSpeed Data Efficiency

May 19, 2022

Dataset Pruning: Reducing Training Data by Examining Generalization Influence
Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, Ping Li
Deep Learning Training Data Generalization Gap Training Datasets Generalization Behavior Dataset Pruning

May 18, 2022

Large Neural Networks Learning from Scratch with Very Few Data and without Explicit Regularization
Christoph Linse, Thomas Martinetz
Neural Network Convolutional Neural Network Training Data CNN Model Scratch Project Fine Grained Image Classification Explicit Regularization

May 16, 2022

May 14, 2022

What do Models Learn From Training on More Than Text? Measuring Visual Commonsense Knowledge
Lovisa Hagström, Richard Johansson
Language Model Training Data Text Modality Multimodal Model Model Learning Multi Modal Training Visual Commonsense

May 10, 2022

Accelerating the Training of Video Super-Resolution Models
Lijian Lin, Xintao Wang, Zhongang Qi, Ying Shan
Training Data Video Super Resolution Neural Multigrid

May 9, 2022

Transfer Learning Based Efficient Traffic Prediction with Limited Training Data
Sajal Saha, Anwar Haque, Greg Sidebottom
Training Data Transfer Learning Long Short Term Memory Traffic Prediction

May 7, 2022

Training from Zero: Radio Frequency Machine Learning Data Quantity Forecasting
William H. Clark, Alan J. Michaels
Machine Learning Training Data Better Zero Training Datasets Modulation Classification

May 6, 2022

A Data Cartography based MixUp for Pre-trained Language Models
Seo Yeon Park, Cornelia Caragea
Training Data Pre Trained Language Model Training Sample Local Mixup

May 5, 2022

Efficient yet Competitive Speech Translation: FBK@IWSLT2022
Marco Gaido, Sara Papi, Dennis Fucci, Giuseppe Fiameni, Matteo Negri, Marco Turchi
Training Data High Efficiency Speech Translation Simultaneous Speech Translation Lightweight Continual

May 4, 2022

Training Data

Papers

Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead

Understanding new tasks through the lens of training data via exponential tilting

Training and Inference on Any-Order Autoregressive Models the Right Way

On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions

The Document Vectors Using Cosine Similarity Revisited

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

IDEAL: Query-Efficient Data-Free Learning from Black-box Models

SE-MoE: A Scalable and Efficient Mixture-of-Experts Distributed Training and Inference System

Dataset Pruning: Reducing Training Data by Examining Generalization Influence

Large Neural Networks Learning from Scratch with Very Few Data and without Explicit Regularization

Chemical transformer compression for accelerating both training and inference of molecular modeling

Heroes, Villains, and Victims, and GPT-3: Automated Extraction of Character Roles Without Training Data

What do Models Learn From Training on More Than Text? Measuring Visual Commonsense Knowledge

Accelerating the Training of Video Super-Resolution Models

Transfer Learning Based Efficient Traffic Prediction with Limited Training Data

Training from Zero: Radio Frequency Machine Learning Data Quantity Forecasting

A Data Cartography based MixUp for Pre-trained Language Models

Efficient yet Competitive Speech Translation: FBK@IWSLT2022

Provably Confidential Language Modelling

SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training