Training Data

Training data is crucial for machine learning model development, with current research focusing on improving data quality, efficiency, and mitigating biases. Active areas include generating synthetic data to address scarcity or privacy concerns, developing algorithms to optimize data selection and usage (e.g., self-paced learning, active learning), and mitigating issues like data contamination and imbalance through techniques such as data augmentation, selective parameter merging, and novel loss functions. The quality and characteristics of training data significantly impact model performance, generalization, and robustness, influencing various applications from natural language processing and image recognition to scientific computing and medical diagnosis.

Papers

January 1, 2024

Digger: Detecting Copyright Content Mis-usage in Large Language Model Training
Haodong Li, Gelei Deng, Yi Liu, Kailong Wang, Yuekang Li, Tianwei Zhang, Yang Liu, Guoai Xu, Guosheng Xu, Haoyu Wang
Training Data Diverse Datasets LLM Training Large Language Model Training

December 31, 2023

Training towards significance with the decorrelated event classifier transformer neural network
Jaebak Kim
Neural Network Natural Language Processing Training Data Transformer Based Neural Architecture Importance Aware Event Transformer

December 29, 2023

Generating Enhanced Negatives for Training Language-Based Object Detectors
Shiyu Zhao, Long Zhao, Vijay Kumar B. G, Yumin Suh, Dimitris N. Metaxas, Manmohan Chandraker, Samuel Schulter
Training Data Text to Image Diffusion Model Object Detector Negation Detection Negative Information Language Based

December 27, 2023

GRSDet: Learning to Generate Local Reverse Samples for Few-shot Object Detection
Hefei Mei, Taijin Zhao, Shiyuan Tang, Heqian Qiu, Lanxiao Wang, Minjian Zhang, Fanman Meng, Hongliang Li
Training Data Shot Object Detection Feature Transformation

December 25, 2023

Mixture Data for Training Cannot Ensure Out-of-distribution Generalization
Songming Zhang, Yuxiao Luo, Qizhou Wang, Haoang Chi, Xiaofeng Chen, Bo Han, Jinyan Li
Strong Generalization Training Data Distribution Generalization Generalization Error Generalization Problem Data Mixture

December 23, 2023

Human101: Training 100+FPS Human Gaussians in 100s from 1 View
Mingwei Li, Jiachen Tao, Zongxin Yang, Yi Yang
Training Data Gaussian Splatting 3D Human Best View Human Reconstruction Digital Human

December 22, 2023

Flying By ML -- CNN Inversion of Affine Transforms
L. Van Warren
Machine Learning Training Data Synthetic Image Flight Pattern Analog Gauge Neural Interpolation Affine Transform

December 20, 2023

Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin, Xiao Wang, Romain Egele, Guojing Cong, Feiyi Wang, Prasanna Balaprakash
Large Language Model Training Data Multi Scale Efficient Optimization New Frontier Parallel Training Strong Scaling Exascale Computing

December 18, 2023

Training With "Paraphrasing the Original Text'' Improves Long-Context Performance
Yijiong Yu
Training Data Long Context Long Context Task Long Context Scenario Long Context Input

December 17, 2023

Meta-AF Echo Cancellation for Improved Keyword Spotting
Jonah Casebeer, Junkai Wu, Paris Smaragdis
Training Data Speech Recognition Keyword Spotting Adaptive Filter Echo Cancellation

December 16, 2023

Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data
Kang Lin, Reinhard Heckel
Deep Learning Native Robustness Training Data Image Reconstruction Complementary Benefit Robust Model Accelerated Magnetic Resonance Imaging

December 14, 2023

Audio-visual fine-tuning of audio-only ASR models
Avner May, Dmitriy Serdyuk, Ankit Parag Shah, Otavio Braga, Olivier Siohan
Training Data Automatic Speech Recognition Visual Speech One Shot Video Tuning Self Supervised Audio Visual

December 13, 2023

Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach
Lucas Luttner
Neural Network Training Data Mixture Component Expert Knowledge Aleatoric Uncertainty Data Uncertainty Input Uncertainty

December 12, 2023

Deep Internal Learning: Deep Learning from a Single Input
Tom Tirer, Raja Giryes, Se Young Chun, Yonina C. Eldar
Neural Network Deep Learning Training Data Deep Internal Learning

December 11, 2023

December 10, 2023

Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
Oded Ovadia, Menachem Brief, Moshik Mishaeli, Oren Elisha
Large Language Model Training Data Fine Tuning App to App Retrieval Knowledge Injection Knowledge Intensive Task Unsupervised Fine Tuning

Training Data

Papers

Digger: Detecting Copyright Content Mis-usage in Large Language Model Training

Training towards significance with the decorrelated event classifier transformer neural network

Generating Enhanced Negatives for Training Language-Based Object Detectors

The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines

SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks

Layer Attack Unlearning: Fast and Accurate Machine Unlearning via Layer Level Attack and Knowledge Distillation

GRSDet: Learning to Generate Local Reverse Samples for Few-shot Object Detection

Mixture Data for Training Cannot Ensure Out-of-distribution Generalization

Human101: Training 100+FPS Human Gaussians in 100s from 1 View

Flying By ML -- CNN Inversion of Affine Transforms

Optimizing Distributed Training on Frontier for Large Language Models

Training With "Paraphrasing the Original Text'' Improves Long-Context Performance

Meta-AF Echo Cancellation for Improved Keyword Spotting

Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data

Audio-visual fine-tuning of audio-only ASR models

Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach

Deep Internal Learning: Deep Learning from a Single Input

Modyn: A Platform for Model Training on Dynamic Datasets With Sample-Level Data Selection

From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to Fine-grained

Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs