Training Data
Training data is crucial for machine learning model development, with current research focusing on improving data quality, efficiency, and mitigating biases. Active areas include generating synthetic data to address scarcity or privacy concerns, developing algorithms to optimize data selection and usage (e.g., self-paced learning, active learning), and mitigating issues like data contamination and imbalance through techniques such as data augmentation, selective parameter merging, and novel loss functions. The quality and characteristics of training data significantly impact model performance, generalization, and robustness, influencing various applications from natural language processing and image recognition to scientific computing and medical diagnosis.
Papers
Proactive Gradient Conflict Mitigation in Multi-Task Learning: A Sparse Training Perspective
Zhi Zhang, Jiayi Shen, Congfeng Cao, Gaole Dai, Shiji Zhou, Qizhe Zhang, Shanghang Zhang, Ekaterina Shutova
What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applications
Emily Williams, Amanda Howard, Brek Meuris, Panos Stinis
Certified Training with Branch-and-Bound: A Case Study on Lyapunov-stable Neural Control
Zhouxing Shi, Cho-Jui Hsieh, Huan Zhang
Training and Evaluating Language Models with Template-based Data Generation
Yifan Zhang
On the Reconstruction of Training Data from Group Invariant Networks
Ran Elbaz, Gilad Yehudai, Meirav Galun, Haggai Maron
Brain-like emergent properties in deep networks: impact of network architecture, datasets and training
Niranjan Rajesh, Georgin Jacob, SP Arun
FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data
Jiin Im, Yongho Son, Je Hyeong Hong
Cautious Optimizers: Improving Training with One Line of Code
Kaizhao Liang, Lizhang Chen, Bo Liu, Qiang Liu
Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding
Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zenghui Ding, Xianjun Yang, Yining Sun
Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Chen Qi, Peng Cheng
Memory Backdoor Attacks on Neural Networks
Eden Luzon, Guy Amit, Roy Weiss, Yisroel Mirsky
Exploratory Study Of Human-AI Interaction For Hindustani Music
Nithya Shikarpur, Cheng-Zhi Anna Huang
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Haonan Wang, Qian Liu, Chao Du, Tongyao Zhu, Cunxiao Du, Kenji Kawaguchi, Tianyu Pang
Training Physics-Driven Deep Learning Reconstruction without Raw Data Access for Equitable Fast MRI
Yaşar Utku Alçalar, Merve Gülle, Mehmet Akçakaya