Training Framework

Training frameworks encompass the design and implementation of efficient and effective methods for teaching machine learning models. Current research emphasizes optimizing training for diverse scenarios, including resource-constrained environments (federated learning, knowledge distillation), privacy-preserving settings (secure split learning), and handling heterogeneous data (multimodal recommendations, cross-task learning). These advancements aim to improve model accuracy, reduce training time and costs, and address critical issues like data privacy and security, impacting various fields from healthcare (robotic surgery) to computer vision (NeRF) and natural language processing.

Papers

September 3, 2024

July 30, 2024

OmniBal: Towards Fast Instruct-tuning for Vision-Language Models via Omniverse Computation Balance
Yongqiang Yao, Jingru Tan, Jiahao Hu, Feizhao Zhang, Xin Jin, Bo Li, Ruihao Gong, Pengfei Liu
Vision Language Model Vision Language Parallel Training Large Scale 3D Boolean Function Training Framework Cultured Meat Recyclable Tuning

June 20, 2024

SeCoKD: Aligning Large Language Models for In-Context Learning with Fewer Shots
Weixing Wang, Haojin Yang, Christoph Meinel
Context Learning Supervised Fine Tuning Self Knowledge Distillation Fewer Shot Training Framework

March 19, 2024

An Aligning and Training Framework for Multimodal Recommendations
Yifan Liu, Kangning Zhang, Xiangyuan Ren, Yanhua Huang, Jiarui Jin, Yingjie Qin, Ruilong Su, Ruiwen Xu, Weinan Zhang
Alignment Problem Multimodal Information Multimodal Feature Training Framework Multimodal Recommendation

March 16, 2024

Federated Learning based on Pruning and Recovery
Chengjie Ma
Edge Pruning Prompt Recovery Heterogeneous Client Asynchronous Learning Training Framework Asynchronous Algorithm

January 18, 2024

A Fast, Performant, Secure Distributed Training Framework For Large Language Model
Wei Huang, Yinggui Wang, Anda Cheng, Aihui Zhou, Chaofan Yu, Lei Wang
Large Language Model System Performance Secure Approach Trusted Execution Environment Model Slicing Training Framework Domain Specific LLM

December 13, 2023

Cooperative Learning for Cost-Adaptive Inference
Xingli Fang, Richard Bradford, Jung-Eun Kim
Deep Neural Network Knowledge Distillation Cooperative Learning Training Framework Sub Network

August 2, 2023

Orientation-Guided Contrastive Learning for UAV-View Geo-Localisation
Fabian Deuser, Konrad Habel, Martin Werner, Norbert Oswald
UAV Image Training Framework Hierarchical Localization

April 18, 2023

Understand Data Preprocessing for Effective End-to-End Training of Deep Neural Networks
Ping Gong, Yuxin Ma, Cheng Li, Xiaosong Ma, Sam H. Noh
Deep Neural Network Raw Data High Fidelity DNN Training Data Preprocessing End to End Training Training Framework State of the Art NVIDIA

March 10, 2023

Digital Twin-Assisted Knowledge Distillation Framework for Heterogeneous Federated Learning
Xiucheng Wang, Nan Cheng, Longfei Ma, Ruijin Sun, Rong Chai, Ning Lu
Knowledge Distillation Q Learning Heterogeneous Federated Learning Model Distillation Training Framework

January 31, 2023

Friend-training: Learning from Models of Different but Related Tasks
Mian Zhang, Lifeng Jin, Linfeng Song, Haitao Mi, Xiabing Zhou, Dong Yu
Full Model Related Task Self Training Semantic Role Labeling Dialogue Understanding Training Framework Domain Self

June 18, 2022

Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression
Xin Jing, Meishu Song, Andreas Triantafyllopoulos, Zijiang Yang, Björn W. Schuller
Loss Function Redundancy Reduction Training Framework Barlow Twin Twin Neural Network

December 14, 2021

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, Bin Cui
New Framework Jina Embeddings Potential Scalability Model Training High Dimensional Data Learned Embeddings Training Framework

December 9, 2021

A Training Framework for Stereo-Aware Speech Enhancement using Deep Neural Networks
Bahareh Tolooshams, Kazuhito Koishida
Deep Neural Network Speech Enhancement Training Framework

December 6, 2021

End-to-end Adaptive Distributed Training on PaddlePaddle
Yulong Ao, Zhihua Wu, Dianhai Yu, Weibao Gong, Zhiqing Kui, Minxu Zhang, Zilingfeng Ye, Liang Shen, Yanjun Ma, Tian Wu, Haifeng Wang, Wei Zeng, Chao Yang
Training Data Adaptive Importance Efficient Pipeline Efficient Scheduling Training Framework

Training Framework

Papers

Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback

$S^2$NeRF: Privacy-preserving Training Framework for NeRF

OmniBal: Towards Fast Instruct-tuning for Vision-Language Models via Omniverse Computation Balance

SeCoKD: Aligning Large Language Models for In-Context Learning with Fewer Shots

An Aligning and Training Framework for Multimodal Recommendations

Federated Learning based on Pruning and Recovery

A Fast, Performant, Secure Distributed Training Framework For Large Language Model

Cooperative Learning for Cost-Adaptive Inference

Orientation-Guided Contrastive Learning for UAV-View Geo-Localisation

Understand Data Preprocessing for Effective End-to-End Training of Deep Neural Networks

Digital Twin-Assisted Knowledge Distillation Framework for Heterogeneous Federated Learning

Friend-training: Learning from Models of Different but Related Tasks

Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework

A Training Framework for Stereo-Aware Speech Enhancement using Deep Neural Networks

End-to-end Adaptive Distributed Training on PaddlePaddle