Collaborative Training

Collaborative training focuses on leveraging multiple datasets or computing resources to improve the performance and efficiency of machine learning models while addressing privacy concerns. Current research emphasizes federated learning, exploring strategies for optimal data partitioning and model aggregation across diverse datasets and resource-constrained devices, including the use of techniques like parameter-efficient fine-tuning and model splitting. This approach is significant for enhancing model accuracy and robustness in various applications, particularly where data is distributed across multiple parties or devices, such as in healthcare and personalized recommendations, while mitigating data leakage risks.

Papers

September 18, 2024

Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization
Zhi Chen, Lingxiao Jiang
Training Data Limited Memorization Balancing Strategy Centralized Training Promise Collaborative Training

January 24, 2024

How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning
Yuchang Sun, Marios Kountouris, Jun Zhang
Strong Generalization Generalization Performance Non Convex Loss Function Cross Silo Federated Learning Collaborative Training

January 23, 2024

Securing Recommender System via Cooperative Training
Qingyang Wang, Chenwang Wu, Defu Lian, Enhong Chen
Privacy Preserving Collaborative Training Robust Recommendation Training Set Attack

December 1, 2023

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu
Large Language Model Pre Trained Model Parameter Efficient Fine Tuning Natural Language Processing Task Effective Approach Collaborative Training

November 22, 2023

Confidant: Customizing Transformer-based LLMs via Collaborative Edge Training
Yuhao Chen, Yuxuan Yan, Qianqian Yang, Yuanchao Shu, Shibo He, Jiming Chen
Large Language Model Natural Language Processing Transformer Based Large Language Model Parallel Training Collaborative Training Collaborative Edge Training

October 15, 2023

VFLAIR: A Research Library and Benchmark for Vertical Federated Learning
Tianyuan Zou, Zixuan Gu, Yu He, Hideaki Takahashi, Yang Liu, Ya-Qin Zhang
New Benchmark Vertical Federated Learning Collaborative Training VFL Algorithm

August 6, 2023

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining
Xidong Wu, Zhengmian Hu, Jian Pei, Heng Huang
Multi Party Collaborative Training Precision Recall Curve

July 19, 2023

FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning
Chia-Hsiang Kao, Yu-Chiang Frank Wang
Federated Learning Collaborative Training Client Drift Gradual Unfreezing

July 18, 2023

Integration of Large Language Models and Federated Learning
Chaochao Chen, Xiaohua Feng, Yuyuan Li, Lingjuan Lyu, Jun Zhou, Xiaolin Zheng, Jianwei Yin
Large Language Model Large Scale Language Model Decentralized Data Large Language Model Training Collaborative Training Federated Large Language Model

March 16, 2023

Enhancing Text Generation with Cooperative Training
Tong Wu, Hao Wang, Zhongshen Zeng, Wei Wang, Hai-Tao Zheng, Jiaxing Zhang
Language Model Generative Model Pre Trained Language Model Text Generation Discriminative Model Collaborative Training Training Instability

December 2, 2022

Matching DNN Compression and Cooperative Training with Resources and Data Availability
Francesco Malandrino, Giuseppe Di Giacomo, Armin Karamzade, Marco Levorato, Carla Fabiana Chiasserini
Machine Learning Machine Learning Model New Resource NP Complete Collaborative Training Approximate Dynamic Programming DNN Compression Historical Data

November 24, 2022

Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels
Soroosh Tayebi Arasteh, Peter Isfort, Marwin Saehn, Gustav Mueller-Franzes, Firas Khader, Jakob Nikolas Kather, Christiane Kuhl, Sven Nebelung, Daniel Truhn
Deep Learning Medical Image Analysis AI Model Non Uniform Collaborative Training

November 22, 2022

DETRs with Collaborative Hybrid Assignments Training
Zhuofan Zong, Guanglu Song, Yu Liu
Group DETR V2 DETR Training Collaborative Training Non Maximum Suppression

November 21, 2022

Motor Imagery Decoding Using Ensemble Curriculum Learning and Collaborative Training
Georgios Zoumpourlis, Ioannis Patras
Motor Imagery Robust Generalization Motor Imagery Classification Collaborative Training Subject Based Training

November 20, 2022

FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training
Quan Nguyen, Hieu H. Pham, Kok-Seng Wong, Phi Le Nguyen, Truong Thao Nguyen, Minh N. Do
Neural Network Convolutional Neural Network Edge Device Resource Constrained Device Collaborative Training High Performing

September 2, 2022

Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning
Benshun Yin, Zhiyong Chen, Meixia Tao
Multi Objective Optimization Split Learning Edge Intelligence Splitting Algorithm Collaborative Training Model Splitting Federated Split Learning

July 7, 2022

Training Transformers Together
Alexander Borzunov, Max Ryabinin, Tim Dettmers, Quentin Lhoest, Lucile Saulnier, Michael Diskin, Yacine Jernite, Thomas Wolf
Training Data Text to Image Transformer Training Collaborative Training

February 24, 2022

Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?
Alain Andres, Esther Villar-Rodriguez, Javier Del Ser
Environment Feature Sparse Reward Intrinsic Reward Heterogeneous Agent Intrinsic Motivation Reinforcement Learning Environment Collaborative Training Exploration Strategy Heterogeneous Multi Agent Reinforcement Learning

December 7, 2021

Collaborative Learning over Wireless Networks: An Introductory Overview
Emre Ozfatura, Deniz Gunduz, H. Vincent Poor
Machine Learning Model Wireless Network Collaborative Learning Brief Introduction Scalable Algorithm Collaborative Training Time Varying Network

November 11, 2021

AlphaDDA: Strategies for Adjusting the Playing Strength of a Fully Trained AlphaZero System to a Suitable Human Training Partner
Kazuhisa Fujita
Artificial Intelligence System General Strategy Game AI AlphaZero Based Artificial Intelligence Collaborative Training Single Shot Alpha MEPOL Difficulty Adjustment

Collaborative Training

Papers

Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization

How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning

Securing Recommender System via Cooperative Training

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way

Confidant: Customizing Transformer-based LLMs via Collaborative Edge Training

VFLAIR: A Research Library and Benchmark for Vertical Federated Learning

Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining

FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning

Integration of Large Language Models and Federated Learning

Enhancing Text Generation with Cooperative Training

Matching DNN Compression and Cooperative Training with Resources and Data Availability

Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels

DETRs with Collaborative Hybrid Assignments Training

Motor Imagery Decoding Using Ensemble Curriculum Learning and Collaborative Training

FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training

Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning

Training Transformers Together

Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?

Collaborative Learning over Wireless Networks: An Introductory Overview

AlphaDDA: Strategies for Adjusting the Playing Strength of a Fully Trained AlphaZero System to a Suitable Human Training Partner