Global Aggregation

Global aggregation in machine learning focuses on efficiently combining information from distributed sources or across different levels of representation within a model to improve performance and scalability. Current research emphasizes developing novel aggregation techniques for federated learning, often employing hierarchical architectures and incorporating mechanisms to handle data heterogeneity, Byzantine attacks, and communication constraints. These advancements are crucial for enabling large-scale model training while preserving data privacy and improving the accuracy and efficiency of various applications, including image segmentation, object detection, and natural language processing. The development of efficient and robust global aggregation methods is driving progress in distributed machine learning and impacting the performance of numerous AI systems.

Papers

September 10, 2024

Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models
Yao Shu, Wenyang Hu, See-Kiong Ng, Bryan Kian Hsiang Low, Fei Richard Yu
Parameter Efficient Fine Tuning Visual Analogue Scale Global Aggregation Federated Tuning

September 2, 2024

From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregation
Yunfei Xie, Cihang Xie, Alan Yuille, Jieru Mei
Arbitrary Object Segmentation Performance Tetromino Pixel Object Segmentation Part Segmentation Hierarchical Model Global Aggregation segmentatiOn Challenge

May 10, 2024

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data
Rongyu Zhang, Yun Chen, Chenrui Wu, Fangxin Wang, Bo Li
Model Bias Long Tailed Data Heterogeneous Setting Global Aggregation

April 12, 2024

Anti-Byzantine Attacks Enabled Vehicle Selection for Asynchronous Federated Learning in Vehicular Edge Computing
Cui Zhang, Xiao Xu, Qiong Wu, Pingyi Fan, Qiang Fan, Huiling Zhu, Jiangzhou Wang
Deep Reinforcement Learning Vehicle Plan Asynchronous Federated Learning Byzantine Attack Vehicle Data Global Aggregation Vehicular Edge Computing Vehicle Edge Computing

December 13, 2023

Accelerating the Global Aggregation of Local Explanations
Alon Mor, Yonatan Belinkov, Benny Kimelfeld
Adversarial Example Local Explanation Anchor Graph Global Aggregation

July 17, 2023

A Secure Aggregation for Federated Learning on Long-Tailed Data
Yanna Jiang, Baihe Ma, Xu Wang, Guangsheng Yu, Caijun Sun, Wei Ni, Ren Ping Liu
Secure Aggregation Long Tailed Data Byzantine Attack Model Attack Global Aggregation

June 23, 2023

FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learning
Rishub Tamirisa, John Won, Chengjun Lu, Ron Arel, Andy Zhou
Fine Tuning Many Parameter Personalized Federated Learning Global Aggregation

May 17, 2023

SIMGA: A Simple and Effective Heterophilous Graph Neural Network with Efficient Global Aggregation
Haoyu Liu, Ningyi Liao, Siqiang Luo
Graph Neural Network Graph Learning GNN Architecture Global Aggregation

April 21, 2023

Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs
Matthieu Cordonnier, Nicolas Keriven, Nicolas Tremblay, Samuel Vaiter
Early Stage Convergence Graph Laplacian Message Passing Graph Neural Network Non Asymptotic Random Graph Model Global Aggregation

April 17, 2023

LLA-FLOW: A Lightweight Local Aggregation on Cost Volume for Optical Flow Estimation
Jiawei Xu, Zongqing Lu, Qingmin Liao
Optical Flow Feature Map Global Aggregation Local Similarity Cost Volume Local Aggregation Region Adaptive Constraint

April 14, 2023

DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection
Zongheng Tang, Yifan Sun, Si Liu, Yi Yang
Weak Supervision Weakly Supervised Group DETR V2 Cross Domain Object Detection Image Prediction Global Aggregation

August 24, 2022

PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model
Tao Guo, Song Guo, Junxiao Wang, Wenchao Xu
Foundation Model Federated Learning Complex Prompt Federated Prompt Cooperation Learnable Prompt Global Aggregation

July 19, 2022

Over-the-Air Federated Edge Learning with Hierarchical Clustering
Ozan Aygün, Mohammad Kazemi, Deniz Gündüz, Tolga M. Duman
Federated Learning Hierarchical Clustering Faster Convergence Air Federated Learning Global Aggregation Air Aggregation Two Stage Aggregation

May 3, 2022

Revisiting Communication-Efficient Federated Learning with Balanced Global and Local Updates
Zhigang Yan, Dong Li, Zhichao Zhang, Jiguang He
Loss Function Global Context Local Model Local Context Optimal Trade Global Aggregation

March 8, 2022

DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction
Jiajun Fei, Ziyu Zhu, Wenlei Liu, Zhidong Deng, Mingyang Li, Huanjun Deng, Shuo Zhang
Neural Network Feature Extraction Permutation Invariant Global Aggregation Local Aggregation

December 8, 2021

Improving Image Restoration by Revisiting Global Information Aggregation
Xiaojie Chu, Liangyu Chen, Chengpeng Chen, Xin Lu
Video Deblurring Image Denoising Image Restoration Task Global Aggregation

Global Aggregation

Papers

Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models

From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregation

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Anti-Byzantine Attacks Enabled Vehicle Selection for Asynchronous Federated Learning in Vehicular Edge Computing

Accelerating the Global Aggregation of Local Explanations

A Secure Aggregation for Federated Learning on Long-Tailed Data

FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learning

SIMGA: A Simple and Effective Heterophilous Graph Neural Network with Efficient Global Aggregation

Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs

LLA-FLOW: A Lightweight Local Aggregation on Cost Volume for Optical Flow Estimation

DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection

PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model

Over-the-Air Federated Edge Learning with Hierarchical Clustering

Revisiting Communication-Efficient Federated Learning with Balanced Global and Local Updates

DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction

Improving Image Restoration by Revisiting Global Information Aggregation