Gradient Clustering

Gradient clustering is a family of machine learning techniques that leverage gradients of model parameters to perform data clustering, addressing challenges in traditional methods. Current research focuses on developing distributed and robust gradient clustering algorithms, including those handling non-independent and identically distributed (non-IID) data and long-tailed distributions, often employing strategies like gradient balancing and neighborhood information exchange. These advancements improve clustering performance in complex scenarios, such as those with outliers or imbalanced class distributions, and enable efficient decentralized learning across networks of devices. The resulting improvements have significant implications for various applications, including robust group inference and improved model training on real-world datasets.

Papers

February 2, 2024

A Unified Framework for Gradient-based Clustering of Distributed Data
Aleksandar Armacki, Dragana Bajović, Dušan Jakovetić, Soummya Kar
Unified Framework Data Clustering Data Distribution Clustering Loss Gradient Clustering

October 31, 2023

Long-Tailed Learning as Multi-Objective Optimization
Weiqi Li, Fan Lyu, Fanhua Shang, Liang Wan, Wei Feng
Multi Objective Optimization Long Tailed Recognition Long Tailed Learning Gradient Clustering

October 13, 2022

Outlier-Robust Group Inference via Gradient Space Clustering
Yuchen Zeng, Kristjan Greenewald, Kangwook Lee, Justin Solomon, Mikhail Yurochkin
Machine Learning Model Group Robustness Group Annotation Downstream Performance Gradient Clustering

September 28, 2022

Neighborhood Gradient Clustering: An Efficient Decentralized Learning Method for Non-IID Data Distributions
Sai Aparna Aketi, Sangamesh Kodge, Kaushik Roy
Decentralized Learning Data Distribution Local Gradient Model Gradient Gradient Clustering

Gradient Clustering

Papers

A Unified Framework for Gradient-based Clustering of Distributed Data

Long-Tailed Learning as Multi-Objective Optimization

Outlier-Robust Group Inference via Gradient Space Clustering

Neighborhood Gradient Clustering: An Efficient Decentralized Learning Method for Non-IID Data Distributions