Local Learning

Local learning is a machine learning paradigm that addresses the limitations of traditional end-to-end training by partitioning models into independently trained modules, reducing memory consumption and enabling parallelism. Current research focuses on improving the accuracy and efficiency of local learning methods, particularly within deep neural networks (including transformers and convolutional neural networks), often employing auxiliary networks to facilitate information exchange between modules and employing techniques like knowledge distillation and mutual information maximization. This approach holds significant promise for training larger, more complex models on resource-constrained platforms and for enhancing the interpretability and biological plausibility of artificial neural networks, impacting both theoretical understanding and practical applications in various domains.

Papers

November 6, 2024

Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation
Shreya Gummadi, Mateus V. Gasparino, Deepak Vasisht, Girish Chowdhary
Autonomous Robot Real Robot Autonomous Robot Navigation Local Learning Dynamic Bandwidth Federated Clustering

November 4, 2024

Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation
Satoki Ishikawa, Rio Yokota, Ryo Karakida
Loss Function Predictive Coding Deep Linear Local Learning Infinite Width Square Root

September 20, 2024

A Ring-Based Distributed Algorithm for Learning High-Dimensional Bayesian Networks
Jorge D. Laborda, Pablo Torrijos, José M. Puerta, José A. Gámez
Practical Algorithm Bayesian Network Local Learning Greedy Equivalence Search

August 21, 2024

LAKD-Activation Mapping Distillation Based on Local Learning
Yaoze Zhang, Yuming Zhang, Yu Zhao, Yue Zhang, Feiyu Zhu
Knowledge Distillation Local Learning Map Distillation

June 24, 2024

MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network
Yuming Zhang, Shouxin Zhang, Peizhe Wang, Feiyu Zhu, Dongzhi Guan, Junhao Su, Jiabin Liu, Changpeng Cai
Local Learning Auxiliary Network

June 7, 2024

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation
Yibo Yang, Xiaojie Li, Motasem Alfarra, Hasan Hammoud, Adel Bibi, Philip Torr, Bernard Ghanem
Dynamic ModulE Local Learning Local Gradient Gradient Correction Global Deep

June 1, 2024

Advancing Supervised Local Learning Beyond Classification with Long-term Feature Bank
Feiyu Zhu, Yuming Zhang, Changpeng Cai, Chenghao He, Xiuyuan Guo, Jiao Li, Peizhe Wang, Junhao Su, Jialin Gao
End to End Local Learning Traditional Deep Learning Auxiliary Network

February 27, 2024

Scaling Supervised Local Learning with Augmented Auxiliary Networks
Chenxiang Ma, Jibin Wu, Chenyang Si, Kay Chen Tan
Local Learning Non Local Learning Augmented Auxiliary Network

December 20, 2023

Unlocking Deep Learning: A BP-Free Approach for Parallel Block-Wise Training of Neural Networks
Anzhe Cheng, Zhenkun Wang, Chenzhong Yin, Mingxi Cheng, Heng Ping, Xiongye Xiao, Shahin Nazarian, Paul Bogdan
Neural Network Deep Learning End to End Back Propagation Local Learning Backpropagation Free

December 4, 2023

FLea: Improving federated learning on scarce and label-skewed data via privacy-preserving feature augmentation
Tong Xia, Abhirup Ghosh, Cecilia Mascolo
LeArning Abstract Label Distribution Model Aggregation Parallel Training Local Learning Rat Image

December 2, 2023

Learning county from pixels: Corn yield prediction with attention-weighted multiple instance learning
Xiaoyu Wang, Yuchi Ma, Qunying Huang, Zhengwei Yang, Zhou Zhang
Multiple Instance Learning Satellite Imagery Tetromino Pixel Yield Prediction Local Learning

November 29, 2023

The Trifecta: Three simple techniques for training deeper Forward-Forward networks
Thomas Dooms, Ing Jyh Tsang, Jose Oramas
Back Propagation Forward Forward Algorithm Local Learning Local Training Simple Method Forward Forward

August 24, 2023

FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning
Gihun Lee, Minchan Jeong, Sangmook Kim, Jaehoon Oh, Se-Young Yun
Local Learning Orthogonal Statistical Learning Weight Perturbation

August 15, 2023

MOLE: MOdular Learning FramEwork via Mutual Information Maximization
Tianchao Li, Yulong Pei
Neural Network Local Learning Mutual Information Maximization Local Optimization Modular Learning

June 2, 2023

May 5, 2023

All models are local: time to replace external validation with recurrent local validation
Alex Youssef, Michael Pencina, Anshul Thakur, Tingting Zhu, David Clifton, Nigam H. Shah
Full Model Time Matter External Validation Local Learning Clinical Machine Learning Predictive Reliability

January 18, 2023

Local Learning with Neuron Groups
Adeetya Patel, Michael Eickenberg, Eugene Belilovsky
Deep Learning Local Learning Local Learning Rule Parallel Learning

December 29, 2022

Local Learning on Transformers via Feature Reconstruction
Priyank Pathak, Jingwei Zhang, Dimitris Samaras
Convolutional Neural Network Transformer Megatron Decepticons CNN Model Local Learning Feature Reconstruction

December 11, 2022

Client Selection for Federated Bayesian Learning
Jiarong Yang, Yuan Liu, Rahif Kassab
Client Selection Bayesian Learning Model Convergence Local Learning Stein Variational Gradient Descent Stein Discrepancy

Local Learning

Papers

Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation

Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation

A Ring-Based Distributed Algorithm for Learning High-Dimensional Bayesian Networks

LAKD-Activation Mapping Distillation Based on Local Learning

MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation

Advancing Supervised Local Learning Beyond Classification with Long-term Feature Bank

Scaling Supervised Local Learning with Augmented Auxiliary Networks

Unlocking Deep Learning: A BP-Free Approach for Parallel Block-Wise Training of Neural Networks

FLea: Improving federated learning on scarce and label-skewed data via privacy-preserving feature augmentation

Learning county from pixels: Corn yield prediction with attention-weighted multiple instance learning

The Trifecta: Three simple techniques for training deeper Forward-Forward networks

FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning

MOLE: MOdular Learning FramEwork via Mutual Information Maximization

Learning Local to Global Feature Aggregation for Speech Emotion Recognition

Federated Learning of Models Pre-Trained on Different Features with Consensus Graphs

All models are local: time to replace external validation with recurrent local validation

Local Learning with Neuron Groups

Local Learning on Transformers via Feature Reconstruction

Client Selection for Federated Bayesian Learning