Training Convergence

Training convergence in machine learning focuses on achieving efficient and accurate model training, a crucial aspect impacting both performance and resource utilization. Current research emphasizes improving convergence speed and stability across diverse settings, including federated learning (with algorithms like FedRank and AdaptSFL addressing client selection and resource constraints) and object detection (using techniques like hybrid pooling and novel loss functions). These advancements are significant because faster and more reliable convergence translates to reduced training time, lower energy consumption, and improved model accuracy across various applications, from recommendation systems to 3D registration.

Papers

October 30, 2024

ELMGS: Enhancing memory and computation scaLability through coMpression for 3D Gaussian Splatting
Muhammad Salman Ali, Sung-Ho Bae, Enzo Tartaglione
Neural Radiance Field Gaussian Splatting Memory Trace 3D Model Scalable Algorithm Training Convergence

October 15, 2024

On the Training Convergence of Transformers for In-Context Classification
Wei Shen, Ruida Zhou, Jing Yang, Cong Shen
Transformer Megatron Decepticons Classification Code Transformer Based Training Dynamic Gaussian Mixture Different Context Training Convergence

September 13, 2024

Optimization and Generalization Guarantees for Weight Normalization
Pedro Cisneros-Velarde, Zhijie Chen, Sanmi Koyejo, Arindam Banerjee
Optimization Purpose Generalization Guarantee Training Convergence Smooth Activation Function Weight Normalization

May 7, 2024

Ranking-based Client Selection with Imitation Learning for Efficient Federated Learning
Chunlin Tian, Zhan Shi, Xinpeng Qin, Li Li, Chengzhong Xu
Federated Learning Imitation Learning Client Selection Pairwise Learning Training Convergence Device Selection

March 19, 2024

AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks
Zheng Lin, Guanqiao Qu, Wei Wei, Xianhao Chen, Kin K. Leung
Edge Device Split Federated Learning Resource Constrained Edge Model Splitting Training Convergence

January 18, 2024

Improving Local Training in Federated Learning via Temperature Scaling
Kichang Lee, Songkuk Kim, JeongGil Ko
Federated Learning Model Convergence Temperature Scaling Local Training Training Convergence

January 2, 2024

Hybrid Pooling and Convolutional Network for Improving Accuracy and Training Convergence Speed in Object Detection
Shiwen Zhao, Wei Wang, Junhui Hou, Hai Wu
Object Detection Deep Convolutional Neural Network Pooling Layer PAPER Demand \Cite{meir2021market Enhancing Accuracy High Precision Training Convergence

September 14, 2023

FedFNN: Faster Training Convergence Through Update Predictions in Federated Recommender Systems
Francesco Fabbri, Xianghang Liu, Jack R. McKenzie, Bartlomiej Twardowski, Tri Kurniawan Wijaya
Model Training Decentralized Federated Learning Federated Recommendation Accurate Training Training Convergence

April 15, 2023

Align-DETR: Improving DETR with Simple IoU-aware BCE loss
Zhi Cai, Songtao Liu, Guodong Wang, Zheng Ge, Xiangyu Zhang, Di Huang
LD Align Group DETR V2 Deformable DETR Training Convergence BCE Loss Average Precision Loss

February 20, 2023

Hadamard Layer to Improve Semantic Segmentation
Angello Hoyos, Mariano Rivera
Semantic Segmentation Hadamard Layer Training Convergence

October 28, 2022

When does mixup promote local linearity in learned representations?
Arslan Chaudhry, Aditya Krishna Menon, Andreas Veit, Sadeep Jayasumana, Srikumar Ramalingam, Sanjiv Kumar
Meaningful Representation Spatio Temporal Mixup Mechanism Local Linearity Training Convergence Mixup Training Mixup Loss

January 21, 2022

How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Shuai Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong
Strong Generalization Self Training Unlabeled Data Theoretical Analysis Training Convergence

November 19, 2021

What Stops Learning-based 3D Registration from Working in the Real World?
Zheng Dang, Lizhou Wang, Junning Qiu, Minglei Lu, Mathieu Salzmann
Point Cloud Synthetic Data Real World 3D Registration Stop or Go Decision Training Convergence