Performance Guaranteed Regularization

Performance-guaranteed regularization aims to improve the generalization ability and robustness of machine learning models by controlling their complexity and preventing overfitting, thereby enhancing performance on unseen data. Current research focuses on developing novel regularization techniques for various architectures, including federated learning, vision transformers, and retriever-reader models, often employing strategies like prototype aggregation, filter orthogonality, and learnable masking to achieve this goal. These advancements are significant because they offer theoretically grounded methods to improve model reliability and efficiency, impacting fields ranging from image recognition and natural language processing to distributed machine learning. The ultimate goal is to move beyond heuristic regularization methods towards approaches with demonstrably improved performance guarantees.

Papers

October 15, 2024

Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank
Donald Loveland, Xinyi Wu, Tong Zhao, Danai Koutra, Neil Shah, Mingxuan Ju
Real World Human Understanding Visual Perspective Collaborative Filtering Item Embeddings Large Scale Recommender System Low Rank Regularization Performance Guaranteed Regularization

July 20, 2023

Boosting Federated Learning Convergence with Prototype Regularization
Yu Qiao, Huy Q. Le, Choong Seon Hong
Heterogeneous Data FedAvg Algorithm Prototype Based Regularization Global Prototype Performance Guaranteed Regularization

June 16, 2023

Group Orthogonalization Regularization For Vision Models Adaptation and Robustness
Yoav Kurtz, Noga Bar, Raja Giryes
Vision Transformer Native Robustness Adversarial Training Regularization Model Convolutional Filter Orthogonality Regularization Performance Guaranteed Regularization

March 29, 2023

Performance-guaranteed regularization in maximum likelihood method: Gauge symmetry in Kullback -- Leibler divergence
Akihisa Ichiki
Regularization Model Maximum Likelihood Kullback Leibler Divergence Regularization Technique Error Correcting Code Performance Guaranteed Regularization

November 2, 2022

Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models
Shujian Zhang, Chengyue Gong, Xingchao Liu
Question Answering Answer Generation Passage Retrieval Retriever Reader Reading Passage Weight Mask Performance Guaranteed Regularization

June 22, 2022

Information Geometry of Dropout Training
Masanari Kimura, Hideitsu Hino
Regularization Model Structured Dropout Information Geometry Regularization Method Performance Guaranteed Regularization