Feature Normalization

Feature normalization is a crucial preprocessing step in machine learning, aiming to standardize the scale and distribution of input features to improve model performance and prevent bias. Current research focuses on developing normalization techniques tailored to specific data characteristics and model architectures, including applications in federated learning, contrastive learning, and various deep learning models like transformers and convolutional neural networks. These advancements enhance model robustness, generalization, and accuracy across diverse applications, from medical image analysis and speech enhancement to natural language processing and generative adversarial networks. The impact extends to improving the reliability and efficiency of machine learning algorithms in various fields.

Papers

October 31, 2024

Nearest Neighbor Normalization Improves Multimodal Retrieval
Neil Chowdhury, Franklin Wang, Sumedh Shenoy, Douwe Kiela, Sarah Schwettmann, Tristan Thrush
Cross Modal Retrieval Retrieval Model Multimodal Retrieval Feature Normalization Contrastive Model

September 17, 2024

Normalization in Proportional Feature Spaces
Alexandre Benatti, Luciano da F. Costa
Normalization Dictionary Feature Normalization Normalization Method

September 11, 2024

A Simplified Retriever to Improve Accuracy of Phenotype Normalizations by Large Language Models
Daniel B. Hier, Thanh Son Do, Tayo Obafemi-Ajayi
Large Language Model Retriever Reader Feature Normalization Entity Normalization Phenotype Recognition Continuous Valued Phenotype

May 1, 2024

Deep Learning Descriptor Hybridization with Feature Reduction for Accurate Cervical Cancer Colposcopy Image Classification
Saurabh Saini, Kapil Ahuja, Siddartha Chennareddy, Karthik Boddupalli
Local Descriptor Computer Aided Diagnosis Feature Normalization Cervical Cancer Feature Reduction

February 2, 2024

Two Approaches to Diachronic Normalization of Polish Texts
Kacper Dudzic, Filip Graliński, Krzysztof Jassem, Marek Kubis, Piotr Wierzchoń
Constructive Approach Polish Language Feature Normalization Text to Text Transfer Transformer Normalization Method Diachronic Perspective Diachronic Semantic

November 22, 2023

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning
Seongyoon Kim, Gihun Lee, Jaehoon Oh, Se-Young Yun
Data Heterogeneity Federated Averaging Feature Normalization Feature Norm

November 9, 2023

Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse
Ruijie Jiang, Thuan Nguyen, Shuchin Aeron, Prakash Ishwar
Contrastive Learning Negative Sampling Neural Collapse Supervised Contrastive Learning Adam Optimizer unsupeRvised Contrastive Learning Feature Normalization Dimensional Collapse

September 28, 2023

Feature Normalization Prevents Collapse of Non-contrastive Learning Dynamics
Han Bao
Contrastive Learning Self Supervised Representation Learning Learning Dynamic Event Collapse Feature Normalization Non Contrastive Learning

August 18, 2023

Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
Guojun Zhang, Mahdi Beitollahi, Alex Bie, Xi Chen
Integral Role Normalization Dictionary Layer Normalization Feature Normalization

June 14, 2023

Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancement
Hejung Yang, Hong-Goo Kang
Speech Enhancement Supervised Fine Tuning Speech Generation Pre Trained Representation Pre Trained Speech Model Feature Normalization Speech Enhancement Task

June 1, 2023

May 28, 2023

T2FNorm: Extremely Simple Scaled Train-time Feature Normalization for OOD Detection
Sudarshan Regmi, Bibek Panthi, Sakar Dotel, Prashnna K. Gyawali, Danail Stoyanov, Binod Bhattarai
Distribution Sample Understanding Out of Distribution Feature Normalization OOD Detection Overconfident Prediction

April 7, 2023

Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learning
Jae-Hun Lee, Doyoung Yoon, ByeongMoon Ji, Kyungyul Kim, Sangheum Hwang
Self Supervised Learning Visual Representation Learned Representation Feature Normalization Evaluation Practice Transfer Hyperparameter Optimisation

March 31, 2023

VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization
Bingfan Zhu, Yanchao Yang, Xulong Wang, Youyi Zheng, Leonidas Guibas
Neural Radiance Field Novel View Synthesis Feature Normalization REF$^2$ NeRF Non Lambertian Surface

January 27, 2023

Robust Transformer with Locality Inductive Bias and Feature Normalization
Omid Nejati Manzari, Hossein Kashiani, Hojat Asgarian Dehkordi, Shahriar Baradaran Shokouhi
Vision Transformer Adversarial Perturbation Feature Normalization Sign Classification Robust Transformer

December 2, 2022

Compound Batch Normalization for Long-tailed Image Classification
Lechao Cheng, Chaowei Fang, Dingwen Zhang, Guanbin Li, Gang Huang
Batch Normalization Long Tailed Gaussian Mixture Feature Normalization

September 19, 2022

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold
Can Yaras, Peng Wang, Zhihui Zhu, Laura Balzano, Qing Qu
Deep Network Geometric Analysis Neural Collapse Feature Normalization Riemannian Geometry Approach

September 1, 2022

Effective Class-Imbalance learning based on SMOTE and Convolutional Neural Networks
Javad Hassannataj Joloudari, Abdolreza Marefat, Mohammad Ali Nematollahi, Solomon Sunday Oyelere, Sadiq Hussain
Convolutional Neural Network Imbalanced Data Persian Dataset Feature Normalization Synthetic Minority Oversampling Technique Imbalance Learning

July 22, 2022

Adaptive Graph-Based Feature Normalization for Facial Expression Recognition
Yangtao Du, Qingqing Wang, Yujie Xiong
Adaptive Importance Facial Expression Recognition Poisson Surface Reconstruction Feature Normalization Graph Based Feature Normalization

Feature Normalization

Papers

Nearest Neighbor Normalization Improves Multimodal Retrieval

Normalization in Proportional Feature Spaces

A Simplified Retriever to Improve Accuracy of Phenotype Normalizations by Large Language Models

Deep Learning Descriptor Hybridization with Feature Reduction for Accurate Cervical Cancer Colposcopy Image Classification

Two Approaches to Diachronic Normalization of Polish Texts

FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning

Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse

Feature Normalization Prevents Collapse of Non-contrastive Learning Dynamics

Understanding the Role of Layer Normalization in Label-Skewed Federated Learning

Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancement

On the Weight Dynamics of Deep Normalized Networks

Normalization Enhances Generalization in Visual Reinforcement Learning

T2FNorm: Extremely Simple Scaled Train-time Feature Normalization for OOD Detection

Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learning

VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization

Robust Transformer with Locality Inductive Bias and Feature Normalization

Compound Batch Normalization for Long-tailed Image Classification

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold

Effective Class-Imbalance learning based on SMOTE and Convolutional Neural Networks

Adaptive Graph-Based Feature Normalization for Facial Expression Recognition