Normalization

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Normalization.

Period ending 2026-09-14

4 new papers

A weekly snapshot of new work published in Normalization.

Inside this field

Focused directions

Focused topic

Normalization

Within Normalization

80 papersExplore

Focused topic

Batch Normalization

Within Normalization

67 papersExplore

Focused topic

Norm

Within Normalization

33 papersExplore

166 papers

Latest in Normalization

Browse all topics →

Attention Sinks and Outliers in Attention Residuals
May 18, 2026Haozheng Luo, Haoran Dai, Shaoyang Zhang +10Batch Normalization Outliers
No Free Swap: Protocol-Dependent Layer Redundancy in Transformers
May 15, 2026Gabriel GarciaTransformer Layers Transformer Architectures
TAPIOCA: Why Task- Aware Pruning Improves OOD model Capability
May 14, 2026Krish Sharma, Omar Naim, Soumadeep Saha +3Out-Of-Distribution Chain-Of-Thought Distillation
Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm
May 14, 2026Yuxin Guo, Yihao Yue, Yunhao Ni +4Batch Normalization Graph Neural Network Inference
The fitness landscape of social norms in social dilemmas
May 13, 2026Maximilian Puelma TouzelGame Theory Social Norms
Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models
May 12, 2026Furkan Mumcu, Michael J. Jones, Anoop Cherian +1Video Anomaly Understanding Anomaly Detection
Trajectory-Agnostic Asteroid Detection in TESS with Deep Learning
May 12, 2026Brian P. Powell, Jorge Martinez-Palomera, Amy Tuson +3Phase-Folded Light Curves Deep Learning
Information theoretic underpinning of self-supervised learning by clustering
May 12, 2026Josef Kittler, Sara Atito, Muhammad AwaisSelf-Supervised Kullback-Leibler Divergence
Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shaping
May 11, 2026Haoyuan Sun, Jing Wang, Yuxin Song +9Text-To-Image Reinforcement Learning Post-Training
NoRIN: Backbone-Adaptive Reversible Normalization for Time-Series Forecasting
May 11, 2026Shun Zhang, Yuyang XiaoNormalization Reparameterization
When Normality Shifts: Risk-Aware Test-Time Adaptation for Unsupervised Tabular Anomaly Detection
May 11, 2026Wei Huang, Hezhe Qiao, Kailai Zhang +3Unsupervised Unlabeled Data
Simply Stabilizing the Loop via Fully Looped Transformer
May 11, 2026Rao Fu, Zixuan Yang, Jiankun Zhang +4Transformer Architectures Batch Normalization
Learning Unified Representations of Normalcy for Time Series Anomaly Detection
May 10, 2026Prithul Sarker, Sushmita Sarker, Nicholas G. Murray +1Time-Series Anomaly Detection Anomaly Detection
SCENE: Recognizing Social Norms and Sanctioning in Group Chats
May 8, 2026Mateusz Jacniacki, Maksymilian BilskiSocial Norms Social Dynamics
QuadNorm: Resolution-Robust Normalization for Neural Operators
May 8, 2026Bum Jun Kim, Makoto Kawano, Yusuke Iwasawa +1Batch Normalization Neural Operators
Normalized Architectures are Natively 4-Bit
May 7, 2026Maxim Fishman, Brian Chmiel, Ron Banner +2Full-Precision Performance Low-Bit
Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
May 6, 2026Xuan Qi, Yi Wei, Fanqi Yu +3Batch Normalization Affine
Demystifying Manifold Constraints in LLM Pre-training
May 6, 2026Kang An, Jiaxiang Li, Donald Goldfarb +1Riemannian Optimization Large Language Model Pretraining
Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising
May 5, 2026Youssef Saied, François FleuretExact Equivariance Batch Normalization
Batch Normalization for Neural Networks on Complex Domains
May 1, 2026Xuan Son Nguyen, Nistor GrozavuBatch Normalization Neural Network
PAMod: Modeling Cyclical Shifts via Phase-Amplitude Modulation for Non-stationary Time Series Forecasting
May 1, 2026Yingbo Zhou, Yutong Ye, Shuhao Li +5Periodic Domains Modulation
Towards Seamless Lunar Mosaics: Deep Radiometric Normalization for Cross-Sensor Orbital Imagery Using Chandrayaan-2 TMC Data
Apr 28, 2026Pratincha Singh, Jai Gopal Singla, Prashant Hemrajani +3Lunar Surface Satellite Imagery
Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
Apr 26, 2026Giansalvo CirrincioneTransformer Architectures Batch Normalization
Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
Apr 26, 2026Dawon Choi, Hana Kim, Ji-Hoon KimBatch Normalization Gumbel-Softmax Relaxation
When Does Removing LayerNorm Help? Activation Bounding as a Regime-Dependent Implicit Regularizer
Apr 25, 2026Lucky VermaBatch Normalization Activation Functions
K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning
Apr 24, 2026Zixuan Xia, Quanxi LiPolicy Gradient Extended Kalman Filter
Voice Under Revision: Large Language Models and the Normalization of Personal Narrative
Apr 24, 2026Tom van NuenenRewrite Linguistics
ZC-Swish: Stabilizing Deep BN-Free Networks for Edge and Micro-Batch Applications
Apr 21, 2026Suvinava BasakBatch Normalization Well-Tuned Static Batching Policy
Nexusformer: Nonlinear Attention Expansion for Stable and Inheritable Transformer Scaling
Apr 21, 2026Weijie Zhao, Mingquan Liu, Bolun Wang +4Mixformer Softmax Attention
An `Inverse' Experimental Framework to Estimate Market Efficiency
Apr 20, 2026Thomas Asikis, Heinrich H. NaxMatching Markets Market
FLARE: Task-agnostic embedding model evaluation through a normalization process
Apr 19, 2026Jingzhou Jiang, Yixuan Tang, Yi Yang +1Embedding Dimension Model Evaluation
A Mechanism Study of Delayed Loss Spikes in Batch-Normalized Linear Models
Apr 18, 2026Peifeng Gao, Wenyi Fang, Yang Zheng +1Batch Normalization Learning Rates
Hierarchical Codec Diffusion for Video-to-Speech Generation
Apr 17, 2026Jiaxin Ye, Gaoxiang Cong, Chenhui Wang +4Joint Audio-Video Generation Audio-Visual Consistency
Geometry Reinforced Efficient Attention Tuning Equipped with Normals for Robust Stereo Matching
Apr 10, 2026Jiahao Li, Xinhong Chen, Zhengmin Jiang +3Stereo Matching Synthetic-To-Real Domain Gap
Fine-Tuning Integrity for Modern Neural Networks: Structured Drift Proofs via Norm, Rank, and Sparsity Certificates
Apr 6, 2026Zhenhang Shang, Yingzhe Yu, Kani ChenFinetuning Zero-Knowledge Proofs
When Does Sparsity Mitigate the Curse of Depth in LLMs
Mar 16, 2026Dilxat Muhtar, Xinyuan Song, Sebastian Pokutta +4Sparse Expert Activation Sparsity
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
Feb 17, 2026Jialu Wang, Heinrich Peters, Asad A. Butt +6Preference Alignment Group Relative Policy Optimization
Entropy-Generated Attention Beyond Softmax and Entmax: Kaniadakis and Reciprocal-Symmetric Abe Operators
Feb 9, 2026Gunn KimEntropy-Regularized Equilibrium Selection Gumbel-Softmax Relaxation
BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
Nov 17, 2025Min Gu Kwak, Yeonju Lee, Hairong Wang +2Brain Tumor Segmentation Anatomical Priors
CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
Oct 29, 2025Xuchen Feng, Siyu LiaoNormalizing Flows Equivariant Neural Networks
NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
May 23, 2025Donghyun Son, Euntae Choi, Sungjoo YooKey-Value Cache Quantization Calibration-Free Low-Bit Vector Quantization
Numerical stability analysis of large language models
Mar 13, 2025Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng +1Full-Precision Performance Large Models
A Mechanistic Study of Transformers Training Dynamics
Oct 31, 2024Ambroise Odonnat, Wassim Bouaziz, Vivien CabannesTransformer Architectures Model Pretraining
Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis
Date pendingHiwa AsadpourMultilingual Automatic Speech Recognition Latin
Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best
Date pendingKevin Baum, Rūta Binkytė, Felix JahnTraining-Free Reward Alignment Compliance
The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
Date pendingAaryaman Kalani, Murari Mandal, Dhruv Kumar +2Machine Unlearning Batch Normalization

May 18, 2026 · cs.LGJ/K move · Enter open · S save

Attention Sinks and Outliers in Attention Residuals

Haozheng Luo, Haoran Dai, Shaoyang Zhang, Xi Chen+9

Department of Computer Science, Northwestern University · Department of Computer Science, Illinois Institute of Technolog · Department of Computer Science and Engineering, University of Michigan · Department of Statistics and Data Science, University of California Los Angeles · Department of Electrical and Computer Engineering, University of California San Diego

We propose OASIS, an outlier- and sink-aware technique built on inter-layer null signaling. As AttnResidual architectures introduce an additional depth-wise normalization channel, they improve inter-layer routing flexibility but also exacerbate attention sinks, activation outliers, and the resulting degradation in inference stability and quantization robustness. OASIS addresses this issue by introducing a Softmax1-based null space and coupling token-level null evidence to depth routing through an inter-layer null signal, thereby reducing sink-dominated routing and improving structural robustness. Theoretically, we show that the dual-normalization design of AttnResidual intensifies sink formation and quantization brittleness. Experimentally, we compare OASIS against five baselines on three real-world datasets and observe consistent improvements in both attention sink and post-quantization performance. Notably, OASIS achieves an average reduction of 9.26% in maximum infinity norm and 2.60% in average kurtosis across the evaluated settings, while lowering perplexity by 75.85% under W8A8 and improving GSM8K Pass@1 by 12.42% under W4A4.

Batch Normalization Outliers

Open paper Ask about it PDF