Batch Normalization

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 81

All topics

Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Rates
Sep 30, 2026Elizaveta Iashchinskaia, Egor GladinTime-To-Event Data Logistic Regression
HSE University
Aperture: Merge-Consistent Rotary States for Compressed Tokens
Sep 29, 2026Yuhao Du, Shunian ChenToken Compression Rotary Position Embedding
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute
ZonoGPT: Towards An Abstract Domain for Verifying Large GPT Models
Sep 28, 2026Hai Duong, Thanh Le, ThanhVu NguyenGenerative Pretrained Transformers Transformer Architectures
Department of Computer Science George Mason University Fairfax, VA, USA · Unaffiliated Yokosuka, Japan
PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty
Sep 28, 2026Hongxu Ma, Guang Li, Shijie Wang +6Diffusion-Based Dataset Distillation Dataset Distillation
Zhejiang University · Hokkaido University · The University of Queensland +3
Rethinking Length-Based Training: Batch Composition and Loss Normalization in Speech Token Language Models
Sep 22, 2026Hongjin Song, Runwu Shi, Weiqiao Shan +4Speech Language Models Batch Normalization
Beijing Institute of Technology, Zhuhai · Institute of Science Tokyo · Northeastern University +3
FreqCondNorm: Towards Cross-domain Predictive Maintenance through a Frequency-Conditioned Transformer Foundation Model
Sep 17, 2026Zaynab Raounak, Camille LHermine, Zhiguo ZengPredictive Maintenance Hybrid Transformer
Laboratoire Génie Industriel, CentraleSupélec, Université Paris-Saclay, France
Published Unlearning Numbers Move Per Checkpoint, and Not Because the Removed Data Survives: An Audit of 263 Released Batch-Normalized Checkpoints
Sep 12, 2026Junlong Shen Xingyu LiModel Checkpoints Unlearning Method
University of Alberta Edmonton, Canada
Tracing distinguishability through transformer processing with stochastic LayerNorm
Aug 31, 2026Kieran MurphyTransformer Architectures Batch Normalization
New Jersey Institute of Technology
Rethinking Normalization Placement for LLMs: Post-Norm under Curriculum Depth Growing
Aug 13, 2026Sheng Ren, Yadong Wang, Naiqiang Tan +7Batch Normalization Normalization
Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Didichuxing Co. Ltd, Beijing, China
Why Post-Norm Transformers Collapse: Attention Amplification and Gradient Repair Failure
Aug 10, 2026Xingjian Wang, Qingyu Han, Xiaodong Luo +1Batch Normalization Transformer Encoder
The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data
One Model to Magnify Them All: Efficient Scale-Invariant Histopathology via Conditional Normalization and Continuous Magnification Training
Aug 10, 2026Agnieszka Florkowska, Henning Müller, Marek WodzinskiDigital Pathology Batch Normalization
Sano Centre for Computational Medicine, Krakow, Poland · AGH University of Krakow, al. Adama Mickiewicza 30, 30-059 Kraków, Poland · Institute of Informatics, HES-SO Valais-Wallis, Sierre, Switzerland +1
LASA: Language-and-Source-Anchored Alignment for Domain Generalized Semantic Segmentation
Aug 9, 2026Jinhong Zhu, Weiqi Yan, Shengchuan Zhang +1Multimodal Domain Generalization Efficient Semantic Segmentation
Xiamen University · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China · Xiamen, Fujian, China
Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model
Aug 4, 2026Abdallah KhemaisCross-Layer Interactions Ablation
ISITCOM, University of Sousse
Geometry-Guided Layerwise FFN Width Allocation in Transformers
Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateBatch Normalization Transformer Architectures
University of Luxembourg · London Institute of Mathematical Sciences
Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates
Jul 24, 2026Zhang Gongyue, Sheng Yixuan, Liu donghan +3Adam Affine
arXiv:2607.22927v1
Searching the Space of Feed-Forward Neural-Network Weight-Update Rules with Fixed Depth Symbolic Regression
Jul 23, 2026Charles Brum, Edward FinkelsteinSymbolic Regression Neural Network
San Diego, California, USA · UCI, USA
Scalable Keyword Spotting via Modular Network Expansion
Jul 22, 2026Viktor Khaymonenko, Dzmitry Saladukha, Aliaksei Rak +1Keyword Spotting Parameter-Efficient Adaptation
Embedded Voice Input Team, Yandex, Russia · Embedded Voice Input Team, Yandex, Belarus
Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation
Jul 20, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungBatch Normalization Mean Squared Error
College of Pharmacy, Chungnam National University, Daejeon, Republic of Korea · Department of Computer Science & Engineering, Chungnam National University, Daejeon, Republic of Korea
In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention
Jul 17, 2026Katsuyuki HagiwaraIn-Context Learning Transformer Architectures
Faculty of Education, Mie University · Kurima-Machiya-cho, Tsu, 514-8507, Japan
C-Norm: Cell-Distribution Normalization Enables Precision Recognition of Medical-Cell Image
Jul 14, 2026Yang Qianl, Liu Xiany, Dai Daw +5Cancer Detection Artificial Intelligence Detection
Chongqing University Cancer Hospital, Chongqing, China · Chongqing University of Posts and Telecommunications, Chongqing, China · Shanghai First Maternity and Infant Hospital, Tongji University, Shanghai, China
Lesion Segmentation in Moderate to Severe Traumatic Brain Injury: An nnU-Net Based Approach with Adaptive Normalization in the AIMS-TBI 2025 Challenge
Jul 14, 2026Inhwa Son, Gaeun Lee, Sohyeon Sim +1Brain Tumor Segmentation Lesion Segmentation
Gachon University, Republic of Korea · MEDAI, Republic of Korea
What Does Goodness Measure? A Likelihood-Ratio Account of Forward-Forward Learning
Jul 14, 2026Paolo GiannitrapaniForward Propagation Batch Normalization
Department of Information Engineering, Electronics and Telecommunications (DIET) Sapienza University of Rome Rome, Italy
Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps
Jul 14, 2026Subham Singh, Ashutosh Mishra, Subha RautBatch Batch Normalization
Department of Mathematics and Statistics, Mississippi State University · Independent Researcher
LayerNorm as Implicit Gain Control in Looped Transformers
Jul 12, 2026Matthias M. M. BuehlmaierBatch Normalization Recurrent Model
Faculty of Business and Economics, The University of Hong Kong · Pokfulam Road, Hong Kong
AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating
Jul 12, 2026Piyush Kaushik Bhattacharyya, Divyanshu Rai, Swastik Singh +3Batch Normalization Transformer Architectures
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
Jul 9, 2026Sarthak Kumar Maharana, Shambhavi Mishra, Yunbei Zhang +6Test-Time Adaptation Mitigating Domain Shift
The University of Texas at Dallas, USA · LIVIA ETS Montreal, ILLS International Laboratory on Learning Systems (ILLS), McGill - ÉTS - MILA - CNRS - Université Paris-Saclay - CentraleSupélec · Tulane University, USA +2
Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization
Jul 8, 2026Shuo Huai, Di Liu, Hao Kong +4Neural Network Inference Nvidia Jetson Orin Nano
aSchool of Computer Science and Engineering, Nanyang Technological University, 639798, Singapore · cDepartment of Computer Science, Norwegian University of Science and Technology, 7491, Trondheim, Norway · bHP-NTU Digital Manufacturing Corporate Lab, Nanyang Technological University, 639798, Singapore +1
HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference
Jul 5, 2026Hui Dong, Yanzhao Li, Jie Gao +5Block Sparse Flash Attention LLM Inference Optimization
Huawei Technologies
Group-Equivariant Poincaré Convolutional Networks
Jul 1, 2026Aiden Durrant, Rahul Baburajan, Georgios LeontidisRotation Equivariance Batch Normalization
School of Computing Sciences, University of East Anglia, NR4 7TJ, Norwich, UK · Department of Physics and Technology, UiT The Arctic University of Norway, NO-9037, Tromsø, Norway
Signed-Permutation Coordinate Transport for RMSNorm Transformers
Jun 30, 2026John SweeneyBatch Normalization Transformer Architectures
Sideplane AI
Review Residuals: Update-Conditioned Residual Gating for Transformers
Jun 30, 2026Kyle KramerResidual Connections Batch Normalization
NeraTech LLC
Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization
Jun 30, 2026Ruichen Ma, Xiaoyang Zhang, Jian Bai +5Spiking Neural Networks Neuromorphic Computing
University of Electronic Science and Technology of China (UESTC) · Beijing Institute of Remote-Sensing Equipment · Shenzhen Institute for Advanced Study, UESTC
Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks
Jun 28, 2026Tejas Pradeep ShirodkarAdam Deep Network
IIIT, Hyderabad
TA-SparseMG: Trend-Aware Sparse Forecasting via Multi-Scale Gating for Long-Term Time Series
Jun 26, 2026Wenchao Liu, Hongbing Wang, Youji Zhu +2Long-Term Batch Normalization
School of Mathematical Sciences, Guizhou Normal University, Guiyang 550025, China. · School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China.
Aurora: A Leverage-Aware Spectral Optimizer
Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Muon Batch Normalization
Tilde Research
TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation
Jun 23, 2026Sachin Sharma, Michele Flammini, Federico SimonettaHandwritten Text Recognition Historical Manuscripts
GSSI – Gran Sasso Science Institute L’Aquila, Italy
Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale
Jun 17, 2026Tejas Pradeep Shirodkar, P. J. NarayananBatch Normalization Transformer Architectures
IIIT, Hyderabad
Smoothness-Based Derandomization of PAC-Bayes Bounds
Jun 17, 2026Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe GiguèreGeneralization Bounds Batch Normalization
Department of Computer Science and Software Engineering Université Laval, Québec, Canada
What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy
Jun 16, 2026Truong Xuan KhanhGrokking Weight Decay
H&K Research Studio, Clevix LLC · Hanoi, Vietnam
MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration
Jun 16, 2026Kosmas Alexandridis, Giorgos DimitrakopoulosMatrix Multiplication Batch Normalization
Integrated Circuits Lab, Electrical and Computer Engineering, Democritus University of Thrace (DUTH), Greece
QK-Normed MLA: QK normalization without full key caching
Jun 15, 2026Yizhou Han, Yao Zhao, Jun Zhou +2Multi-Head Attention Batch Normalization
School of Data Science, The Chinese University of Hong Kong, Shenzhen.
Z-Plane Neural Networks: Bounded Geometric Activation Replaces ReLU and LayerNorm
Jun 14, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungActivation Functions Batch Normalization
College of Pharmacy, Chungnam National University, Daejeon, Republic of Korea · Department of Computer Science & Engineering, Chungnam National University, Daejeon, Republic of Korea
Beyond Backbone Backpropagation: A Decoupled Strategy for Efficient Transfer Learning
Jun 13, 2026Daniel Vila-Cruz, Laura Morán-Fernández, Verónica Bolón-CanedoTransfer Learning Convolutional Neural Networks
CITIC, Universidade da Coruña, A Coruña, Spain
LieBN: Batch Normalization over Lie Groups
Jun 13, 2026Ziheng Chen, Yue Song, Rui Wang +2Batch Normalization Lie Group
Department of Information Engineering and Computer Science, University of Trento, Trento, Italy · Computing and Mathematical Sciences, Caltech, CA, USA · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China
Squeeze-Release: Iterative Pruning with Exact Structural Minimization
Jun 12, 2026Roman Denkin, Ida Akerholm, Prashant Singh +1Unstructured Pruning Batch Normalization
Department of Information Technology, Uppsala University, Uppsala, Sweden · Science for Life Laboratory, Uppsala University, Uppsala, Sweden
Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning
Jun 12, 2026Kaiwen Chen, Shuhai Zhang, Zimo Liu +7Whitening Spectral Preconditioning
South China University of Technology · AIGCode · Hong Kong Baptist University
PostDeg: Placement Beats Parameterization in LayerNorm GNNs
Jun 12, 2026Yash Tomar, Aryav DasBatch Normalization Graph Neural Networks
Purdue University · Park Tudor High School
MechLens: Late Crystallization of Factual Knowledge Explains Intervention Effectiveness in Language Models
Jun 6, 2026Xueping GaoFactual Recall Hallucination Mitigation
Alibaba Cloud
SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks
Jun 1, 2026Mourad ZaiedBatch Normalization Deep Network
University of Gabes, Tuisia · Department of electrical engineering National Engineering School of Gabes (ENIG) University of Gabes Avenue Omar Ibn El Khattab, Zrig Eddakhlania 6029, Gabes, Tunisia
Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition
Jun 1, 2026Shuo Zhang, Chenqi Li, Tingting ZhuBatch Normalization General Grid Search Framework
University of Oxford
FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting
May 31, 2026Mirza Samad Ahmed Baig, Syeda Anshrah GillaniNon-Stationarity Batch Normalization
Fandaqah, Al Khobar, Saudi Arabia · Hamdard University, Karachi, Pakistan
Looped Transformers with Layer Normalization Provably Learn the Power Method
May 30, 2026Lyumin Wu, Chenyang Zhang, Yuan CaoTransformer Architectures Batch Normalization
School of Computing & Data Science, The University of Hong Kong
Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
May 26, 2026Zitao Song, Cedar Site Bai, Zhe Zhang +2Gradient Clipping Stochastic Gradient Descent
Department of Computer Science, Purdue University, West Lafayette, IN, USA. · School of Industrial Engineering, Purdue University, West Lafayette, IN, USA.
SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
May 26, 2026Mohammed Adnan, Rohan Jain, Tom Jacobs +4Batch Normalization Neural Network Optimization
University of Calgary · 2Vector Institute · 3CISPA Helmholtz +2
Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models
May 26, 2026Mingze Wang, Shuchen Zhu, Yuxin Fang +3Batch Normalization Large Language Model Pretraining
ByteDance Seed · Peking University
Llamion Technical Report
May 25, 2026Kisu Yang, Yoonna Jang, Hyeonseok Moon +6Teacher-Student Distillation Batch Normalization
1VAIV Company · University of Copenhagen · 4Samsung Electronics +1
Stabilizing Streaming Video Geometry via Dynamic Feature Normalization
May 25, 2026Xiaoyang Lyu, Muxin Liu, Xiaoshan Wu +5Robust Geometric Model Estimation Optical Flow
1The University of Hong Kong · 2USTC · 3Voyager Research, Didi Chuxing
Momentum Streams for Optimizer-Inspired Transformers
May 23, 2026Jingchu Gai, Nai-Chieh Huang, Jiayun WuTransformer Architectures Batch Normalization
Carnegie Mellon University
Batch Normalization Amplifies Memorization and Privacy Risks
May 23, 2026Ngoc Phu Doan, Chongyan Gu, Ihsen AlouaniBatch Normalization Privacy
Queen’s University Belfast
Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label
May 20, 2026Jingyang Mao, Ningkang Peng, Yanhui GuNoisy Labels High Confidence Pseudo-Labels
Nanjing Normal University Nanjing, China

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Rates

Elizaveta Iashchinskaia, Egor Gladin

HSE University

Time-To-Event Data Logistic Regression Exponential Family Target Risk Batch Normalization

Open paper AskPDF arXiv

Motivated by the computational challenges of large-scale Cox regression, we study stochastic minimization of LogSumExp objectives over large sets. Mini-batch normalizer estimates generally yield biased gradients. We instead use a softplus surrogate that introduces one auxiliary scalar per normalizer and admits unbiased single-sample gradients. For smooth convex LogSumExp objectives, we prove an

O(T^{-1/2})

averaged objective bound, improving the previous

T^{-1/4}

analysis. With a strongly convex regularizer on the original variable, we also obtain a last-iterate squared-error rate of

\widetilde{O}(T^{-1})

without strong convexity in the auxiliary variables. For Cox regression, the normalizers are defined over nested risk sets. We exploit this structure by grouping neighboring failures and sharing one auxiliary variable per group. The resulting compressed objective admits uniform score and curvature bounds that control the errors from grouping and softplus approximation. Together with the general optimization result, these bounds give a mean-square rate of

T^{-4/5}

, up to logarithmic factors, relative to the full Cox solution. The compressed estimator also matches the full estimator's asymptotic distribution. Experiments on synthetic and real survival datasets with slowly decreasing risk sets show a favorable performance relative to stochastic baselines.

Momentum

Related topics

Latest papers 81