Layer-Wise

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 104

All topics

The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP
May 26, 2026Kahyeon Nam, Hyesong ChoiFrozen Contrastive Language-Image Pre-Training Visual Encoder Quantized
Soongsil University Seoul, Republic of Korea
Uncovering the Latent Potential of Deep Intermediate Representations
May 21, 2026Arnesh Batra, Arush Gumber, Aniket Khandelwal +2Layer-Wise Cross-Modal
SBILab, Indraprastha Institute of Information Technology Delhi, Delhi, India.
Relational Linear Properties in Language Models: An Empirical Investigation
May 21, 2026Giovanni Valer, Luigi Gresele, Marco Bronzini +1Language Modeling Layer-Wise
Fondazione Bruno Kessler, Trento, Italy · University of Bologna, Bologna, Italy · University of Pisa, Pisa, Italy +2
Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic
May 18, 2026Thomas Sommariva, Francesca Morandi, Simone Calderara +1Model Fine-Tuning Continual Fine-Tuning
ImageLab, Department of Engineering “Enzo Ferrari”, University of Modena and Reggio Emilia, Italy · University of Pisa, Italy
Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm
May 14, 2026Yuxin Guo, Yihao Yue, Yunhao Ni +4Batch Normalization Neural Network Inference
State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China
Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology
May 14, 2026Jesseba Fernando, Grigori GuitchountsTransformer Residual Streams Residual Stream
Network Science Institute, Northeastern University · Flagship Pioneering
Delta Attention Residuals
May 13, 2026Cheng Luo, Zefan Cai, Junjie HuLayer-Wise Cross-Layer Interactions
Independent Researcher · University of Wisconsin–Madison
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
May 12, 2026Jingzhou Jiang, Yi Yang, Kar Yan TamLayer-Wise Model Selection
The Hong Kong University of Science and Technology
Freeze Deep, Train Shallow: Interpretable Layer Allocation for Continued Pre-Training
May 12, 2026Yu-Hang Wu, Qin-Yuan Liu, Qiu-Yang Zhao +3Pretraining Layer-Wise
Nanhu Research Institute of China Electronic Science and Technology · School of Electronic and Electrical Engineering, Shanghai University of Engineering Science
Don't Look at the Numbers: Visual Anchoring Bias and Layer-wise Representation in VLMs
May 11, 2026M. ShalankinLayer-Wise Representation Quality
The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
May 11, 2026Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1Improving Sparse Autoencoders Scaling Laws
ARC Training Centre for Information Resilience (CIRES) · School of Mathematics and Physics, University of Queensland · Institute for Molecular Bioscience, University of Queensland +2
Sparse Layers are Critical to Scaling Looped Language Models
May 9, 2026Ryan Lee, Jacob Biloki, Edward J. Hu +1Transformer Architectures Layer-Wise
1USC Information Sciences Institute · 2Netflix · 3Independent Researcher
A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
May 9, 2026Gianfranco Lombardo, Giuseppe Trimigno, Stefano CagnoniNext-Token Prediction Layer-Wise
Department of Engineering and Architecture University of Parma Parma, IT 43125
Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
May 8, 2026Zixuan Xie, Xinyu Liu, Claire Chen +3In-Context Learning Linear Attention
University of Virginia · California Institute of Technology · Purdue University
Amortized-Precision Quantization for Early-Exit Vision Transformers
May 8, 2026Rui Fang, Hsi-Wen Chen, Ming-Syan ChenSelf-Supervised Vision Transformers Vision Transformer
National Taiwan University
Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
May 8, 2026Boyu Shi, Chang Liu, ChuanBao Gao +2Layer-Wise Large Language Models Fail
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap
May 8, 2026Feilong LiuMixture-Of-Experts Layer-Wise
UniPool: Learning Expert-to-Layer Ownership from Brief Global Access
May 7, 2026Minbin Huang, Han Shi, Chuanyang Zheng +5Mixture-Of-Experts Experts
The Chinese University of Hong Kong · Huawei Technologies · The University of Hong Kong
Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models
May 7, 2026Amir Rezaei Balef, Mykhailo Koshil, Katharina EggenspergerTabular Foundation Models Transformer Architectures
TU Dortmund University, Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Dortmund, Germany · University of Tübingen, Tübingen, Germany
Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
May 6, 2026Hengyu Shi, Tianyang Han, Peizhe Wang +3Post-Training Layer-Wise
1Independent Researcher · 2D4 Lab · 3Southeast University
Trust, but Verify: Peeling Low-Bit Transformer Networks for Training Monitoring
May 4, 2026Arian Eamaz, Farhang Yeganegi, Mojtaba SoltanalianLayer-Wise Transformer Architectures
Department of Electrical and Computer Engineering University of Illinois Chicago Chicago, IL, USA
Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Apr 27, 2026Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3Language Model Perplexity Layer-Wise
Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States +2
Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models
Apr 27, 2026Nay Myat Min, Long H. Pham, Jun SunFingerprint Jailbreak Success Rates
Singapore Management University
AGNT2: Autonomous Agent Economies on Interaction-Optimized Layer 2 Infrastructure
Apr 22, 2026Anbang Ruan, Xing ZhangBlockchain Infrastructure
NetX Foundation
LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures
Apr 22, 2026Yuhang Wu, Qinyuan Liu, Qiuyang Zhao +1Layer-Wise Frozen Language Model
China Electronic Technology Nanhu Research Institute, Jiaxing, China
Formalising the Logit Shift Induced by LoRA: A Technical Note
Apr 22, 2026Xiang Shi, Shuaizhi Cheng, Mingwei LiLogit Lens Layer-Wise
Imperial College London · Harbin Institute of Technology · KigLand Learning Lab
SimDiff: Depth Pruning via Similarity and Difference
Apr 21, 2026Yuli Chen, Shuhao Zhang, Fanshen Meng +4Unstructured Pruning Large Language Model Compression
ZC-Swish: Stabilizing Deep BN-Free Networks for Edge and Micro-Batch Applications
Apr 21, 2026Suvinava BasakBatch Normalization Deep Network
TU Braunschweig, Universitätspl. 2, 38106 Braunschweig, Germany
RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models
Apr 21, 2026Yusuf Çelebi, Yağız Asker, Özay Ezerceli +4Large Language Model Fine-Tuning Parameter-Efficient Adaptation
Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control
Apr 21, 2026Julian Skifstad, Xinyue Annie Yang, Glen ChouLinear Activation Steering Model Activations
1Georgia Institute of Technology, Atlanta, GA, USA. Schools of Electrical and Computer Engineering · 2Computer Science · 3Cybersecurity and Privacy, Aerospace Engineering
Probing for Reading Times
Apr 20, 2026Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re +4Readability Eye Tracking
ETH Zürich · Toyota Technological Institute at Chicago · University College London
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Apr 20, 2026Yunjia Xi, Menghui Zhu, Jianghao Lin +4Multimodal Representations Layer-Wise
Shanghai Jiao Tong University Shanghai, China · Huawei Noah’s Ark Lab Shanghai, China · Antai College of Economics and Management, Shanghai Jiao Tong University Shanghai, China
Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
Apr 20, 2026Charles Ye, Bo Yuan, Lee SharkeyMixture-Of-Experts Layer-Wise
Independent · Georgia Institute of Technology · Goodfire
Decomposing the Depth Profile of Fine-Tuning
Apr 19, 2026Jayadev BillaLayer-Wise Model Fine-Tuning
Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
Apr 17, 2026Yanli Wang, Peng Kuang, Xiaoyu Han +2Online Conformal Prediction Layer-Wise
Imperial College London · Zhejiang University · University of Illinois Urbana-Champaign +1
DLink: Distilling Layer-wise and Dominant Knowledge from EEG Foundation Models
Apr 16, 2026Jingyuan Wang, Zhihao Jia, Chenyu Liu +7Electroencephalography Foundation Models Small-Sample Electroencephalogram Datasets
1Xiamen University · 2Nanyang Technological University · Institute of Automation, Chinese Academy of Sciences +1
Intermediate Layers Encode Optimal Biological Representations in Single-Cell Foundation Models
Apr 16, 2026Vincenzo Yuto Civale, Roberto Semeraro, Andrew David Bagdanov +1Cells Layer-Wise
University of Siena · University of di Firenze · University of Firenze
When Does Sparsity Mitigate the Curse of Depth in LLMs
Mar 16, 2026Dilxat Muhtar, Xinyuan Song, Sebastian Pokutta +4Sparsity Layer-Wise
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +4
Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
Mar 1, 2026Theophilus Amaefuna, Hitesh Vaidya, Anshuman Chhabra +1Layer-Wise Less Data
1Bellini College of Artificial Intelligence, Cybersecurity and Computing University of South Florida, Tampa, Florida, USA
LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation
Feb 4, 2026Ruixiao Yang, Yuanhe Tian, Di Dong +3Radiology Report Generation Medical Vision-Language Models
Beijing Institute of Technology · Zhongguancun Academy · Zhongguancun Institute of Artificial Intelligence +1
Attentive multilayer fusion for vision transformers
Jan 14, 2026Laure Ciernik, Marco Morik, Lukas Thede +4Self-Supervised Vision Transformers Vision Transformer
Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Hector Fellow Academy, Karlsruhe, Germany · Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany +7
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Large Language Model Quantization Multimodal Large Language Models
University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1
On the Effect of Uncertainty on Layer-wise Inference Dynamics
Jul 9, 2025Sunwoo Kim, Haneul Yoo, Alice OhLarge Language Model Uncertainty Uncertainty
School of Computing, KAIST, Daejeon, South Korea.
Tuning Language Models by Mixture-of-Depths Ensemble
Oct 16, 2024Haoyan Luo, Lucia SpeciaLarge Language Model Fine-Tuning Layer-Wise
University of Cambridge · Imperial College London

May 26, 2026, cs.CVJ/K move · Enter open · S save

The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP

Kahyeon Nam, Hyesong Choi

Soongsil University Seoul, Republic of Korea

Frozen Contrastive Language-Image Pre-Training Visual Encoder Quantized Layer-Wise Imagenet Signal-To-Noise Ratio Impact Analysis

Open paper AskPDF arXiv

Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as CLIP this introduces a failure mode distinct from quantized CNN classifiers: activation noise accumulated across transformer blocks perturbs the direction of the multimodal embedding, eroding the cosine alignment on which zero-shot retrieval depends. We characterize this as Quantization-Induced Representation Collapse (QIRC) and quantify it on INT8 CLIP ViT-B/32, where the layer-wise noise-to-signal ratio grows from below 10% in shallow blocks to 52% at Layer 11. We propose LRA-EE (Layer-wise Representation-Aware Early Exit), which bypasses noise-saturated deep layers via Spatio-Semantic Aggregation (replacing the immature shallow [CLS] with a global patch-token average), a learned multi-feature gate (confidence, top-2 margin, spatial-activation variance), and Layer-adaptive Confidence Thresholding calibrated to each layer's Information-to-Noise Ratio. On ImageNet-1K zero-shot classification, LRA-EE reduces FLOPs by 13.4% and improves Top-1 accuracy by +2.44%p (58.72% -> 61.16%) over the INT8 baseline. A four-quadrant decomposition isolates the Rescue Effect: 9.5% of samples are correctly classified at shallow exits but lost to noise at full depth, against only 7.1% suffering the inverse.

Momentum

Related topics

Latest papers 104