Model Weights

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

3 new papers

A weekly snapshot of new work published in Model Weights.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Model Weights.

Inside this field

Focused directions

Focused topic

Model Weights

Within Model Weights

149 papersExplore

Focused topic

Weight

Within Model Weights

50 papersExplore

Focused topic

Weight-Space

Within Model Weights

30 papersExplore

Focused topic

Weighted

Within Model Weights

29 papersExplore

235 papers

Latest in Model Weights

Browse all topics →

Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation
Jun 16, 2026Ido Nitzan Hidekel, Dan RavivCatastrophic Forgetting Neural Tangent Kernel
Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization
Jun 15, 2026Kaiyue Wen, Xingyu Dang, Kaifeng Lyu +2Large Language Model Pretraining Model Weights
Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages
Jun 15, 2026Alvaro Javier Vargas Guerrero, Xinguang Wang, Quang Manh Doan +1Federated Learning Model Weights
Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning
Jun 12, 2026Kaiwen Chen, Shuhai Zhang, Zimo Liu +7Spectral Preconditioning Batch Normalization
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Jun 11, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +3Large Language Model Quantization LLM Inference Efficiency
Weibull Weight-Scale Parameter Evolution under AdamW Training Dynamics
Jun 11, 2026Tiexin DingAdam Model Weights
Recoverable but Not Stationary:Local Linear Structures in Weights and Activations
Jun 9, 2026Irina Piontkovskaia, Sergey NikolenkoLow-Rank Adaptation Low-Rank Factorization
What Spatial Memory Must Store: Occlusion as the Test for Language-Agent Memory
Jun 9, 2026Doeon Kwon, Junho BangSpatial Memory Occlusion
BrainSurgery: Reproducible and Reliable Declarative Weight Manipulations for Model Editing and Upcycling
Jun 8, 2026Gianluca Barmina, Annemette Broch Pirchert, Andrea Blasi Núñez +2Pytorch Gradient Surgery
PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
Jun 4, 2026Senmiao Wang, Tiantian Fang, Haoran Zhang +4Large Language Model Pretraining Spectral Preconditioning
Hyperparameter Learning for Latent Factorization of Tensors for Representation Learning to Large-scale Dynamic Weighted Directed Network
Jun 3, 2026Yaqian Zhan, Jialan He, Tianzhu ChenHyperparameter Optimization Model Weights
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Large Language Model Quantization LLM Inference Efficiency
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarMuon Optimizer Muon
Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference
Jun 1, 2026Chi-Wei Huang, Chia-Chi TsaiWeight-Only Quantization Quantization
Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition
Jun 1, 2026Shuo Zhang, Chenqi Li, Tingting ZhuHyperparameter Optimization Normalization
Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization
May 31, 2026Jia-Qi Yang, Lei ShiNeural Operators Neural-Network Approximation
On the Difficulty of Learning a Meta-network for Training Data Selection
May 30, 2026Zilin Du, Junqi Zhao, Boyang Albert LiMeta-Learning Training Data
Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence
May 30, 2026Wanying Ren, Xin Song, Futing Wang +2Multimodal Knowledge Editing Collapse
Exploiting weight-space symmetries for approximating curvature
May 30, 2026Artem Artemev, Rui Xia, Benjamin M. Boyd +4Curvature Condition Number
Eigenvectors of Experts are Training-free Non-collapsing Routers
May 29, 2026Giang Do, Hung Le, Truyen TranMixture-of-Experts Models Experts
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarLarge Language Model Compression Cross-Layer Interactions
DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
May 28, 2026Hyuck Lee, Taemin Park, Heeyoung KimExtreme Class Imbalance Mixture-of-Experts Models
Learning Compositional Latent Structure with Vector Networks
May 27, 2026Niclas Pokel, Benjamin F. GreweModel Weights Mnist
Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models
May 26, 2026Mingze Wang, Shuchen Zhu, Yuxin Fang +3Batch Normalization Large Language Model Pretraining
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekWeight-Only Quantization Model Weights
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
May 25, 2026Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1Contrastive Decoding Finetuning
Analogies between Transformer Layers and Power Method
May 25, 2026Chenglong Li, Claudio AltafiniTransformer Layers Transformer Architectures
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLarge Language Model Quantization LLM Inference Efficiency
Random Neural Network Expressivity for Non-Linear Partial Differential Equations
May 24, 2026Muhammed Ali Mehmood, Lukas GononPartial Differential Equations Neural-Network Approximation
Motion-Compensated Weight Compression
May 23, 2026Ismail LamaakalModel Weights Codec
PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Large Language Model Reasoning Low-Rank Adaptation
A lift for input-convex neural net training
May 22, 2026Ali SiahkoohiLoss Landscape Neural Network Optimization
What Linear Probes Miss: Multi-View Probing for Weight-Space Learning
May 22, 2026Eunwoo Heo, Kyeongkook Seo, Jaejun YooWeight-Space Multi-View
LLM Sparsity Prior for Robust Feature Selection
May 21, 2026Caleb Skinner, Yihan Guo, Meng LiLasso Feature Selection
Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
May 21, 2026Simon Dennis, Rivaan Patil, Kevin Shabahang +1Agentic Workflows Orchestration
Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws
May 20, 2026Nandan Kumar Jha, Brandon ReagenOptimizer Design Scaling Laws
A New Framework to Analyse the Distributional Robustness of Deep Neural Networks
May 20, 2026Divij Khaitan, Subhashis BanerjeeDistribution Shifts Model Weights
Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
May 19, 2026Lucky VermaModel Weights Transformer Attention
MAM-CLIP: Vision-Language Pretraining on Mammography Atlases for BI-RADS Classification
May 19, 2026Halil Ibrahim Gulluk, Olivier GevaertMammography Medical Vision-Language Models
Position: Weight Space Should Be a First-Class Generative AI Modality
May 18, 2026Zhangyang Wang, Peihao Wang, Kai WangModel Weights Generative Models
A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
May 18, 2026Lei DongBatch Normalization Rectified Linear Unit
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
May 18, 2026Tim Tsz-Kit Lau, Weijie SuOptimizer Design Equivariance
AMO: Adaptive Muon Orthogonalization
May 18, 2026Xinlin Zhuang, Panyi Ouyang, Yichen Li +7Muon Optimizer Adam
Bug or Feature $^2$ : Weight Drift, Activation Sparsity and Spikes
May 17, 2026Egor Shvetsov, Aleksandr Serkov, Shokorov Viacheslav +3Model Weights Neural Architectures
Weighted Reverse Convolution for Feature Upsampling
May 17, 2026Wentong Li, Zhiyuan Qi, Zichen Zhao +2Upsampling Feature Matching
A Two-Parameter Weibull Framework for Diagnosing Transformer Weight Distributions
May 17, 2026Tiexin DingModel Weights Transformer Architectures
A Unified Non-Parametric and Interpretable Point Cloud Analysis via t-FCW Graph Representation
May 14, 2026Haijian Lai, Bowen Liu, Man Xu +4Graph Representation Learning Graph Representations
GQA-μP: The maximal parameterization update for grouped query attention
May 14, 2026Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7Model Weights Large Models
When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
May 14, 2026ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3Mechanistic Interpretability Semantic Similarity Metrics
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianLarge Models Model Weights
Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
May 14, 2026Pengkai Wang, Pengwei Liu, Yuanyi Wang +7Neural Partial Differential Equation Solvers Partial Differential Equations
Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights
May 13, 2026Wenrui Bao, Huan Wang, Jian Wang +3Multi-Agent Large Language Model Weight
Unweighted ranking for value-based decision making with uncertainty
May 13, 2026Aarón López García, Natalia Criado, Jose SuchHuman Values Decisions
When and Why is Optimistic Multiplicative Weights Slow? The Geometry of Energy Dissipation
May 13, 2026John Lazarsfeld, Anas Barakat, Georgios Piliouras +2Convergence Analysis Imperfect-Information Games
Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization
May 12, 2026Zhehang Du, Hangfeng He, Weijie SuSymmetry Model Weights
CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks
May 12, 2026Mushir Akhtar, M. Tanveer, Mohd. ArshadModel Weights Gaussian Copula
FedOUI: OUI-Guided Client Weighting for Federated Aggregation
May 12, 2026Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3Federated Learning Fedavg
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
May 11, 2026Ryan Lucas, Mehdi Makni, Xiang Meng +2Quantization Model Weights
Neural Weight Norm = Kolmogorov Complexity
May 11, 2026Tiberiu MusatModel Weights
Muown: Row-Norm Control for Muon Optimization
May 11, 2026Kai Lion, Florian Hübler, Bingcong Li +2Muon Optimizer Muon

Jun 16, 2026 · cs.LGJ/K move · Enter open · S save

Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation

Ido Nitzan Hidekel, Dan Raviv

Tel Aviv University, Tel Aviv, Israel.

Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions are vulnerable. We give a function-space account in the NTK regime: new-task training induces old-task prediction drift through the cross-task kernel, yielding a closed-form predictor for the forgetting vector before any new-task gradient step. In frozen-backbone linear-head PEFT-CL, where the model is linear in the trainable parameters, the predictor is exact up to numerical precision; for nonlinear adapters/full fine-tuning, it is a local NTK approximation. The same expression reveals that forgetting concentrates in a small number of old-task NTK eigenmodes and under frozen linear heads gives a Kronecker scaling rule for the vulnerable rank. These results clarify the relation to prior NTK-overlap theory, explain why parameter-space regularizers can miss output-space interference, and motivate a targeted spectral regularizer.

Catastrophic Forgetting Neural Tangent Kernel Regularization Techniques Weight-Space

Open paper Ask about it PDF