Model Weights

Follow this topic Ask about this topic

Recent momentum

-50%

11 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

3 new papers

A weekly snapshot of new work published in Model Weights.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Model Weights.

Period ending 2026-09-07

3 new papers

A weekly snapshot of new work published in Model Weights.

149 papers

Latest in Model Weights

Browse all topics →

ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval
Jun 26, 2026Siqiao Xue, Chunxue XuLarge Vision Language Models Garments
Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization
Jun 25, 2026Ziyuan Tang, Tianshi Xu, Yousef Saad +1Muon Optimizer Muon
Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors
Jun 24, 2026Alexander Hägele, Alejandro Hernández-Cano, Atli Kosson +1Neural Network Optimization Model Weights
Pre-Warm: Initializing Convolutional Filters from First-Batch Patch Dictionaries
Jun 24, 2026Rowan MartnishnConvolutional Neural Networks Convolutional
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?
Jun 22, 2026Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2Adam Long-Tailed Distribution
Muown Implicitly Performs Angular Step-size Decay
Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1Muon Optimizer Muon
Scaling Linear Mode Connectivity and Merging to Billion Parameter Pretrained Transformers
Jun 22, 2026Tianyi Li, Zhiqiang ShenGenerative Pretrained Transformers Transformer Architectures
Evolving Spatial Weights for Cartographic Synthesis
Jun 20, 2026Gesiel R. Lopes, Roberto F. da Silva, Mellina Yamamura +4Geospatial Spatio-Temporal Correlations
On the Expressive Power of Weight Quantization in Large Language Models
Jun 20, 2026Shao-Qun ZhangLarge Language Model Quantization Quantization
Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models
Jun 20, 2026Colin Samplawski, Ramneet Kaur, Manoj Acharya +2Large Language Model Adaptation Low-Rank Adaptation
Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs
Jun 19, 2026Charbel El Feghali, Arkil Patel, Nicholas Meade +3Training Language Models Model Weights
Spotlight: Synergizing Seed Exploration and Spot GPUs for DiT RL Post-Training
Jun 17, 2026Ruiqi Lai, Dakai An, Wei Gao +6Reinforcement Learning Post-Training Multimodal Diffusion Transformers
What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy
Jun 16, 2026Truong Xuan KhanhGrokking Batch Normalization
Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization
Jun 15, 2026Kaiyue Wen, Xingyu Dang, Kaifeng Lyu +2Large Language Model Pretraining Model Weights
Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages
Jun 15, 2026Alvaro Javier Vargas Guerrero, Xinguang Wang, Quang Manh Doan +1Federated Learning Model Weights
Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning
Jun 12, 2026Kaiwen Chen, Shuhai Zhang, Zimo Liu +7Spectral Preconditioning Batch Normalization
Weibull Weight-Scale Parameter Evolution under AdamW Training Dynamics
Jun 11, 2026Tiexin DingAdam Model Weights
PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
Jun 4, 2026Senmiao Wang, Tiantian Fang, Haoran Zhang +4Large Language Model Pretraining Spectral Preconditioning
Hyperparameter Learning for Latent Factorization of Tensors for Representation Learning to Large-scale Dynamic Weighted Directed Network
Jun 3, 2026Yaqian Zhan, Jialan He, Tianzhu ChenHyperparameter Optimization Model Weights
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Large Language Model Quantization LLM Inference Efficiency
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarMuon Optimizer Muon
Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference
Jun 1, 2026Chi-Wei Huang, Chia-Chi TsaiWeight-Only Quantization Quantization
Eigenvectors of Experts are Training-free Non-collapsing Routers
May 29, 2026Giang Do, Hung Le, Truyen TranMixture-of-Experts Models Experts
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarLarge Language Model Compression Cross-Layer Interactions
DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
May 28, 2026Hyuck Lee, Taemin Park, Heeyoung KimExtreme Class Imbalance Mixture-of-Experts Models
Learning Compositional Latent Structure with Vector Networks
May 27, 2026Niclas Pokel, Benjamin F. GreweModel Weights Mnist
Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models
May 26, 2026Mingze Wang, Shuchen Zhu, Yuxin Fang +3Batch Normalization Large Language Model Pretraining
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekWeight-Only Quantization Model Weights
Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing
May 25, 2026Michał Brzozowski, Zuzanna Dubanowska, Enrico Cassano +1Contrastive Decoding Finetuning
Analogies between Transformer Layers and Power Method
May 25, 2026Chenglong Li, Claudio AltafiniTransformer Layers Transformer Architectures
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLarge Language Model Quantization LLM Inference Efficiency
Random Neural Network Expressivity for Non-Linear Partial Differential Equations
May 24, 2026Muhammed Ali Mehmood, Lukas GononPartial Differential Equations Neural-Network Approximation
Motion-Compensated Weight Compression
May 23, 2026Ismail LamaakalModel Weights Codec
PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Large Language Model Reasoning Low-Rank Adaptation
A lift for input-convex neural net training
May 22, 2026Ali SiahkoohiLoss Landscape Neural Network Optimization
LLM Sparsity Prior for Robust Feature Selection
May 21, 2026Caleb Skinner, Yihan Guo, Meng LiLasso Feature Selection
Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
May 21, 2026Simon Dennis, Rivaan Patil, Kevin Shabahang +1Agentic Workflows Orchestration
Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws
May 20, 2026Nandan Kumar Jha, Brandon ReagenOptimizer Design Scaling Laws
A New Framework to Analyse the Distributional Robustness of Deep Neural Networks
May 20, 2026Divij Khaitan, Subhashis BanerjeeDistribution Shifts Model Weights
Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
May 19, 2026Lucky VermaModel Weights Transformer Attention
MAM-CLIP: Vision-Language Pretraining on Mammography Atlases for BI-RADS Classification
May 19, 2026Halil Ibrahim Gulluk, Olivier GevaertMammography Medical Vision-Language Models
Position: Weight Space Should Be a First-Class Generative AI Modality
May 18, 2026Zhangyang Wang, Peihao Wang, Kai WangModel Weights Generative Models
A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
May 18, 2026Lei DongBatch Normalization Rectified Linear Unit
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
May 18, 2026Tim Tsz-Kit Lau, Weijie SuOptimizer Design Equivariance
AMO: Adaptive Muon Orthogonalization
May 18, 2026Xinlin Zhuang, Panyi Ouyang, Yichen Li +7Muon Optimizer Adam
Bug or Feature $^2$ : Weight Drift, Activation Sparsity and Spikes
May 17, 2026Egor Shvetsov, Aleksandr Serkov, Shokorov Viacheslav +3Model Weights Neural Architectures
A Two-Parameter Weibull Framework for Diagnosing Transformer Weight Distributions
May 17, 2026Tiexin DingModel Weights Transformer Architectures
A Unified Non-Parametric and Interpretable Point Cloud Analysis via t-FCW Graph Representation
May 14, 2026Haijian Lai, Bowen Liu, Man Xu +4Graph Representation Learning Graph Representations
GQA-μP: The maximal parameterization update for grouped query attention
May 14, 2026Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7Model Weights Large Models
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianLarge Models Model Weights
Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization
May 12, 2026Zhehang Du, Hangfeng He, Weijie SuSymmetry Model Weights
CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks
May 12, 2026Mushir Akhtar, M. Tanveer, Mohd. ArshadModel Weights Gaussian Copula
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
May 11, 2026Ryan Lucas, Mehdi Makni, Xiang Meng +2Quantization Model Weights
Neural Weight Norm = Kolmogorov Complexity
May 11, 2026Tiberiu MusatModel Weights
Muown: Row-Norm Control for Muon Optimization
May 11, 2026Kai Lion, Florian Hübler, Bingcong Li +2Muon Optimizer Muon
Locking Pretrained Weights via Deep Low-Rank Residual Distillation
May 11, 2026Keitaro Sakamoto, Pierre Ablin, Federico Danieli +1Model-Agnostic Defense Model Weights
On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
May 11, 2026Romain Petit, Clarice Poon, Gabriel PeyréTwo-Layer Neural Networks Gradient Descent
Data-driven transport modelling without overfit
May 9, 2026Peter Vanya, Katarína Šimková, Rastislav FarkašData-Driven Overfitting
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
May 9, 2026Beshr IslamBouli, David JinLarge Language Model Quantization Weight-Only Quantization
Robust Server Defense Against Unreliable Clients in One-Shot Fair Collaborative Machine Learning
May 9, 2026Chia-Yuan Wu, Frank E. Curtis, Daniel P. RobinsonFairness-Aware Learning Unreliable Reasoning

Jun 26, 2026 · cs.CVJ/K move · Enter open · S save

ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval

Siqiao Xue, Chunxue Xu

Adapting a foundation vision-language encoder to a specialized retrieval task creates a fundamental tradeoff: gains on the target distribution come at the cost of the foundation model's broad generalization, and fashion retrieval is a stringent instance of this problem. We present ZooClaw-FashionSigLIP2, a fashion-specialized SigLIP2-base model that resolves this tradeoff with a simple recipe -- full fine-tuning with knowledge distillation on curated in-domain data, followed by \wiseft~\citep{wortsman2022wiseft} weight interpolation with the base model -- and outperforms LoRA, larger backbones (up to 1B parameters), and external training data. Under fair evaluation, ZooClaw-FashionSigLIP2 outperforms all baselines on every benchmark in our suite. In addition, we release ZooClaw-Fashion, a new high-quality fashion retrieval benchmark, and a systematic quality analysis of widely-used benchmarks that exposes and mitigates structural biases in their public ground truth. We open-source the model weights and all evaluation artifacts to facilitate future research.

Large Vision Language Models Garments Finetuning Model Weights Claws

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

3 new papers

3 new papers

3 new papers

Latest in Model Weights