Model Compression

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 79

All topics

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression
Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model Compression LLM Compression
Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3
SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models
Jul 2, 2026Qi Lyu, Jiahua Dong, Baichen Liu +7Model Compression Vision-Language Models
State Key Laboratory of Robotics and Intelligent Systems · Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2
Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers
Jul 1, 2026Jaeah Lee, Hyunjin Kim, Jaewoong Cho +1Model Compression Diffusion Transformer
KRAFTON AI, Republic of Korea · Amazon, Australia
Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study
Jun 26, 2026Qiong Tang, Xiangkun Hu, Xiangyang Liu +2Model Compression LLM Compression
Analemma
Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?
Jun 26, 2026Guoheng Sun, Kaixi Feng, Shwai He +8Model Compression Efficient VLA Models
University of Maryland, College Park · 2Cisco Research
TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing
Jun 25, 2026Peizhen Zhang, Yang Li, Xunsong Li +10Model Compression Diffusion-Based Image Generation
Multimodal Model Department, Tencent
What Survives When You Compress a Recursive Reasoner for the Edge?
Jun 25, 2026Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2Model Compression Efficient Neural Network Inference
ML Collective · Carnegie Mellon University
Agentic evolution of physically constrained foundation models
Jun 24, 2026Jiangwei Zhang, Wen Sun, Chong Wang +7Model Compression LLM Quantization
State Key Laboratory of Cyberspace Security Defense, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100084, China. · University of Chinese Academy of Sciences, Beijing, 101408, China. · Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China.
Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
Jun 22, 2026Minh-Loi Nguyen, Long-Bao Nguyen, Van-Hieu Huynh +2Model Compression Efficient Neural Network Inference
University of Science, Ho Chi Minh city, Vietnam · Vietnam National University, Ho Chi Minh city, Vietnam
Scaling Audio Models Efficiently: Joint Optimization of Scale, Resolution, Adaptation, Precision, and Sparsity
Jun 22, 2026Vyom Agarwal, Mokshda Gangrade, Siddharth Pal +1Model Compression Structured Sparsity
Computer, Mathematical, and Natural Sciences, University of Maryland, College Park, MD, USA · RTX BBN Technologies, Columbia, MD, USA · Electrical and Computer Engineering, University of Maryland, College Park, MD, USA
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Jun 19, 2026Siyang Lyu, Xinghao Chen, Zhijing Sun +3Model Compression CoT Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Viterbi School of Engineering, University of Southern California · The Hong Kong Polytechnic University +2
Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
Jun 18, 2026Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha +18Model Compression Efficient VLA Model Inference
Center for AI Research, VinUniversity · VinRobotics · University of Arkansas +10
Complementary Attention Head Pruning for Efficient Transformers
Jun 17, 2026Yaniv Livertovsky, Shahar Somin, Gonen SingerModel Compression Transformer Attention
Bar-Ilan University Ramat Gan, Israel
Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization
Jun 12, 2026Rishit Chatterjee, Tahiya ChowdhuryModel Compression Speaker Diarization
Department of Computer Science, Colby College, Waterville, Maine, United States
Simplex-Constrained Sparse Bagging: Transitioning from Uniform Priors to Sparse Posteriors in Ensemble Learning
Jun 11, 2026Meher Sai Preetam Madiraju, Meher Bhaskar MadirajuModel Compression Probability Calibration
SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices
Jun 5, 2026Ernests Lavrinovics, Marco Letizia, Roy Janco +3Model Compression LLM Compression
Department of Computer Science, Aalborg University Copenhagen, Denmark · MaLGa-DIBRIS, University of Genoa, Genoa, Italy · INFN, Sezione di Genova, Genoa, Italy +2
LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models
Jun 4, 2026Rui Wang, Yan Zhao, Li Song +1Model Compression LLM Quantization
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University. Shanghai, China
Spatially Distributed Task-Oriented Compression for Multi-Emitter Localization and Characterization with Spectral Overlap
May 31, 2026H. Nazim Bicer, J. Nicholas LanemanModel Compression Wireless Communications
Dept. of Electrical Engineering University of Notre Dame Notre Dame, IN, USA
DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models
May 30, 2026Abdullah Al Shafi, Kazi Saeed Alam, Sk Imran Hossain +1Model Compression Classifier-Free Guidance
Khulna University of Engineering & Technology, Bangladesh · University Clermont Auvergne, France
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarModel Compression LLM Compression
Independent Researcher
Smaller and Faster 3DGS via Post-Training Dictionary Learning
May 28, 2026Jiarong Gong, Jonas Unger, Ehsan Miandji3DGS Compression Model Compression
Linköping University Department of Science and technology
PINE: Pruning Boosted Tree Ensembles with Conformal In-Distribution Prediction Equivalence
May 27, 2026Haruki Yajima, Yusuke MatsuiModel Compression Ensemble Learning
The University of Tokyo.
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
May 19, 2026Hyunsoo Han, Sangyeop Yeo, Jaejun YooModel Compression Diffusion Model Distillation
Ulsan National Institute of Science and Technology (UNIST)
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
May 18, 2026Chengjie Hong, Feixiang He, Yiheng Zeng +2Model Compression Low-Rank Compression
University College London, UK · Central South University, China · University of Massachusetts at Amherst, US +1
MedCore: Boundary-Preserving Medical Core Pruning for MedSAM
May 13, 2026Cenwei Zhang, Suncheng Xiang, Lei YouModel Compression Image Segmentation
Shanghai Jiao Tong University Shanghai, China · Technical University of Denmark Ballerup, Denmark
Minimax Rates and Spectral Distillation for Tree Ensembles
May 12, 2026Binh Duc Vu, David S. WatsonModel Compression Gradient-Boosted Decision Trees
King’s College London
Robust Basis Spline Decoupling for the Compression of Transformer Models
May 11, 2026Joppe De Jonghe, Van Tien Pham, Mariya IshtevaModel Compression Tensor Decomposition
aNUMA, Department of Computer Science, KU Leuven, Kleinhoefstraat 4, Geel, 2440, Antwerpen, België · bUniversité Paris-Saclay, CEA, List, Palaiseau, F-91120, France
XTinyU-Net: Training-Free U-Net Scaling via Initialization-Time Sensitivity
May 10, 2026Alvin Kimbowa, Moein Heidari, David Liu +1Model Compression U-Net
School of Biomedical Engineering, The University of British Columbia, Vancouver, Canada · Department of Radiology, The University of British Columbia, Vancouver, Canada · Department of Medicine, The University of British Columbia, Vancouver, Canada
PACE: Prune-And-Compress Ensemble Models
May 7, 2026Fabian Akkerman, Julien Ferry, Théo Guyard +1Model Compression Ensemble Learning
Industrial Engineering and Management Science University of Twente, Netherlands · SCALE-AI Chair and CIRRELT Polytechnique Montréal, Canada
Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces
May 4, 2026Jingze Ge, Yun Liu, Xue Geng +4Model Compression Adapter Tuning
National University of Singapore · Nankai University · Institute for Infocomm Research (I2R), A*STAR

Jul 3, 2026, cs.LGJ/K move · Enter open · S save

LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

Zhuowen Liu, Longkun Hao, Shiyu Feng, Xiaowen Chang+2

Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3+3

Model Compression LLM Compression Low-Rank Matrix Decomposition Low-Rank Compression Low-Rank Approximation Singular Value Decomposition

Open paper AskPDF arXiv

The rapid growth in the parameter scale of large language models (LLMs) has created a strong demand for efficient compression techniques. As a hardware-agnostic and highly compatible approach, low-rank compression has been widely adopted to reduce both memory footprint and computational cost. However, existing SVD-based methods are still largely driven by local reconstruction objectives, overlooking two critical limitations: rank budgets are often allocated without explicitly considering layer-wise loss sensitivity, and local approximation errors can propagate and accumulate through the residual stream, leading to amplified global deviations from the original model. To address these issues, we propose LACE-SVD, a Loss-Aware SVD framework with Cumulative Error correction for LLM compression. LACE-SVD first estimates the calibration negative-log-likelihood increase induced by candidate layer-wise compression ratios and solves a budget-constrained allocation problem to assign rank budgets. It then refines the compressed model with closed-form local updates and introduces a propagation-aware correction for residual-stream output modules, reducing layer-output discrepancy as a proxy for cumulative error propagation. Experimental results demonstrate that at a high compression ratio (0.6), the WikiText-2 PPL of our method on LLaMA-7B (32.57) is significantly better than that of Dobi-SVD (46.18).

Momentum

Related topics

Latest papers 79