Low-Rank Compression

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 58

All topics

EinSort: Sorting is All We Need for Tensorizing LLM
Jun 7, 2026Toshiaki Koike-Akino, Jing Liu, Ye WangTensor Networks LLM Compression
Mitsubishi Electric Research Laboratories (MERL), 20 Broadway, Cambridge, MA 02139, USA.
STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
Jun 7, 2026Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3Mixed-Precision Quantization KV Caching
University of California San Diego, San Diego, CA, USA · Dnotitia Inc., Seoul, Republic of Korea · Hanyang University, Seoul, Republic of Korea
SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices
Jun 5, 2026Ernests Lavrinovics, Marco Letizia, Roy Janco +3Model Compression LLM Compression
Department of Computer Science, Aalborg University Copenhagen, Denmark · MaLGa-DIBRIS, University of Genoa, Genoa, Italy · INFN, Sezione di Genova, Genoa, Italy +2
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
Jun 3, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLLM Compression Communication-Efficient Distributed Training
Concordia University · Mila Quebec AI Institute · CNRS, Sorbonne University
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Jun 2, 2026Zhengbao He, Ruiqi Ding, Zhehao Huang +3Adapter Tuning Low-Rank Adaptation
Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
May 31, 2026Shihao Zhang, Rayan SaabLLM Quantization Low-Rank Adaptation
Department of Mathematics, University of California San Diego
LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language Models Efficient VLM Inference
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarModel Compression LLM Compression
Independent Researcher
Efficient Pre-Training of LLMs through Truncated SVD Representations
May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Language Model Pretraining Low-Rank Matrix Decomposition
Cognizant AI Lab
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
May 18, 2026Chengjie Hong, Feixiang He, Yiheng Zeng +2Model Compression Low-Rank Compression
University College London, UK · Central South University, China · University of Massachusetts at Amherst, US +1
IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
May 15, 2026Ali Abbasi, Chayne Thrash, Haoran Qin +2LLM Compression Low-Rank Matrix Decomposition
Vanderbilt University · University of California, Davis
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University
Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression
May 9, 2026Hengyi Zhu, Zhendong Mi, Grace Li Zhang +1LLM Compression Low-Rank Matrix Decomposition
1Stevens Institute of Technology, Hoboken, USA · 2Technical University of Darmstadt, Darmstadt, Germany
FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
May 8, 2026Wenhao Wu, Zishan Shao, Kangning Cui +5Efficient Transformer Inference LLM Compression
Department of Electrical & Computer Engineering, Duke University · Department of Statistical Science, Duke University · Department of Computer Science, Wake Forest University +1
Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization
May 7, 2026Tatsuhiro Nakamori, Laura Gomezjurado Gonzalez, Ganesh Talluri +5Communication-Efficient Distributed Training Distributed Optimization
1Keio University · 2Stanford University · 3Midwestern University +3
Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference
May 5, 2026Mohammed Sabry, Anya BelzLLM Compression Low-Rank Adaptation
ADAPT Centre, Dublin City University, Ireland
ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
May 5, 2026Jiaxi Li, Lu Yin, Li Shen +5Language Model Pretraining Activation Sparsity
University of Surrey · Sun Yat-sen University · Bytedance +2
Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces
May 4, 2026Jingze Ge, Yun Liu, Xue Geng +4Model Compression Adapter Tuning
National University of Singapore · Nankai University · Institute for Infocomm Research (I2R), A*STAR
Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models
May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM Pruning LLM Compression
Iowa State University · Meta
Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithm
May 2, 2026Wen-Da Wei, Han-Bin Fang, Yang-Di Liu +3LLM Compression Gradient Compression
Nanjing University, Nanjing, China · Tsinghua University, Beijing, China · Huazhong University of Science and Technology, Wuhan, China +1
Post-Optimization Adaptive Rank Allocation for LoRA
Apr 30, 2026Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. SrijithLow-Rank Adaptation Low-Rank Compression
1Indian Institute of Technology Hyderabad · 2Deakin University
Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis
Apr 25, 2026Sisipho Hamlomo, Marcellin Atemkeng, Habte Tadesse Likassa +5Low-Rank Compression Low-Rank Approximation
Department of Mathematics, Rhodes University, Makhanda 6140, South Africa · Department of Statistics, Rhodes University, Makhanda 6140, South Africa · National Institute for Theoretical and Computational Sciences, Stellenbosch 7600, South Africa +8
Predicting LLM Compression Degradation from Spectral Statistics
Apr 20, 2026Mingxue XuLLM Compression Low-Rank Matrix Decomposition
Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom
LASER: Low-Rank Activation SVD for Efficient Recursion
Apr 19, 2026Ege Çakar, Ketan Ali Raghu, Lia ZhengLow-Rank Matrix Decomposition Recurrent Neural Networks
Harvard University Cambridge, MA, USA
Attention-Weighted Value Projection for KV-Cache Compression
Apr 13, 2026Samuel SalfatiGrouped-Query Attention KV Caching
fraQtl AI Research
Stabilizing Native Low-Rank LLM Pretraining
Feb 12, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLanguage Model Pretraining Low-Rank Matrix Decomposition
Concordia University, Montreal, Canada · Mila Quebec AI Institute, Montreal, Canada · Sorbonne University, CNRS, Paris, France
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
Oct 7, 2025Ryan Solgi, Parsa Madinei, Jiayi Tian +4Model Compression LLM Compression
University of California-Santa Barbara, USA · Amazon, USA
BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
Sep 29, 2025David González-MartínezLow-Rank Matrix Decomposition Low-Rank Compression
Department of Computer Science and Artificial Intelligence, Universidad de Sevilla · Tübingen AI Center

Jun 7, 2026, cs.LGJ/K move · Enter open · S save

EinSort: Sorting is All We Need for Tensorizing LLM

Toshiaki Koike-Akino, Jing Liu, Ye Wang

Mitsubishi Electric Research Laboratories (MERL), 20 Broadway, Cambridge, MA 02139, USA.

Tensor Networks LLM Compression Low-Rank Compression Low-Rank Approximation

Open paper AskPDF arXiv

Tensor networks provide efficient representations for compressing large neural networks. By carefully designing shapes and topologies, they can significantly reduce memory and computational costs. However, identifying implicit low-rank structures in large foundation models remains challenging due to their enormous scale and un-structured weight distributions. We propose an adaptive tensorization method that discovers inherent low-rank structure in a target tensor by index ordering. Experiments on weight and KV-cache compression demonstrate improved reconstruction quality compared to baselines.

Momentum

Related topics

Latest papers 58