Low-Rank Factorization

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

9 new papers

A weekly snapshot of new work published in Low-Rank Factorization.

Inside this field

Focused directions

Focused topic

Low-Rank Factorization

Within Low-Rank Factorization

169 papersExplore

Focused topic

Singular Value Decomposition

Within Low-Rank Factorization

82 papersExplore

Focused topic

Gradient-Free Rank Allocator

Within Low-Rank Factorization

17 papersExplore

253 papers

Latest in Low-Rank Factorization

Browse all topics →

TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
Jun 4, 2026Marius Dragoi, Ioana Pintilie, Alexandra Dragomir +2Efficient Continual Learning Continual Learning Methods
Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models
Jun 4, 2026Sunny Gupta, Shambhavi Shanker, Amit SethiLow-Rank Adaptation Wireless Foundation Models
Sparse Functional Singular Value Decomposition for Biclustering and Triclustering Longitudinal Data
Jun 3, 2026Yue Zhao, Thierry Chekouo, Sandra SafoLongitudinal Data Singular Value Decomposition
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
Jun 3, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLarge Language Model Compression Low-Rank Factorization
HalfNet: Randomized Neural Networks with Learned Subspace Geometry
Jun 3, 2026Ethem AlpaydinNeural Network Low-Rank Factorization
LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
Jun 3, 2026Yuanrui Wang, Xingxuan Zhang, Han Yu +7Tabular Foundation Models Tensor Programs
Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View
Jun 3, 2026Mingyu LiLow-Rank Factorization Regularization Techniques
Parameter-Efficient Fine-Tuning with Learnable Rank
Jun 3, 2026Arpit Garg, Simon Lucey, Hemanth SaratchandranParameter-Efficient Fine-Tuning Low-Rank Adaptation
Low-rank Distributional Matrix Completion
Jun 2, 2026Jiayi Wang, Raymond K. W. WongTensor Completion Low-Rank Factorization
Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs
Jun 2, 2026Wei Ding, Yudong Zhang, Ruobing Xie +3Encoding Models Encoders
Bayesian Tensor Decomposition with Diffusion Model Prior
Jun 2, 2026Zerui Tao, Qibin ZhaoDiffusion Priors Low-Rank Factorization
Riemannian Gradient Descent for Low-Rank Architectures
Jun 1, 2026Nicholas KnightLow-Rank Factorization Riemannian Optimization
GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
May 31, 2026Shihao Zhang, Rayan SaabWeight-Only Quantization Quantization
Riemannian Optimization for Hadamard Products of Low-Rank Matrices
May 31, 2026Pratik Jawanpuria, Ankish Chandresh, Bamdev MishraRiemannian Optimization Low-Rank Factorization
PRISM: Gauge-Invariant Tangent-Space Differentially Private LoRA
May 31, 2026Shihao Wang, Xueru ZhangDifferential Privacy Low-Rank Adaptation
LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Large Language Model Compression Low-Rank Factorization
Stochastic Rounding Increases Small Singular Values
May 29, 2026Linkai Ma, Tingzhou Yu, Petros DrineasSingular Value Decomposition Full-Precision Performance
Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
May 29, 2026Valérie Castin, Kimia Nadjahi, Pierre Ablin +1Low-Rank Adaptation Finetuning
CLUBench: A Clustering Benchmark
May 28, 2026Feng Xiao, Dazhi Fu, Chris Ding +1Image Clustering Low-Rank Factorization
Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
May 28, 2026Junlin He, Yihong Tang, Tong Nie +5Chain-Of-Thought Distillation Large Language Model Reasoning
Manifold-based Algorithms for the Hadamard Decomposition
May 27, 2026Nicolas Gillis, Subhayan Saha, Stefano Sicilia +1Singular Value Decomposition Low-Rank Factorization
History-aware adaptive reduced-order models via incremental singular value decomposition
May 27, 2026Amirpasha Hedayat, Ali Mohaghegh, Laura Balzano +2Reduced-Order Models Singular Value Decomposition
Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability
May 27, 2026Jingzhe Wang, Hung-Hsu ChouLow-Rank Factorization Matrix Factorization
Efficient Pre-Training of LLMs through Truncated SVD Layers
May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Singular Value Decomposition Large Language Model Pretraining
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
May 27, 2026Cheng Chen, Pengpeng Zeng, Yuyu Guo +3Low-Rank Adaptation Efficient Continual Learning
Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V
May 26, 2026Junhao Wu, Dezhong Yao, Hai JinW4A4 Quantization Singular Value Decomposition
The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training
May 26, 2026Hongtao Zhang, Wenjie Zhou, Chenxi Jia +2Large Language Model Pretraining Learning Rates
Learning Nonlinear Factor Models with Unknown Monotone Links from Incomplete and Noisy Data
May 25, 2026Yutong Chao, Resat Gökhan, Jalal Etesami +1Exploratory Factor Analysis Low-Rank Factorization
Acting on the Unseen: Communication-Free Collaborative Filtering for Decentralized Multi-Robot Task Allocation
May 25, 2026Alexander Apartsin, Yigal Meshulam, Yehudit ApersteinMulti-Robot Systems Human-To-Robot Transfer
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5Mixture-Of-Expert Language Models Mixture-of-Experts Models
Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent
May 22, 2026Anherutowa CalvoCondition Number Curvature
MARS: Magnitude-Aware Rank Statistics
May 22, 2026Muhammad Rajabinasab, Afsaneh M. Nejad, Arthur ZimekStatistical Models Low-Rank Factorization
Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation
May 22, 2026Guoming Li, Shangyu Zhang, Junwei Pan +7Recommendation Low-Rank Factorization
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
May 21, 2026Connall Garrod, Jonathan P. Keating, Christos ThrampoulidisNeural Collapse Feature Learning
Optimization over the intersection of manifolds
May 21, 2026Yan Yang, Bin Gao, Ya-xiang YuanManifolds Low-Rank Factorization
Check Your LLM's Secret Dictionary! Five Lines of Code Reveal What Your LLM Learned (Including What It Shouldn't Have)
May 21, 2026Hisashi MiyashitaLarge Language Singular Value Decomposition
Group-Aware Matrix Estimation and Latent Subspace Recovery
May 19, 2026Hamza Golubovic, Matthew Shen, Genevera I. Allen +1Tensor Completion Low-Rank Factorization
FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
May 19, 2026Yequan Zhao, Ruijie Zhang, Liyan Tan +3Parameter-Efficient Fine-Tuning Low-Rank Adaptation
Group-Algebraic Tensors: Provably-optimal Equivariant Learning and Physical Symmetry Discovery
May 19, 2026Paulina Hoyos, Shashanka Ubaru, Dongsung Huh +5Tensor Symmetry
Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
May 19, 2026Aarash Abro, Muhammad TahirFinetuning Catastrophic Forgetting
Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
May 19, 2026Chongyu Fan, Gaowen Liu, Mingyi Hong +2Muon Reinforcement Learning With Verifiable Reward
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
May 18, 2026Chengjie Hong, Feixiang He, Yiheng Zeng +2Wireless Foundation Models Learned Image Compression
Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings
May 17, 2026Wenxuan XiaoQuantizer Embedding Dimension
FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
May 16, 2026Ramakrishnan SathyavageeswaranLow-Rank Adaptation Gradient-Free Rank Allocator
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
May 16, 2026Sayeh Sharify, Mahsa Salmani, Hesham MostafaMultimodal Diffusion Transformers Low-Bit
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
May 15, 2026Ali Abbasi, Chayne Thrash, Haoran Qin +2Large Language Model Compression Singular Value Decomposition
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Visual Tokens Large Vision Language Models
LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learning
May 14, 2026Changryeol Choi, Hyewon Park, Yujin Kwon +1Tabular Learning Ensemble
Beyond Perplexity: A Geometric and Spectral Study of Low-Rank Pre-Training
May 13, 2026Namrata Shivagunde, Vijeta Deshpande, Sherin Muckatira +1Low-Rank Factorization Language Model Perplexity
Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
May 13, 2026Kunil Lee, Ki-Young Shin, Jong-Hyeok Lee +1Multimodal Knowledge Editing Multilingual
Multi-Task Representation Learning for Conservative Linear Bandits
May 12, 2026Jiabin Lin, Shana MoothedathLinear Bandits Bandits
Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
May 12, 2026Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1Gradient Clipping Spectral Norm
TCP-SSM: Efficient Vision State Space Models with Token-Conditioned Poles
May 12, 2026Sara Shoouri, Morteza Tavakoli Taba, Hun-Seok KimState Space Models Self-Supervised Vision Transformers
Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models
May 11, 2026Nikolaos Nakis, Panagiotis Promponas, Konstantinos Tsirkas +4Graph Representation Learning Low-Rank Factorization
Elucidating Representation Degradation Problem in Diffusion Model Training
May 11, 2026Zhipeng Yao, Dazhou Li, Zitong Zhang +6Discrete Diffusion Model Low-Rank Factorization
Valid Best-Model Identification for LLM Evaluation via Low-Rank Factorization
May 11, 2026Elad Tolochinsky, Yaniv Tenzer, Yaniv RomanoLarge Language Model Evaluation Model Selection
Scalable Gaussian process inference via neural feature maps
May 11, 2026Anthony StephensonGaussian Process Kernel Method
Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation
May 11, 2026Yujia Cai, Boxuan Li, Chenghao Xu +1Large Vision Language Models Retrievers
Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
May 10, 2026Shabarish Chenakkod, Michał DerezińskiLow-Rank Factorization Principal Component Analysis

Jun 4, 2026 · cs.LGJ/K move · Enter open · S save

TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning

Marius Dragoi, Ioana Pintilie, Alexandra Dragomir, Antonio Barbalau+1

Bitdefender, Romania

Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.

Efficient Continual Learning Continual Learning Methods Singular Value Decomposition Quadratic Slack Penalty Principal Component Analysis

Open paper Ask about it PDF