Low-Rank Adaptation

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

23 new papers

A weekly snapshot of new work published in Low-Rank Adaptation.

Period ending 2026-09-14

8 new papers

A weekly snapshot of new work published in Low-Rank Adaptation.

Period ending 2026-09-07

14 new papers

A weekly snapshot of new work published in Low-Rank Adaptation.

Inside this field

Focused directions

Focused topic

Low-Rank Adaptation

Within Low-Rank Adaptation

293 papersExplore

Focused topic

Finetuning

Within Low-Rank Adaptation

140 papersExplore

Focused topic

Large Language Model Fine-Tuning

Within Low-Rank Adaptation

139 papersExplore

Focused topic

Large Language Model Adaptation

Within Low-Rank Adaptation

117 papersExplore

Focused topic

Supervised Fine-Tuning

Within Low-Rank Adaptation

109 papersExplore

Focused topic

Parameter-Efficient Fine-Tuning

Within Low-Rank Adaptation

108 papersExplore

Focused topic

Low-Rank Adaptation Adapters

Within Low-Rank Adaptation

63 papersExplore

Focused topic

Continual Fine-Tuning

Within Low-Rank Adaptation

28 papersExplore

858 papers

Latest in Low-Rank Adaptation

Browse all topics →

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
May 12, 2026Chengqian Zhang, Wei Zhu, Kyumin LeeLow-Rank Adaptation Finetuning
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
May 12, 2026Arijit Sehanobish, Charles LoveringLow-Rank Adaptation Adapters Gradient Concentration
A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning
May 12, 2026Haibo Chen, Xin Wang, Jiaheng Chao +2Gnn-Based Detectors Graph Language Models
Robust Promptable Video Object Segmentation
May 12, 2026Sohyun Lee, Yeho Gwon, Lukas Hoyer +3Video Object Segmentation Low-Rank Adaptation
LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection
May 12, 2026Lanxin Zhao, Bamdev Mishra, Pratik Jawanpuria +4Parameter-Efficient Fine-Tuning Ceiling
Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs
May 12, 2026Amr Abourayya, Jens Kleesiek, Michael KampLarge Language Model Fine-Tuning Communication Efficiency
Probabilistic Calibration Is a Trainable Capability in Language Models
May 12, 2026Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh +2Training Language Models Finetuning
PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head
May 12, 2026Chieh-Yen Lin, Shao-Hua SunPrism Large Language
Efficient LLM-based Advertising via Model Compression and Parallel Verification
May 12, 2026Wenxin Dong, Chang Gao, Guanghui Yu +9Ads LLM Inference Efficiency
TB-AVA: Text as a Semantic Bridge for Audio-Visual Parameter Efficient Finetuning
May 12, 2026Seongah Kim, Dinh Phu Tran, Hyeontaek Hwang +3Audio Latent Cross-Modal Learning
A Mechanistic Investigation of Supervised Fine Tuning
May 12, 2026Ruhaan ChopraSupervised Fine-Tuning Model Activations
Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning
May 11, 2026Han Zheng, Yining Ma, Karthick Gunasekaran +4Curriculum Reinforcement Learning Reinforcement Fine-Tuning
Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models
May 11, 2026Shourya Bose, Helgi Hilmarsson, Dhruv SuriAc Optimal Power Flow Newton
CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
May 11, 2026Wenxuan Song, Han Zhao, Fuhao Li +7Vision-Language Model Adaptation Vision-Language-Action Models
V4FinBench: Benchmarking Tabular Foundation Models, LLMs, and Standard Methods on Corporate Bankruptcy Prediction
May 11, 2026Marcin Kostrzewa, Sebastian Tomczak, Roman Furman +5Credit Risk Prediction Tabular Foundation Models
Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
May 11, 2026Saba Pourhanifeh, AbdulAziz AbdulGhaffar, Ashraf MatrawyThreat Models Large Language Model Adaptation
DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
May 11, 2026Eleonora Gualdoni, Sonia Laguna, Louis Bethune +3Large Language Model Fine-Tuning Target Domain
AdaPaD: Adaptive Parallel Deflation for PEFT with Self-Correcting Rank Discovery
May 11, 2026Barbara Su, Fangshuo Liao, Anastasios KyrillidisLow-Rank Adaptation Parameter-Efficient Fine-Tuning
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs
May 11, 2026Krishak Aneja, Manas Mittal, Anmol Goel +2Harmful Fine-Tuning Large Language Model Alignment
Where Does Long-Context Supervision Actually Go? Effective-Context Exposure Balancing
May 11, 2026Jinchang Zhu, Jindong Li, Chengyu Zou +4Efficient Long-Context Inference Token-Level Supervision
Can Muon Fine-tune Adam-Pretrained Models?
May 11, 2026Xingyu Qu, Peigeng Huang, Samuel HorvathAdam Muon
Filtering Memorization from Parameter-Space in Diffusion Models
May 11, 2026Yu Zhe, Yang Jiayan, Wei Junhao +2Low-Rank Adaptation Memorization
Phoenix-VL 1.5 Medium Technical Report
May 11, 2026Team Phoenix, :, Arka Ray +30Large Language Model Adaptation Domain Adaptation
HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
May 11, 2026Jia Wei, Zhonghao Zhang, Ping Chen +5Mixture-Of-Expert Language Models Low-Rank Adaptation
Team-Based Self-Play With Dual Adaptive Weighting for Fine-Tuning LLMs
May 11, 2026Wu Li, Yigeng Zhou, Zesheng Shi +3Large Language Model Alignment Self-Play
NaiAD: Initiate Data-Driven Research for LLM Advertising
May 11, 2026Yihang Zhang, Zimeng Huang, Ren Zhai +2Ads Large Language Model Adaptation
Concordia: Self-Improving Synthetic Tables for Federated LLMs
May 11, 2026Jimin Huang, Duanyu Feng, Nuo Chen +8Federated Learning Large Language Model Adaptation
Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction
May 11, 2026Anushree BerliaLarge Vision Language Models Low-Rank Adaptation
EvoPref: Multi-Objective Evolutionary Optimization Discovers Diverse LLM Alignments Beyond Gradient Descent
May 10, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLarge Language Model Alignment Evolutionary Search
Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning
May 10, 2026Ruiying Peng, Mengyu Yang, Jing Lei +3Supervised Fine-Tuning Mathematical Reasoning
SearchSkill: Teaching LLMs to Use Search Tools with Evolving Skill Banks
May 9, 2026Jinchao Hu, Meizhi Zhong, Kehai Chen +1Agentic Search Multi-Hop Question Answering Benchmarks
LLiMba: Sardinian on a Single GPU -- Adapting a 3B Language Model to a Vanishing Romance Language
May 9, 2026Luca BalloreLarge Language Model Adaptation Text Corpora
Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning
May 9, 2026Minmin Zhang, Sina Aghaei, Soroush SaghafianIn-Context Learning Large Language Model Fine-Tuning
When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneity
May 9, 2026Kiran Naseer, Umar ShoaibFederated Learning Fairness-Aware Learning
Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT
May 9, 2026Tianyi Zhang, Shaopeng Zhai, Haoran Zhang +2Flow-Matching Vision-Language-Action Supervised Fine-Tuning
Semantic Alignment in Hyperbolic Space for Open-Vocabulary Semantic Segmentation
May 9, 2026Hoang M. Truong, Hai Nguyen-Truong, Dang HuynhDense Semantic Segmentation Hyperbolic Space
AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation
May 9, 2026Ziyun Liu, Fengmiao Bian, Jian-Feng CaiLow-Rank Adaptation Spectral Preconditioning
Gate-and-Merge: Zero-shot Compositional Personalization of Vision Language Models
May 9, 2026Guodong Ding, Angela YaoPersonalization Low-Rank Adaptation
EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints
May 9, 2026Jiaxiang Geng, Yiyi Lu, Lunyu Zhao +3Heterogeneous Edge Devices Large Language Model Fine-Tuning
S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain
May 9, 2026Baoquan Zhang, Zhehao Yu, Lisai Zhang +5Parameter-Efficient Fine-Tuning Finetuning
ZAYA1-VL-8B Technical Report
May 8, 2026Hassan Shapourian, Kasra Hejazi, Olabode M. Sule +1Large Vision Language Models Vision Encoders
MARLaaS: Multi-Tenant Asynchronous Reinforcement Learning as a Service
May 8, 2026Timothy Tin Long Yu, Gursimran Singh, Ge Shi +3Reinforcement Fine-Tuning Large Language Model Fine-Tuning
Rotation-Preserving Supervised Fine-Tuning
May 8, 2026Hangzhan Jin, Tianwei Ni, Lu Li +3Supervised Fine-Tuning Fisher Information
Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms
May 8, 2026Omatharv Bharat Vaidya, Connor T. Jerzak, Nhat Ho +1Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Decoupling Endpoint and Semantic Transition Learning for Zero-Shot Composed Image Retrieval
May 8, 2026Mingyu Liu, Sihan Huang, Yijia Fan +4Composed Image Retrieval Single Images
Private Vertical Federated Inference for Time-Series
May 8, 2026Lucas Fenaux, Larris Xie, Aditya Bang +3Federated Learning Homomorphic Encryption
Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs
May 8, 2026Hanlin Cai, Kai Li, Houtianfu Wang +4Large Language Model Fine-Tuning Language Model Evasion Attacks
Convergent Stochastic Training of Attention and Understanding LoRA
May 8, 2026Zhengkai Sun, Dibyakanti Kumar, Alejandro F Frangi +2Low-Rank Adaptation Trainable Parameter
MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
May 8, 2026Ionut-Vlad Modoranu, Mher Safaryan, Dan AlistarhLow-Rank Adaptation Low-Rank Adaptation Adapters
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Low-Rank Adaptation Parameter-Efficient Fine-Tuning
mHC-SSM: Manifold-Constrained Hyper-Connections for State Space Language Models with Stream-Specialized Adapters
May 8, 2026Abdulvahap Mutlu, Şengül Doğan, Türker TuncerLarge Language Model Adaptation Residual Connections
Gradient-Based LoRA Rank Allocation Under GRPO: An Empirical Study
May 8, 2026Yash Ganpat SawantGradient-Free Rank Allocator Group Relative Policy Optimization
Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation
May 8, 2026Junjie Yu, Yue Wang, Zihan Deng +3Model Pretraining Finetuning
TAS-LoRA: Transformer Architecture Search with Mixture-of-LoRA Experts
May 8, 2026Jeimin Jeon, Hyunju Lee, Bumsub HamSelf-Supervised Vision Transformers Neural Architecture Search
PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat
May 8, 2026Srikar Kashyap PulipakaToxicity Harmful Content
Goal-Conditioned Supervised Learning for LLM Fine-Tuning
May 8, 2026Shijun Li, Kaiwen Dong, Xiang Gao +1Large Language Model Fine-Tuning Large Language Model Alignment
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3Low-Rank Adaptation Large Language Model Adaptation
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
May 7, 2026Yuxing Liu, Jianyu Wang, Tong ZhangSupervised Fine-Tuning Model Pretraining
Crafting Reversible SFT Behaviors in Large Language Models
May 7, 2026Yuping Lin, Pengfei He, Yue Xing +5Supervised Fine-Tuning Model Activations
UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
May 7, 2026Yiqiao Jin, Yiyang Wang, Lucheng Fu +7Unsupervised On-Policy Self-Distillation Large Language Model Adaptation

May 12, 2026 · cs.LGJ/K move · Enter open · S save

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training

Chengqian Zhang, Wei Zhu, Kyumin Lee

Worcester Polytechnic Institute · University of Hong Kong

Post-training has become essential for adapting large language models (LLMs) to complex downstream behaviors, including instruction following, preference alignment, and multi-step reasoning. Reinforcement learning with verifiable rewards (RLVR) has recently emerged as a particularly effective post-training paradigm for improving reasoning capabilities, with critic-free algorithms such as GRPO and GSPO enabling scalable optimization. However, RLVR post-training with full fine-tuning (FFT) requires substantial GPU memory and incurs high training costs. Although parameter-efficient fine-tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), effectively reduce computational costs, they often suffer from a noticeable performance gap compared to full fine-tuning in post-training for complex reasoning tasks. In this paper, we propose Hybrid-LoRA, an efficient hybrid post-training framework that selectively applies full fine-tuning to a small subset of modules less suited to low-rank adaptation, while adapting the remaining components with LoRA. We introduce a novel Hybrid-LoRA Score to rank candidate modules according to their sensitivity to low-rank adaptation under a fixed parameter budget. Experiments show that Hybrid-LoRA closely matches full fine-tuning performance under a 10% full fine-tuning module budget, with the remaining candidate modules adapted by LoRA, consistently outperforming four state-of-the-art PEFT post-training baselines, achieving improvements of up to 5.65% and on average 4.36% over the best baseline.

Low-Rank Adaptation Finetuning LLM Post-Training Methods

Open paper Ask about it PDF