Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

538 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

Improving JEV Entailment Reranking with Span Readouts and Selective Adaptation
Oct 26, 2026Haocheng Mei, Yuchuan Tian, Ye Yuan +5LLM Reranking Learning to Rank
Peking University · SUIBE · University of Macau +2
Mechanistic Insights into LoRA: Layer Sparsity for Adaptive Fine-Tuning via Path Patching
Oct 26, 2026Fei Zuo, Yizhou Huang, Kezhi WangLow-Rank Adaptation Parameter-Efficient Fine-Tuning
East China Normal University · Brunel University of London
HARP-LoRA: Hierarchy-Conditioned Rank Projection for Low-Rank Adaptation in Scientific Table Understanding
Oct 25, 2026Hossein Sholehrasa, Mohammadhossein Amini, Doina Caragea +8Low-Rank Adaptation Table QA
1DATA Consortium and FARAD Program, Kansas State University, Olathe, KS, USA · Department of Computer Science, Kansas State University, Manhattan, KS, USA · Olin Business School, Washington University in St. Louis, St. Louis, Missouri, USA +5
In-Context Adapter Generation for Compositional Generalization in Large Audio Language Models
Oct 25, 2026Yiwei Guo, Siqi Huang, Bohan Li +4Compositional Generalization Audio-Language Models
X-LANCE Lab, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University · Jiangsu Key Lab of Language Computing · Xi’an Jiaotong University
Task Format as a Factor in Adaptation: A Controlled Comparison of LoRA and Full Fine-Tuning for Prompt-Safety Classification
Oct 24, 2026John Alderete, Srirakshith Betageri, Jennifer Flynn +3Fine-Tuning Language Models Low-Rank Adaptation
Fiddler AI · Simon Fraser University · Georgia Institute of Technology
Singular Geometry of Low-Rank Adaptation
Oct 23, 2026Kamillo Ferry, Anthea Monod, Keisuke YanoLow-Rank Adaptation Singular Learning Theory
Department of Mathematics, Imperial College London, UK · Institute of Statistical Mathematics, Japan
Rethinking LoRA Initialization for Robust Asymmetric Learning Rates
Oct 23, 2026Disen Liao, Yaoliang YuLow-Rank Adaptation Parameter-Efficient Fine-Tuning
University of Waterloo Vector Institute
LOCI: Localized Continuous Control for Multi-Subject Image Editing
Oct 22, 2026Sai Vidyaranya Nuthalapati, Shlok Kumar MishraImage Editing Text-Guided Image Editing
Meta AI
Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters
Oct 22, 2026Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov +1Low-Rank Adaptation Language Model Merging
Bosch Center for Artificial Intelligence, Germany · LMU Munich, Germany · Munich Center for Machine Learning, Germany
LoRA Superposition across Pretrained Models
Oct 22, 2026Keigo Watanabe, Hiroshi Kera, Kazuhiko KawamotoLow-Rank Adaptation Model Merging
Chiba University, Chiba, Japan · National Institute of Informatics, Tokyo, Japan
Convergent Performance, Divergent Mechanisms in LoRA-Adapted Vision Foundation Models for Brain MRI
Oct 20, 2026Cecilia Diana-Albelda, Fiona Kekwick, Raghav Mehta +6Vision Foundation Model Adaptation Low-Rank Adaptation
Department of Electronic Technology and Communications, Universidad Autónoma de Madrid, Madrid, Spain · Department of Computing, Imperial College London, London, United Kingdom · Department of Brain Sciences, Imperial College London, London, United Kingdom +3
Parameter-Efficient Adaptation of Time Series Foundation Models for Electrical Load Forecasting
Oct 20, 2026Aliullah, Julian Oelhaf, Edouard Fouché +4Electricity Load Forecasting Time-Series Foundation Models
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg · Siemens AG, Smart Infrastructure · Department of Electrical Engineering, Media and Computer Science, Ostbayerische Technische Hochschule Amberg-Weiden
Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs
Oct 20, 2026Jiapu WangTemporal Reasoning in Language Models Temporal KGs
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, 210094, China
LoRASpace: A Pool-Wide Shared Substrate for Static and Dynamic Multi-LoRA Composition
Oct 20, 2026Thamirawaran Sathiyalogeswaran, Navindu De Silva, Chamikara Siriwardane +4Low-Rank Adaptation LLM Inference Acceleration
University of Moratuwa, Sri Lanka · University of Michigan, USA
WeAct: Dynamic Weight Switching as Agent Actions
Oct 19, 2026Rui Shi, Xunjian Yin, Jiacheng Sang +2LLM Agent Training Continual Learning for LLM Agents
Carnegie Mellon University · City University of Hong Kong · Duke University +1
Scaling Parameter-Efficient Fine-Tuning for Brain Encoding
Oct 19, 2026Zhuofan Josh Ying, Guanglong Sun, Nolan Ho +6Neural Encoding Parameter-Efficient Fine-Tuning
Columbia University · Mind Lab · Tsinghua University +1
An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs
Oct 19, 2026Minglei Li, Peng Ye, Haonan He +3Mixture-of-Experts Fine-Tuning Parameter-Efficient Fine-Tuning
Fudan University · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong +3
Residual-Guided Adaptive Tucker Rank Allocation for Parameter-Efficient MCI Conversion Prediction
Oct 19, 2026Guanghua He, Hancan Zhu, Gaohang Yu +1Adaptive Rank Allocation Parameter-Efficient Fine-Tuning
Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China
PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models
Oct 19, 2026Qihan Huang, Lingxia Zhang, Guye Zhang +2Federated Fine-Tuning Federated Learning
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, Sichuan, China · Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China, Chengdu, Sichuan, China · Sichuan Provincial Key Laboratory for Human Disease Gene Study and the Center for Medical Genetics, Department of Laboratory Medicine, Sichuan Academy of Medical Sciences and Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, Sichuan, China +1
First-Order LoRA Training with Phone NPUs
Oct 17, 2026Xu Zheng, Wei Cheng, Zhuomin Chen +4On-Device Training LLM Fine-Tuning
Florida International University, Miami, FL, USA · NEC Laboratories America, Princeton, NJ, USA · North Carolina State University, Raleigh, NC, USA +2
Safety Kernel: Mechanistic Alignment for Cross-Lingual Defense in LLMs
Oct 17, 2026Chengyu Xie, Fei Shen, Shuyi Miao +5LLM Safety Language Model Robustness
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Beihang University, Beijing, China +3
ASR Personalization in Pathological Voice: Accuracy, Efficiency, and Duration Effects
Oct 17, 2026Giuseppe Giaimis, Sergei Katkov, Muhammad Azfar Yaqub +1Automatic Speech Recognition Personalized Speech Recognition
Free University of Bolzano, Bolzano, Italy
How Finite-Rank Bottlenecks Shape the Low-Rank Adaptation Landscape
Oct 16, 2026Long Nguyen-Chi, Quynh Nguyen, Thanh Nguyen-Cung +1Low-Rank Adaptation Nonconvex Optimization
College of Engineering & Computer Science, VinUniversity · Center for AI Research, VinUniversity, Hanoi, Vietnam.
Ori: Drift-Aware Streaming $k$ -PCA for Very Low-Rank LLM Training
Oct 16, 2026Xinyu Luo, Jincheng Zhou, Mengbo Wang +5LLM Training Low-Rank Adaptation
Department of Computer Science, Purdue University. · Purdue Institute for Cancer Research, Purdue University. · Department of Comparative Pathobiology, Purdue University.
Don't Retrain, Remix: Adaptation of Web Agents using Mixture Hypernetworks
Oct 15, 2026Md Farhan Ishmam, Minh Pham Dinh, Kenneth MarinoWeb Agents Hypernetworks
Kahlert School of Computing, University of Utah
MorphAct: Robot Policies Morphing with the Worlds They Act In
Oct 14, 2026Ziyan Liu, Jiting Liu, Weihong Zhong +7Robot Policy Adaptation VLMs for Robotics
University of Science and Technology of China · School of Artificial Intelligence, Shanghai Jiao Tong University · Tongji University +3
Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters
Oct 14, 2026Taha Sabir, Alberto Montero Molina, Thinh Quang Ly +3Medical Image Analysis VLM Adaptation
Department of Computer Science, Gettysburg College, Gettysburg, PA 17325, USA · Higher School of Computing (ESCOM), Instituto Politécnico Nacional (IPN), Mexico City 07738, Mexico · Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA +2
ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning
Oct 14, 2026Fanyao Li, Yuxi Liu, Xiaohao Liu +3Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Communication University of China · National University of Singapore · Xidian University
How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts
Oct 13, 2026Mei Okonkwo, Owen Castellanos, Ingrid Solberg +1Low-Rank Adaptation Adaptive Rank Allocation
University of Edinburgh · Delft University of Technology
Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models
Oct 13, 2026Yupeng Qiu, Han Fang, Dhananjai Sharma +1T2I Diffusion Models Diffusion Models
National University of Singapore · University of Science and Technology of China

Oct 26, 2026J/K move · Enter open · S save

Improving JEV Entailment Reranking with Span Readouts and Selective Adaptation

Haocheng Mei, Yuchuan Tian, Ye Yuan, Enhua Wu+4

Peking University · SUIBE · University of Macau+2

LLM Reranking Learning to Rank Selective Fine-Tuning Low-Rank Adaptation

Open paper AskPDF arXiv

JEV decision models pair a language model backbone with a lightweight scoring head to select among natural-language options. Direct selection compares candidate answers within one request, whereas entailment reranking evaluates each candidate in a separate natural language inference (NLI) request over fixed relation options. We examine whether the readout and adaptation choices used for direct selection remain effective for cross-request reranking. We combine a mean-span readout with selective backbone adaptation. Mean-span pooling replaces the option-ending pointer with an average of the contextualized states inside each NLI option, without adding head parameters. Selective adaptation retains low-rank updates in attention and gated DeltaNet projections while removing feed-forward adapters. On MC11, which averages accuracy over eleven reranking configurations from seven sources, selective adaptation improves NeoHorse-1-4B accuracy over the all-module baseline while cutting trainable parameters by more than half, and mean-span pooling yields further consistent gains at the same parameter count.