Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

LoRA-based Adaptation Alone Is Not Enough: Understanding the Limits of Foundation Models for Face Presentation Attack Detection
Aug 10, 2026Peter Lorenz, Anjith George, Marcel SébastienVision Foundation Models Vision Foundation Model Adaptation
Idiap Research Institute, Martigny, Switzerland · University of Lausanne (UNIL), Lausanne, Switzerland
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Topology-Aware Parameter-Efficient Adaptation for Cross-Dataset Retinal Vessel Segmentation
Aug 10, 2026Yongsong Huang, Tomo Miyazaki, Kai Xu +3Retinal Vessel Segmentation Domain Adaptation
Tohoku University · Yale University
LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks
Aug 7, 2026Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj +2Domain Generalization Vision Foundation Model Adaptation
Department of Oncology, McMaster University, Hamilton, Canada · Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada. +1
Unsupervised Adaptation of PDE Foundation Models
Aug 7, 2026Ziye Song, Zhao Wei, Xin Yu +2Unsupervised Domain Adaptation Low-Rank Adaptation
Nanyang Technological University · CFAR, A*STAR · Adelaide University
LoRAScan: Detecting Backdoor Prompts in Low-Rank Adapters for Large Language Models via Down-Projection Activation Spikes
Aug 7, 2026Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo +1LLM Backdoor Attacks Backdoor Detection
Michigan Technological University · Miami University · Lehigh University
Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization
Aug 6, 2026Grigory MalinovskyByzantine-Robust Federated Learning Communication-Efficient Distributed Training
King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia
Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning
Aug 5, 2026Yue Han, Ziniu LiuSupervised Fine-Tuning Multi-Task Learning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing
Aug 5, 2026Simon Lösche, Barış Büyüktaş, Mathis Adler +3Remote Sensing Image Understanding Prompt Learning
BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany · Technische Universität Berlin, 10623 Berlin, Germany · Orion Lab, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15772 Athens, Greece +2
From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models
Aug 4, 2026Fusheng LuoLLM Evaluation Quantitative Finance
Johns Hopkins University Baltimore, Maryland, USA
MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization
Aug 4, 2026Gengyuan Liu, Nanzhou Wang, Chang Liu +5Vision-Language Models VLM Adaptation
Tsinghua University Shenzhen, China · Tsinghua University Beijing, China · Huazhong University of Science and Technology Wuhan, China +1
FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation
Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning Aggregation Communication-Efficient Distributed Training
Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden
Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving
Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference Efficiency LLM Serving
1Purdue University · 2Oregon State University
MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation
Aug 4, 2026Yiming Zeng, Lei Lu, Zexin Li +10LLM Fine-Tuning Low-Rank Adaptation
University of Connecticut · Northeastern University · Nanyang Technological University +5
LoCA: Forward-Only LLM Tuning after One-Shot Calibration with Local Credit Assignment
Aug 4, 2026Linhan Xia, Rui Liu, Zhaofeng Zhang +3Fine-Tuning LLM Fine-Tuning
University of Oklahoma · 4Tencent · 2Imperial College London +5
Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts
Aug 3, 2026Tom Saliencro, Rohan Desai, Priya Nair +2Adaptive Model Routing Selective Prediction
University of California, Irvine · University of Washington
SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models
Aug 3, 2026Xingyan Li, Jordan A. Caraballo-Vega, Jie Gong +2Remote Sensing Image Segmentation Geospatial Foundation Models
University of Maryland, Baltimore County Baltimore, Maryland, USA · NASA Goddard Space Flight Center Greenbelt, Maryland, USA
ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation
Aug 2, 2026Mohamed Farag, Genc Hoxha, Yahia Maleki +2Image Segmentation Ensemble Learning
Machine Learning in Agriculture Lab, Institute of Geodesy and Geoinformation, University of Bonn, Bonn, Germany · Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia · Lamarr Institute for Machine Learning and Artificial Intelligence, University of Bonn, Bonn, Germany
EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuning
Aug 2, 2026Srinivas Anumasa, Dianbo LiuUncertainty Quantification Low-Rank Adaptation
National University of Singapore
Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selection
Jul 31, 2026Chaimae Abouzahir, Musa Khan, Hala Ali-Hassan +7LLM Fine-Tuning Low-Rank Adaptation
New York University Abu Dhabi · Cleveland Clinic Abu Dhabi
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs
Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMs Low-Rank Adaptation
The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK
Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection
Jul 30, 2026Arunan JSample Complexity Low-Rank Adaptation
Independent Researcher, Chennai, India.
Compliance2LoRA: Personalizable On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters
Jul 30, 2026Pankayaraj Pathmanathan, Furong HuangLLM Alignment LLM Safety Alignment
University of Maryland College Park
Between Gradient and Natural Gradient: A Continuum of LoRA Initializations
Jul 28, 2026Dianze Liu, Farshid GhezelbashLow-Rank Adaptation
Georgia Institute of Technology
Detecting CSAM Text-to-Image LoRAs From Weights
Jul 28, 2026David Demitri Africa, Cate Heine, Nadine Staes-Polet +1T2I Generation Low-Rank Adaptation
UK AI Security Institute
Localized Adaptation Reveals Distinct Learning Signatures in Transformers
Jul 28, 2026Rebecca Ramnauth, Brian ScassellatiFine-Tuning Transformer
Yale University
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamMemory-Efficient Fine-Tuning Small Language Models
Independent Researcher · Jaipur, India · Hamburg, Germany
Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
Jul 28, 2026Jaeha Kim, Kyoung Mu LeeDiffusion Models for Image Restoration Low-Rank Adaptation
Dept. of ECE & ASRI, Seoul National University, Seoul, Korea · IPAI, Seoul National University, Seoul, Korea
Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning
Jul 28, 2026Yuan Zhang, Jiang Hu, Zhijian Lai +2Stiefel Manifold Optimization Riemannian Optimization
Center for Data Science, Peking University, Beijing 100871, China · Yau Mathematical Sciences Center, Tsinghua University, Beijing 100084, China · Beijing International Center for Mathematical Research, Peking University, Beijing 100871, China +2
The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning
Jul 26, 2026Peng XieLLM Fine-Tuning Low-Rank Adaptation
Technical University of Munich, Campus Heilbronn

Aug 10, 2026J/K move · Enter open · S save

LoRA-based Adaptation Alone Is Not Enough: Understanding the Limits of Foundation Models for Face Presentation Attack Detection

Peter Lorenz, Anjith George, Marcel Sébastien

Idiap Research Institute, Martigny, Switzerland · University of Lausanne (UNIL), Lausanne, Switzerland

Vision Foundation Models Vision Foundation Model Adaptation Low-Rank Adaptation Face Presentation Attack Detection Cross-Dataset Generalization

Open paper AskPDF arXiv

Face presentation attack detection (PAD) aims to reliably detect a wide range of presentation attacks. While PAD methods achieve strong performance within individual datasets, their performance degrades under cross-dataset evaluation. Variations in sensors or lighting conditions can reduce the effectiveness of detectors from near-perfect to nearly random. Foundation models (FMs) have emerged as a promising alternative because typical PAD datasets, such as the MCIO benchmarks (MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU), are small relative to the scale used for web-based pretraining. However, existing PAD systems primarily focus on CLIP-based foundation models, while overlooking other FMs with different architectures and training procedures. This study addresses this question by systematically evaluating 32 FMs. Zero-shot prompting achieves performance near chance across model families and scales. The vision encoders, when low-rankadapted (LoRA) with fewer than 1% trainable weights, achieve below 2% intra-dataset ACER in most cases, while cross-dataset ACER is substantially higher. LoRA primarily refines the decision boundary within a dataset, suggesting that pretrained representations and the adaptation dataset play a larger role in cross-dataset generalization than the evaluated lightweight adaptation strategy.