Bayesian LoRA

LoRA: Low-Rank Adaptation

Ask about this topic

Overview

Bayesian LoRA studies how to represent uncertainty in the low-rank updates used to adapt large models, aiming for reliable predictions without full fine-tuning. Approaches include Gaussian or variational posteriors over adapter factors, sparse and adaptive-rank gating, projected-subspace inference, and geometry-aware priors on Stiefel manifolds; some also condition updates on inputs or multimodal context. Evaluations span language understanding and reasoning, domain shift, selective prediction, multimodal tasks, and personalization, with calibration often improving while task accuracy remains competitive. A central tension is how to scale uncertainty estimation and obtain trustworthy guarantees without sacrificing accuracy, compute efficiency, or reliable performance under distribution shift.

Latest papers

CardsList

18 papers0 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Prior Predictive Matching for Bayesian LoRA

    Oct 26, 2026Tomasz Kuśmierczyk, Eliezer de Souza da Silva, Marcin SenderaBayesian LoRABayesian Neural Networks

  2. Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancing

    Sep 26, 2026Joyanta Jyoti Mondal, Ibne Farabi ShihabBayesian LoRAPAC-Bayesian Generalization Bounds

    Department of Computer and Information Sciences, University of Delaware, USA · Department of Computer Science, Iowa State University, USA

  3. Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent

    Sep 8, 2026Quang-Duy Tran, Trung Le, Bao Duong +2Stiefel Manifold OptimizationUncertainty Quantification

    Deakin Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia · Department of Computer Science, Aalto University, Espoo, Finland · Department of Data Science & AI, Monash University, Melbourne, Australia

  4. Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization

    Aug 1, 2026Xuefei Wang, Jun Han, Zixuan Wang +4Few-Shot Domain AdaptationLLM Personalization

    Beihang University · Nankai University

  5. Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation

    Jul 2, 2026Jijie Zhang, Zhe Ren, Quan Zhang +1Low-Rank AdaptationLanguage Model Calibration

    School of Artificial Intelligence, Jilin University · Michigan State University

  6. BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

    Jun 28, 2026Zhibin Duan, Yuhong Wang, Jiahong Fu +3Bayesian Neural NetworksLow-Rank Adaptation

    National Key Lab of Radar Signal Processing, Xidian University, Xi’an, Shaanxi 710071, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, 710049, China

  7. Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models

    Jun 20, 2026Colin Samplawski, Ramneet Kaur, Manoj Acharya +2Distribution Shift RobustnessMultimodal Large Language Models

    Neuro-Symbolic Computing and Intelligence Research Group, Computer Science Laboratory, SRI International

  8. iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis

    May 28, 2026Yang Song, Yixuan Zhang, Lingfa Meng +5Graph Structure LearningMedical Diagnosis

    Section of Health Data Science & AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · University of Copenhagen, Copenhagen, Denmark · Rutgers University, New Brunswick, NJ, USA +1

  9. On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference

    May 28, 2026Daniel Dold, Emanuel Sommer, Julius Kobialka +2Mode ConnectivityBayesian Inference

    HTWG Konstanz · LMU Munich · Munich Center for Machine Learning (MCML)

  10. AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation

    May 9, 2026Ziyun Liu, Fengmiao Bian, Jian-Feng CaiLow-Rank AdaptationAdaptive Gradient Methods

    Department of Mathematics The Hong Kong University of Science and Technology

  11. Bayesian Fine-tuning in Projected Subspaces

    May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Bayesian Neural NetworksFine-Tuning

    Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland

  12. Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks

    May 3, 2026Zongqian Li, Yixuan Su, Han Zhou +2Low-Rank AdaptationBayesian LoRA

    University of Cambridge

  13. BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models

    Apr 27, 2026Dario Coscia, Sindy Löwe, Max WellingBayesian Neural NetworksUncertainty Quantification

    mathLab, SISSA · CuspAI

  14. Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank AdaptationUncertainty-Aware Multimodal Fusion

    Dalhousie University, Halifax NS, Canada

  15. Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank AdaptationBayesian LoRA

    Dalhousie University, Halifax NS, Canada

  16. Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters

    Apr 3, 2026Haotian Xiang, Bingcong Li, Qin LuLLM Uncertainty EstimationBayesian LoRA

    School of Electrical and Computer Engineering, University of Georgia, Athens, GA, USA · Department of Computer Science, ETH Zurich, Zurich, Switzerland

  17. Calibrated Adaptation: Bayesian Stiefel Manifold Priors for Reliable Parameter-Efficient Fine-Tuning

    Feb 19, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaBayesian LoRAParameter-Efficient Fine-Tuning

    Department of Computer Science, Iowa State University · Department of Civil, Construction and Environmental Engineering, Iowa State University

  18. Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

    Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural NetworksConfidence Estimation in Language Models

    School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland