Bayesian LoRA

LoRA: Low-Rank Adaptation

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 14

All topics

Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent
Sep 8, 2026Quang-Duy Tran, Trung Le, Bao Duong +2Stiefel Manifold Optimization Uncertainty Quantification
Deakin Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia · Department of Computer Science, Aalto University, Espoo, Finland · Department of Data Science & AI, Monash University, Melbourne, Australia
Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization
Aug 1, 2026Xuefei Wang, Jun Han, Zixuan Wang +4Few-Shot Domain Adaptation LLM Personalization
Beihang University · Nankai University
Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation
Jul 2, 2026Jijie Zhang, Zhe Ren, Quan Zhang +1Low-Rank Adaptation Language Model Calibration
School of Artificial Intelligence, Jilin University · Michigan State University
BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning
Jun 28, 2026Zhibin Duan, Yuhong Wang, Jiahong Fu +3Bayesian Neural Networks Low-Rank Adaptation
National Key Lab of Radar Signal Processing, Xidian University, Xi’an, Shaanxi 710071, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, 710049, China
Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models
Jun 20, 2026Colin Samplawski, Ramneet Kaur, Manoj Acharya +2Distribution Shift Robustness Multimodal Large Language Models
Neuro-Symbolic Computing and Intelligence Research Group, Computer Science Laboratory, SRI International
iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis
May 28, 2026Yang Song, Yixuan Zhang, Lingfa Meng +5Graph Structure Learning Medical Diagnosis
Section of Health Data Science & AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · University of Copenhagen, Copenhagen, Denmark · Rutgers University, New Brunswick, NJ, USA +1
On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
May 28, 2026Daniel Dold, Emanuel Sommer, Julius Kobialka +2Mode Connectivity Bayesian Inference
HTWG Konstanz · LMU Munich · Munich Center for Machine Learning (MCML)
AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation
May 9, 2026Ziyun Liu, Fengmiao Bian, Jian-Feng CaiLow-Rank Adaptation Adaptive Gradient Methods
Department of Mathematics The Hong Kong University of Science and Technology
Bayesian Fine-tuning in Projected Subspaces
May 8, 2026Viktar Dubovik, Patryk Marszałek, Jacek Tabor +1Bayesian Neural Networks Fine-Tuning
Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
Flexi-LoRA with Input-Adaptive Ranks: Efficient Finetuning for Speech and Reasoning Tasks
May 3, 2026Zongqian Li, Yixuan Su, Han Zhou +2Low-Rank Adaptation Bayesian LoRA
University of Cambridge
BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models
Apr 27, 2026Dario Coscia, Sindy Löwe, Max WellingBayesian Neural Networks Uncertainty Quantification
mathLab, SISSA · CuspAI
Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning
Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank Adaptation Uncertainty-Aware Multimodal Fusion
Dalhousie University, Halifax NS, Canada
Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation
Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank Adaptation Bayesian LoRA
Dalhousie University, Halifax NS, Canada
Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural Networks Confidence Estimation in Language Models
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland

Sep 8, 2026, cs.LGJ/K move · Enter open · S save

Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent

Quang-Duy Tran, Trung Le, Bao Duong, Phuoc Nguyen+1

Deakin Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia · Department of Computer Science, Aalto University, Espoo, Finland · Department of Data Science & AI, Monash University, Melbourne, Australia

Stiefel Manifold Optimization Uncertainty Quantification Model Calibration Bayesian LoRA Stein Variational Gradient Descent

Open paper AskPDF arXiv

Several geometry-aware approaches to low-rank adaptation have emerged for parameter-efficient fine-tuning of large pre-trained models. These methods aim to take full advantage of the geometric structure of low-rank manifolds for improving the efficiency in subspace utilization and reducing redundancy by enforcing orthogonality constraints during optimization. The strong empirical results of these techniques have motivated further study into whether predictions from such geometry-based adaptation methods could be overconfident. In this paper, we build on the singular value decomposition factorization of adapters to develop a framework based on Stein variational gradient descent (SVGD). In this formulation, the low-rank matrices are transported along the Stiefel manifold to match the targeted distributions while retaining their crucial geometric structure. Since this geometry-aware SVGD approach provides multiple solutions during inference, it supports uncertainty quantification and produces better-calibrated adapters on the Stiefel manifold. Extensive experiments show that our method delivers strong model calibration and attains higher prediction accuracy than SVGD and related uncertainty estimation methods that are formulated in Euclidean space.

Momentum

Related topics

Latest papers 14