Stein Variational Gradient Descent

Also known as SVGD

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 11

All topics

Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent
Sep 8, 2026Quang-Duy Tran, Trung Le, Bao Duong +2Stiefel Manifold Optimization Uncertainty Quantification
Deakin Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia · Department of Computer Science, Aalto University, Espoo, Finland · Department of Data Science & AI, Monash University, Melbourne, Australia
Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits
Jul 16, 2026Trevor Teolis, Maarten V. de HoopKernel Methods Stein Variational Gradient Descent
Rice University
Reinforcement Learning for Flow-Matching Policies with Density Transport
Jun 7, 2026Boshu Lei, Kostas Daniilidis, Antonio LoquercioFlow Matching Robotic RL
University of Pennsylvania · Philadelphia, PA 19103
Variational Proximal Policy Optimization
Jun 6, 2026Ousmane Amadou DiaRL for Language Model Reasoning Proximal Policy Optimization
Stein Kernelized Molecular Dynamics for Active Learning of Interatomic Potentials
Jun 2, 2026Joanna Zou, Fraser Birks, Dallas Foster +1Machine Learning Interatomic Potentials Active Learning
Center for Computational Science & Engineering, Schwarzman College of Computing, MIT · Warwick Centre for Predictive Modelling, School of Engineering, University of Warwick · NVIDIA
Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference
May 26, 2026Jaewoo Lee, Hyeongyu Kang, Dohyun Kim +9Stein Variational Gradient Descent
1KAIST · 2MongooseAI · 3Mila – Quebec AI Institute +3
Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow
May 10, 2026Lénaïc Chizat, Maria Colombo, Roberto Colombo +1Dynamical Systems Optimization Convergence Analysis
From Particles to Perils: SVGD-Based Hazardous Scenario Generation for Autonomous Driving Systems Testing
Apr 20, 2026Linfeng Liang, Xiao Cheng, Tsong Yueh Chen +1Autonomous Driving Safety Evaluation Autonomous Driving Simulation
Macquarie University, Australia · Swinburne University of Technology, Australia
Stein Variational Black-Box Combinatorial Optimization
Apr 17, 2026Thomas Landais, Olivier Goudet, Adrien Goëffon +2Stochastic Optimization Black-Box Optimization
LERIA, Université d’Angers, 2 Boulevard Lavoisier, Angers 49045, France
Stein-based Optimization of Sampling Distributions in Model Predictive Path Integral Control
Nov 3, 2025Jace Aldrich, Odest Chadwicke JenkinsRobotic Control Stochastic Optimal Control
Laboratory for Perceptive RObotics and Grounded REasoning SystemS and Robotics Department at the University of Michigan, Ann Arbor, USA
Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent
Sep 13, 2024Sayan Banerjee, Krishnakumar Balasubramanian, Promit GhosalKernel Methods Variational Inference
Department of Statistics and Operations Research, University of North Carolina, Chapel Hill. · Department of Statistics, University of California, Davis. · Department of Statistics, University of Chicago.

Sep 8, 2026, cs.LGJ/K move · Enter open · S save

Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent

Quang-Duy Tran, Trung Le, Bao Duong, Phuoc Nguyen+1

Deakin Applied Artificial Intelligence Initiative, Deakin University, Geelong, Australia · Department of Computer Science, Aalto University, Espoo, Finland · Department of Data Science & AI, Monash University, Melbourne, Australia

Stiefel Manifold Optimization Uncertainty Quantification Model Calibration Bayesian LoRA Stein Variational Gradient Descent

Open paper AskPDF arXiv

Several geometry-aware approaches to low-rank adaptation have emerged for parameter-efficient fine-tuning of large pre-trained models. These methods aim to take full advantage of the geometric structure of low-rank manifolds for improving the efficiency in subspace utilization and reducing redundancy by enforcing orthogonality constraints during optimization. The strong empirical results of these techniques have motivated further study into whether predictions from such geometry-based adaptation methods could be overconfident. In this paper, we build on the singular value decomposition factorization of adapters to develop a framework based on Stein variational gradient descent (SVGD). In this formulation, the low-rank matrices are transported along the Stiefel manifold to match the targeted distributions while retaining their crucial geometric structure. Since this geometry-aware SVGD approach provides multiple solutions during inference, it supports uncertainty quantification and produces better-calibrated adapters on the Stiefel manifold. Extensive experiments show that our method delivers strong model calibration and attains higher prediction accuracy than SVGD and related uncertainty estimation methods that are formulated in Euclidean space.

Momentum

Related topics

Latest papers 11