stat.MLNov 7, 2026

Bayesian transfer for offline Q-learning with multi-task Gaussian processes

Authors: Wenxing Guo

Organizations: School of Mathematics, Statistics and Actuarial Science, University of Essex, Colchester, United Kingdom

Abstract

Offline reinforcement learning can benefit from data collected in related tasks, yet indiscriminate knowledge transfer may lead to negative transfer under task mismatch and limited target-data coverage. We propose a target-aware Bayesian transfer framework for offline Q-learning based on multi-task Gaussian processes that jointly model task-specific Q-functions. Source tasks are assessed using target predictive evidence and maximum mean discrepancy (MMD), while posterior uncertainty and behavior-supported action sets guide conservative Bellman updates and policy extraction. To account for the dependence induced by Bellman bootstrapping, we distinguish Bellman innovations from bootstrap errors and establish stability and predictive source-selection results under explicit regularity conditions. We further derive selection-adaptive posterior contraction and offline policy-performance bounds, identifying the structural conditions under which source information can be pooled to achieve minimax-optimal rates in a shared-function Gaussian sequence experiment. Simulation studies with heterogeneous source tasks demonstrate improved target Q-function estimation and reduced material negative transfer compared with indiscriminate source pooling. An application to human decision-making data further shows higher average expected immediate payoff than both target-only learning and matched-size random source selection. These results highlight the benefits of combining target-aware source selection with uncertainty-aware conservative learning for reliable and sample-efficient offline reinforcement learning.

Figures & tables

Explore similar work

CardsList
  1. Direct Knowledge Transfer for Offline Q-Learning

    Nov 1, 2026Zhanye Luo, Guangyan Gan, Lin William Congfitted q-iterationTransfer Learning in RL

  2. Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach

    May 27, 2026Guang-Yuan Hao, Lars van der Laan, Aurélien Bibaut +1Reinforcement LearningMinimax Estimation

  3. Transfer Learning in Bandits from Unknown Adaptive Policies: Problem-Dependent Lower and Upper Bounds

    Oct 15, 2026Adrien Prevost, Timothee Mathieu, Waris Radji +1Multi-Armed BanditsTransfer Learning