Adaptive Incentive Design in Dynamic Principal-Agent Problem via Kernelized Bandits

Arghya Mallick; Anuj S. Vora; Sergio Grammatico; Peyman Mohajerin Esfahani

cs.MAAug 18, 2026

Adaptive Incentive Design in Dynamic Principal-Agent Problem via Kernelized Bandits

Authors: Arghya Mallick, Anuj S. Vora, Sergio Grammatico, Peyman Mohajerin Esfahani

Organizations: Delft Center for Systems and Control, Delft University of Technology, Delft, The Netherlands · Jio Institute, Navi Mumbai, India · University of Toronto, Toronto, Canada

Abstract

We consider the dynamic principal-agent problem under asymmetric information, wherein a principal sequentially designs contracts to incentivize an agent with unknown preferences and hidden actions. A fundamental bottleneck in the existing literature is the assumption of deterministic agent utility, which renders the principal's expected utility discontinuous and forces computationally intractable discretizations of the contract space. In this paper, we address this limitation by introducing a stochastic counterpart into the agent's utility model, capturing the inherent physical and behavioral variations in realistic subsystems. We formally prove that this stochastic formulation restores the continuity of the principal's expected utility. Leveraging this continuous geometric structure, we formulate the interaction as a structured multi-armed bandit problem subject to heteroscedastic noise. We propose a Heteroscedastic GP-UCB algorithm that utilizes a Neural Network (Arcsin) kernel, chosen to capture the non-stationary, sigmoidal geometry of the utility landscape. For an $m$ -dimensional compact contract space, we establish a high-probability cumulative regret bound of $O\left(\sqrt{T}(\log T)^{m+1}\right)$ . Finally, we demonstrate the practical efficacy of our theoretical framework by formulating the Vehicle-to-Grid (V2G) incentive design problem, proving its equivalence to a dynamic principal-agent problem, and showing superior economic performance for grid aggregators.

Figures & tables

Figure 1. Schematic of principal-agent feedback model ( 1 ).

Panel of Figure 2 — Figure 2. (High-low example 2.1 ) (a) Visualization of ${U}^{\mathrm{PA}}$ . (b) Visualization of the variance of heteroscedastic noise $\sigma^{2}_{\varepsilon}$ . (c) Geometry of $\tilde{U}^{\mathrm{PA}}$ under the Assumption 3.1 .

Panel of Figure 3 — Figure 3. Left: The agent’s utility $U^{\mathrm{A}}(x,a)$ with optimizer marked at $a^{\star}$ . Right: The modified utility $\tilde{U}^{\mathrm{A}}(x,a):=U^{\mathrm{A}}(x,a)+\langle w,a\rangle$ with a realization $w=0.3$ . Angled pointers clearly distinguish the original optimizer $a^{\star}$ from the newly shifted optimizer $\tilde{a}^{\star}$ .

Figure 4. Representative function of $\tilde{U}^{\mathrm{PA}}$ , which is built with Neural Network kernel using kernel regression where the empirical samples are generated from an instance of ‘High-low’ example 2.1 .

Figure 5. V2G incentive design schematic showing sequential interaction between aggregators and EV users.

Agent	Low	Med.	High
Cost $c(a_{i})$	$\mathcal{N}$ $(0,\,0.05)$	$\mathcal{N}$ $(0.1,\,0.05)$	$\mathcal{N}$ $(0.2,\,0.05)$
Production $p(a_{i})$	$[0.95,$ $0.05]$	$[0.2,$ $0.8]$	$[0.05,$ $0.95]$
Principal
Outcome $i$	Low ( $i=0$ )	High ( $i=1$ )
Value $v$	$v_{0}=0.1$	$v_{1}=0.8$

Table 1. Performance comparison of different methods. Effort levels correspond to

a_{1}

(low),

a_{2}

(medium), and

a_{3}

(high).

Panel of Figure 6 — Figure 6. Performance comparison on the synthetic benchmark and V2G profit analysis: (a) Performance comparison of Heteroscedastic GP-UCB on the synthetic benchmark. The y-axis displays the Average Utility Reward ( $U_{T}$ ), not cumulative regret. Therefore, the convergence of our proposed algorithm (green line) to a high, positive horizontal asymptote ( $\approx 0.45$ ) indicates successful exploitation of the optimal contract, outperforming the baselines, which fail to converge to this optimal yield. (b) Comparison of the net V2G profit ( $J_{\text{net}}$ ) for two scenarios, highlighting the impact of V2G incentive schemes under the benchmarked algorithms.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7. Assessing performance of Heteroscedastic GP-UCB (paired with Neural Network kernel) under two scenarios. In scenario $1$ , it outperforms GP-UCB [ 35 ] with a Neural Network kernel, validating the effect of considering heteroscedastic noise. In scenario $2$ , it also outperforms the radial basis function (RBF) kernel, validating our kernel choice.

Table 2. Details of uncertain and fixed parameters for numerical simulation.

Explore similar work

A No-Regret Framework for Adaptive Incentive Design
Jun 1, 2026Georgios Vasileiou, Lantian Zhang, Silun ZhangMechanism Design Game Theory
Department of Mathematics, KTH Royal Institute of Technology, Stockholm, 10044, Sweden.
Adaptive Incentive Design with Regret Minimization
Apr 7, 2026Georgios Vasileiou, Lantian Zhang, Silun ZhangRegret Minimization Mechanism Design
Dept. of Mathematics at the KTH Royal Institute of Technology, Stockholm, 10044, Sweden.
Incentive Design without Hypergradients: A Social-Gradient Method
Apr 13, 2026Georgios Vasileiou, Lantian Zhang, Silun ZhangMulti-Agent System Optimization Mechanism Design
Department of Mathematics at the KTH Royal Institute of Technology, Stockholm, 10044, Sweden.

Jun 1, 2026J/K move · Enter open · S save

A No-Regret Framework for Adaptive Incentive Design

Georgios Vasileiou, Lantian Zhang, Silun Zhang

Department of Mathematics, KTH Royal Institute of Technology, Stockholm, 10044, Sweden.

Mechanism Design Game Theory Regret Minimization

Open paper AskPDF arXiv

Incentive design studies how a central authority can influence strategic agents through payments, subsidies, or taxes, so that individual objectives align with collective welfare. This paper introduces a No-Regret Adaptive Incentive Design (RAID) framework for nonlinear games with continuous action spaces and private agent costs. In this framework, the authority (planner) designs incentives that regulate the Nash equilibrium toward a socially optimal action profile, while simultaneously learning agents' unknown preferences from repeated strategic responses. We formulate the RAID problem and construct a least-squares estimator whose strong consistency requires only diminishing excitation. Leveraging this weak excitation requirement, we propose a switching incentive policy that alternates between probing (exploration) and estimate-based (exploitation) incentives. The resulting policy achieves an

O(t^{-0.5})

parameter estimation rate and accumulates

O(t^{0.5}\log t)

squared social-cost regret, almost surely. We further extend the framework to an endogenous-noise response model, where standard least-squares estimation is biased due to an error-in-variables correlation between the noise and agent responses. We utilize a repeated-sampling estimator and corresponding switching policy that retain the same almost-sure convergence and regret rates. Numerical experiments validate the effectiveness and predicted convergence rates of the method.

Abstract

Explore similar work

A No-Regret Framework for Adaptive Incentive Design

Adaptive Incentive Design with Regret Minimization

Incentive Design without Hypergradients: A Social-Gradient Method