Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

Oleg Miroshnichenko

cs.LGAug 22, 2026

Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

Authors: Oleg Miroshnichenko

Abstract

We introduce Gated Decoupled Compositional Bandits (GDCB), a family of contextual bandit algorithms with three structural innovations that jointly fall outside the taxonomy of LinUCB, LinTS, HierTS, factored bandits, neural contextual bandits, and RLHF. In a GDCB system: (i) the action delivered to the environment is the composition of a nominal arm, drawn by a discrete or hierarchical bandit, with a context-dependent scaler; (ii) the scaler parameter is learned in a separate supervised loop, not jointly with arm selection; and (iii) every action passes through a pre-execution gate that may modify or veto the composed action before it reaches the environment. We formalise this class of algorithms, prove four structural theorems characterising its statistical behaviour, and show that six industrially significant systems -- short-term rental dynamic pricing, clinical drug dosing, credit origination, grid demand response, content moderation, and LLM tool-use agents -- are all instances of GDCB, differing only in the composition operator, scaler family, and gate. The central result is the Decoupling Variance Reduction theorem: a well-calibrated scaler removes context-induced variance from the arm-to-reward mapping, turning a non-stationary bandit problem into an approximately stationary one. The Gate-Induced Equivalence theorem shows that under a stationary gate, historical data collected under any prior policy is a valid warm-up initialiser without importance-sampling correction, generalising the companion P-HITL result (arXiv:2606.02595) from human approval to arbitrary gates. In regulated, high-stakes domains, constraints usually treated as deployment frictions -- approval gates, compliance rules, safety shields -- are the mechanism that makes fast deployment possible, not an obstacle to it. The companion paper validates instance 1 (STR dynamic pricing) on real production data.

Figures & tables

Figure 1: The three-component GDCB architecture. A nominal-arm bandit samples

a^{\text{nom}}_{t}

from a posterior (HierTS / Factored / Beta-TS); a supervised scaler module emits

\delta_{\theta}(\mathbf{x}_{t})

(Ridge / GP / Neural); the composition operator

\Phi

combines them into a recommended executable

e^{\text{rec}}_{t}

; the pre-execution gate

g

(HITL / safety / compliance) produces the final executed action

e^{\text{exec}}_{t}

that reaches the environment. The outcome

y_{t}

is the only shared signal — bandit and supervised module update disjoint parameter sets (bandit loop in blue; supervised loop in orange).

P-HITL result (instance #1)	Generic GDCB result (this paper)	Specialisation
Dual cold-start algorithm	Theorem 4.8 (Sample-Complexity Lifting)	$\Phi=\times$ , $\delta_{\theta}$ = day-signal, $g$ = human approval
HITL structural equivalence	Theorem 4.3 (Gate-Induced Equivalence)	$g$ = human approval, stationary operator
25-arm factored bandit + day-signal	Corollary 4.2 (from Thm. 4.1 )	$\Phi=\times$ , $\varepsilon$ bounded by ridge error
Cross-domain applicability survey	§ 5 and § 5.3	All six instantiations

Table 1: Mapping from P-HITL private-case results to generic GDCB structural theorems.

Figure 2: Theorem 4.1 : decoupling variance reduction. Left (GDCB, perfect scaler

\varepsilon=0

) : arm value estimates are stationary in context

x

— the calibrated scaler absorbs all context variation, so the bandit sees near-constant arm gaps and concentrates posteriors rapidly. Right (LinTS, no decoupled scaler) : residual context-variance inflates arm-value uncertainty, slowing posterior concentration. The gap in convergence rate directly explains the 150

\to

30 episode cold-start compression observed in the P-HITL instantiation.

Figure 3: Theorem 4.3 empirical validation (P-HITL instantiation, real STR data). Left : recommended arm distributions under the historical policy and the live bandit differ — IS correction would be required in standard OPE. Right : executed arm distributions (post-gate approval) are statistically indistinguishable (KS

p=0.000

indicating distributional match), confirming Eq. ( 3 ) on real production data and validating no-IS warm-up.

Panel of Figure 4 — Figure 4: Regret decomposition across GDCB instances (schematic). Theorem 4.7 decomposes regret into three additive components: arm-selection regret $R_{\text{bandit}}$ (blue), scaler calibration regret $R_{\text{cal}}$ (orange), and gate-misalignment regret $R_{\text{gate}}$ (red). The dominant component varies by domain: STR pricing is dominated by calibration (rich historical data reduces $R_{\text{bandit}}$ ; gate overrides drive $R_{\text{gate}}$ ); clinical systems require more arm exploration; moderation systems with experienced moderators have low gate misalignment.

#	Domain	$\mathcal{A}$ (nominal)	$\Phi$	$g$	Reward
1	STR pricing	5 $\times$ 5 multiplier grid	multiplication	human approval	booked $\times$ price
2	Clinical dosing	5–7 dose levels	addition (offset)	physician + safety bounds	recovery $-$ adverse
3	Credit origination	5 $\times$ 3 (rate $\times$ tenor)	multiplication	compliance cap	repayment $\times$ net interest
4	Grid demand resp.	5 shed magnitudes	addition	grid stability constraint	stability $-$ interrupt cost
5	Content moderation	3 actions (allow/warn/remove)	identity (severity as scale)	moderator + appeal filter	satisfaction $-$ violation rate
6	LLM tool use	$k$ tool choices	compositional prompt	safety policy filter	task success $\times$ latency

Table 2: GDCB canonical instantiations across six domains.

Panel of Figure 5 — Figure 5: GDCB taxonomy. Existing algorithms occupy the left/bottom regions (low gate and/or scaler complexity). The GDCB region (blue) is the under-explored space where composition, supervised scaler, and pre-execution gate are all non-trivial. Six instances populate distinct cells, confirming empirical breadth across the cross-product.

Figure 6: Six industrial systems as GDCB instantiations. Each row specifies the nominal arm space, composition operator

\Phi

, supervised scaler

\delta_{\theta}

, pre-execution gate

g

, and reward signal

r

for one domain. All six satisfy the three GDCB assumptions; the variety of

\Phi

(multiplication, addition, identity, prompt-composition) and

g

(human, safety bound, compliance rule, retrieval prior) confirms the framework is not STR-specific.

#	Domain	Thm. 4.1 (Var. Red.)	Thm. 4.3 (Gate Equiv.)	Thm. 4.7 (Regret)	Thm. 4.8 (Lifting)
1	STR pricing	Day-signal removes cluster-occ. variance; 20–30% cold-start compression	Historical STR logs valid warm-up; empirically confirmed ( Miroshnichenko 2026b )	$R_{\text{cal}}$ dominates; $R_{\text{gate}}$ from operator overrides	1 461 hist. episodes serve both loops simultaneously
2	Clinical dosing	Dose offset absorbs patient-feature variance; dose-response becomes $x$ -invariant	EHR valid warm-up under stationary physician approval + safety bounds	$R_{\text{bandit}}$ dominates (dose exploration is expensive)	Historical EHR seeds both dose posteriors and GP scaler
3	Credit origination	Risk spread removes credit-score variance from rate-to-default mapping	Historical loan applications valid under stationary compliance function	$R_{\text{gate}}$ from regulatory cap forcing suboptimal rates	Historical loan book seeds both loops
4	Grid demand resp.	Demand-feature offset stabilises shed-to-stability mapping	Any shielded historical policy valid (grid constraint is physics-stationary)	$R_{\text{cal}}$ dominates (sparse stability feedback)	Historical SCADA logs seed both loops
5	Moderation	Severity score reduces post-content variance in action-to-satisfaction mapping	Historical moderation logs valid under stationary moderator + appeal filter	$R_{\text{gate}}$ from policy drift (moderator turnover risk)	Historical moderation queue seeds both loops
6	LLM tool use	Retrieval scaler reduces prompt-to-tool-success variance	Historical tool-use logs valid under stationary safety policy	$R_{\text{bandit}}$ dominates (novel tool space requires exploration)	Historical API logs seed both loops

Table 3: Specialisation of GDCB structural theorems across the six canonical instances.

Explore similar work

Apr 16, 2026

Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits

Contextual bandit algorithms suffer from high regret during cold-start, when the learner has insufficient data to distinguish good arms from bad. We propose augmenting Disjoint LinUCB with LLM pseudo-observations: after each round, a large language model predicts counterfactual rewards for the unplayed arms, and these predictions are injected into the learner as weighted pseudo-observations. The injection weight is controlled by a calibration-gated decay schedule that tracks the LLM's prediction accuracy on played arms via an exponential moving average; high calibration error suppresses the LLM's influence, while accurate predictions receive higher weight during the critical early rounds. We evaluate on two contextual bandit environments - UCI Mushroom (2-arm, asymmetric rewards) and MIND-small (5-arm news recommendation) - and find that when equipped with a task-specific prompt, LLM pseudo-observations reduce cumulative regret by 19% on MIND relative to pure LinUCB. However, generic counterfactual prompt framing increases regret on both environments, demonstrating that prompt design is the dominant factor, more important than the choice of decay schedule or calibration gating parameters. We analyze the failure modes of calibration gating on domains with small prediction errors and provide a theoretical motivation for the bias-variance trade-off governing pseudo-observation weight.

Maksim Pershin, Ivan Golovanov, Pavel Baltabaev +1

Contextual Bandits Linear Contextual Bandits

Ask PDF

May 22, 2026

Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning

Dynamic pricing in short-term rental (STR) markets presents a distinctive challenge for online learning algorithms: pricing decisions carry significant financial risk, operators require explainability, and market feedback is sparse (one booking outcome per listed night). We introduce the Human-in-the-Loop Gated Bandit (HITL-GB) framework, in which a contextual bandit algorithm generates price recommendations but a human agent retains authority to accept, modify, or reject each recommendation before it is applied. We show that under this approval constraint, historical pricing data -- collected under a prior deterministic policy -- is structurally equivalent to on-policy warm-up data for initialising the bandit's posterior, bypassing the weeks-to-months cold-start period that renders pure online bandit learning impractical in sparse-feedback markets. We formalise the approval-gated reward signal, derive a regularised ridge-regression warm-up procedure from historical episodes, and validate the approach on real STR production data (anonymised urban market, 2 rooms, April 2022 -- April 2026, 1,461 nightly pricing episodes). Our warm-up procedure compresses effective cold-start from ~150 episodes to ~30 episodes when initialising agents from the Hierarchical Factored Thompson Sampling (HF-TS) family. We further argue that the structural equivalence result is domain-agnostic: any high-stakes domain where human approval is legally or operationally required -- including clinical drug dosing, credit origination, content moderation, and radiological diagnosis -- satisfies the same conditions and benefits from the same warm-up strategy. In regulated industries, mandatory human oversight is thus a statistical asset rather than a deployment constraint.

Oleg Miroshnichenko

Contextual Bandits Dynamic Pricing

Ask PDF

May 7, 2026

Constrained Contextual Bandits with Adversarial Contexts

We study constrained contextual bandits with adversarial contexts, where each action yields a random reward and incurs a random cost. The conditional mean reward and cost belong to known function classes and are learned through online regression oracles. The benchmark is a stationary randomized policy whose expected cost is non-positive at every observed context. We extend the seminal

\mathsf{SquareCB}

algorithm of Foster and Rakhlin (2020), which reduces the contextual bandits to a regression problem, to the constrained setting. A surrogate reward function is constructed which penalizes the positive part of the expected cost of an action distribution, and an auxiliary variable accumulates its predicted value. Then we use an exponential Lyapunov function to control both regret and cumulative constraint violation via a single inequality. If the cumulative squared prediction error is at most

U_T

, our algorithm achieves

O(\sqrt{KTU_T})

regret and

O(\sqrt{KTU_T}\log T)

cumulative expected cost, with high probability, improving upon the previous

O(T^{3/4})

type bounds obtained in stochastic contextual settings with an unknown context distribution. Our results require neither additional margin conditions nor probabilistic assumptions on the context sequence.

Dhruv Sarkar, Abhishek Sinha

Indian Institute of Technology Kharagpur, India · Tata Institute of Fundamental Research, Mumbai, India

Contextual Bandits Adversarial Bandits

Ask PDF