Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

Oleg Miroshnichenko

cs.LGAug 22, 2026

Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

Authors: Oleg Miroshnichenko

Abstract

We introduce Gated Decoupled Compositional Bandits (GDCB), a family of contextual bandit algorithms with three structural innovations that jointly fall outside the taxonomy of LinUCB, LinTS, HierTS, factored bandits, neural contextual bandits, and RLHF. In a GDCB system: (i) the action delivered to the environment is the composition of a nominal arm, drawn by a discrete or hierarchical bandit, with a context-dependent scaler; (ii) the scaler parameter is learned in a separate supervised loop, not jointly with arm selection; and (iii) every action passes through a pre-execution gate that may modify or veto the composed action before it reaches the environment. We formalise this class of algorithms, prove four structural theorems characterising its statistical behaviour, and show that six industrially significant systems -- short-term rental dynamic pricing, clinical drug dosing, credit origination, grid demand response, content moderation, and LLM tool-use agents -- are all instances of GDCB, differing only in the composition operator, scaler family, and gate. The central result is the Decoupling Variance Reduction theorem: a well-calibrated scaler removes context-induced variance from the arm-to-reward mapping, turning a non-stationary bandit problem into an approximately stationary one. The Gate-Induced Equivalence theorem shows that under a stationary gate, historical data collected under any prior policy is a valid warm-up initialiser without importance-sampling correction, generalising the companion P-HITL result (arXiv:2606.02595) from human approval to arbitrary gates. In regulated, high-stakes domains, constraints usually treated as deployment frictions -- approval gates, compliance rules, safety shields -- are the mechanism that makes fast deployment possible, not an obstacle to it. The companion paper validates instance 1 (STR dynamic pricing) on real production data.

Figures & tables

Figure 1: The three-component GDCB architecture. A nominal-arm bandit samples

a^{\text{nom}}_{t}

from a posterior (HierTS / Factored / Beta-TS); a supervised scaler module emits

\delta_{\theta}(\mathbf{x}_{t})

(Ridge / GP / Neural); the composition operator

\Phi

combines them into a recommended executable

e^{\text{rec}}_{t}

; the pre-execution gate

g

(HITL / safety / compliance) produces the final executed action

e^{\text{exec}}_{t}

that reaches the environment. The outcome

y_{t}

is the only shared signal — bandit and supervised module update disjoint parameter sets (bandit loop in blue; supervised loop in orange).

P-HITL result (instance #1)	Generic GDCB result (this paper)	Specialisation
Dual cold-start algorithm	Theorem 4.8 (Sample-Complexity Lifting)	$\Phi=\times$ , $\delta_{\theta}$ = day-signal, $g$ = human approval
HITL structural equivalence	Theorem 4.3 (Gate-Induced Equivalence)	$g$ = human approval, stationary operator
25-arm factored bandit + day-signal	Corollary 4.2 (from Thm. 4.1 )	$\Phi=\times$ , $\varepsilon$ bounded by ridge error
Cross-domain applicability survey	§ 5 and § 5.3	All six instantiations

Table 1: Mapping from P-HITL private-case results to generic GDCB structural theorems.

Figure 2: Theorem 4.1 : decoupling variance reduction. Left (GDCB, perfect scaler

\varepsilon=0

) : arm value estimates are stationary in context

x

— the calibrated scaler absorbs all context variation, so the bandit sees near-constant arm gaps and concentrates posteriors rapidly. Right (LinTS, no decoupled scaler) : residual context-variance inflates arm-value uncertainty, slowing posterior concentration. The gap in convergence rate directly explains the 150

\to

30 episode cold-start compression observed in the P-HITL instantiation.

Figure 3: Theorem 4.3 empirical validation (P-HITL instantiation, real STR data). Left : recommended arm distributions under the historical policy and the live bandit differ — IS correction would be required in standard OPE. Right : executed arm distributions (post-gate approval) are statistically indistinguishable (KS

p=0.000

indicating distributional match), confirming Eq. ( 3 ) on real production data and validating no-IS warm-up.

Panel of Figure 4 — Figure 4: Regret decomposition across GDCB instances (schematic). Theorem 4.7 decomposes regret into three additive components: arm-selection regret $R_{\text{bandit}}$ (blue), scaler calibration regret $R_{\text{cal}}$ (orange), and gate-misalignment regret $R_{\text{gate}}$ (red). The dominant component varies by domain: STR pricing is dominated by calibration (rich historical data reduces $R_{\text{bandit}}$ ; gate overrides drive $R_{\text{gate}}$ ); clinical systems require more arm exploration; moderation systems with experienced moderators have low gate misalignment.

#	Domain	$\mathcal{A}$ (nominal)	$\Phi$	$g$	Reward
1	STR pricing	5 $\times$ 5 multiplier grid	multiplication	human approval	booked $\times$ price
2	Clinical dosing	5–7 dose levels	addition (offset)	physician + safety bounds	recovery $-$ adverse
3	Credit origination	5 $\times$ 3 (rate $\times$ tenor)	multiplication	compliance cap	repayment $\times$ net interest
4	Grid demand resp.	5 shed magnitudes	addition	grid stability constraint	stability $-$ interrupt cost
5	Content moderation	3 actions (allow/warn/remove)	identity (severity as scale)	moderator + appeal filter	satisfaction $-$ violation rate
6	LLM tool use	$k$ tool choices	compositional prompt	safety policy filter	task success $\times$ latency

Table 2: GDCB canonical instantiations across six domains.

Panel of Figure 5 — Figure 5: GDCB taxonomy. Existing algorithms occupy the left/bottom regions (low gate and/or scaler complexity). The GDCB region (blue) is the under-explored space where composition, supervised scaler, and pre-execution gate are all non-trivial. Six instances populate distinct cells, confirming empirical breadth across the cross-product.

Figure 6: Six industrial systems as GDCB instantiations. Each row specifies the nominal arm space, composition operator

\Phi

, supervised scaler

\delta_{\theta}

, pre-execution gate

g

, and reward signal

r

for one domain. All six satisfy the three GDCB assumptions; the variety of

\Phi

(multiplication, addition, identity, prompt-composition) and

g

(human, safety bound, compliance rule, retrieval prior) confirms the framework is not STR-specific.

#	Domain	Thm. 4.1 (Var. Red.)	Thm. 4.3 (Gate Equiv.)	Thm. 4.7 (Regret)	Thm. 4.8 (Lifting)
1	STR pricing	Day-signal removes cluster-occ. variance; 20–30% cold-start compression	Historical STR logs valid warm-up; empirically confirmed ( Miroshnichenko 2026b )	$R_{\text{cal}}$ dominates; $R_{\text{gate}}$ from operator overrides	1 461 hist. episodes serve both loops simultaneously
2	Clinical dosing	Dose offset absorbs patient-feature variance; dose-response becomes $x$ -invariant	EHR valid warm-up under stationary physician approval + safety bounds	$R_{\text{bandit}}$ dominates (dose exploration is expensive)	Historical EHR seeds both dose posteriors and GP scaler
3	Credit origination	Risk spread removes credit-score variance from rate-to-default mapping	Historical loan applications valid under stationary compliance function	$R_{\text{gate}}$ from regulatory cap forcing suboptimal rates	Historical loan book seeds both loops
4	Grid demand resp.	Demand-feature offset stabilises shed-to-stability mapping	Any shielded historical policy valid (grid constraint is physics-stationary)	$R_{\text{cal}}$ dominates (sparse stability feedback)	Historical SCADA logs seed both loops
5	Moderation	Severity score reduces post-content variance in action-to-satisfaction mapping	Historical moderation logs valid under stationary moderator + appeal filter	$R_{\text{gate}}$ from policy drift (moderator turnover risk)	Historical moderation queue seeds both loops
6	LLM tool use	Retrieval scaler reduces prompt-to-tool-success variance	Historical tool-use logs valid under stationary safety policy	$R_{\text{bandit}}$ dominates (novel tool space requires exploration)	Historical API logs seed both loops

Table 3: Specialisation of GDCB structural theorems across the six canonical instances.

Explore similar work

Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits
Apr 16, 2026Maksim Pershin, Ivan Golovanov, Pavel Baltabaev +1Contextual Bandits Linear Contextual Bandits
Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning
May 22, 2026Oleg MiroshnichenkoContextual Bandits Dynamic Pricing
Constrained Contextual Bandits with Adversarial Contexts
May 7, 2026Dhruv Sarkar, Abhishek SinhaContextual Bandits Adversarial Bandits
Indian Institute of Technology Kharagpur, India · Tata Institute of Fundamental Research, Mumbai, India

Apr 16, 2026J/K move · Enter open · S save

Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits

Maksim Pershin, Ivan Golovanov, Pavel Baltabaev, Natalia Trankova

Contextual Bandits Linear Contextual Bandits

Open paper AskPDF arXiv

Contextual bandit algorithms suffer from high regret during cold-start, when the learner has insufficient data to distinguish good arms from bad. We propose augmenting Disjoint LinUCB with LLM pseudo-observations: after each round, a large language model predicts counterfactual rewards for the unplayed arms, and these predictions are injected into the learner as weighted pseudo-observations. The injection weight is controlled by a calibration-gated decay schedule that tracks the LLM's prediction accuracy on played arms via an exponential moving average; high calibration error suppresses the LLM's influence, while accurate predictions receive higher weight during the critical early rounds. We evaluate on two contextual bandit environments - UCI Mushroom (2-arm, asymmetric rewards) and MIND-small (5-arm news recommendation) - and find that when equipped with a task-specific prompt, LLM pseudo-observations reduce cumulative regret by 19% on MIND relative to pure LinUCB. However, generic counterfactual prompt framing increases regret on both environments, demonstrating that prompt design is the dominant factor, more important than the choice of decay schedule or calibration gating parameters. We analyze the failure modes of calibration gating on domains with small prediction errors and provide a theoretical motivation for the bias-variance trade-off governing pseudo-observation weight.

Abstract

Explore similar work

Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits

Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning

Constrained Contextual Bandits with Adversarial Contexts