stat.MLMar 10, 2026

What Do We Care About in Bandits with Noncompliance? BRACE: Bandits with Recommendations, Abstention, and Certified Effects

Authors: Nicolás Della Penna

Organizations: Grouplang

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Bandits with noncompliance separate the learner's recommendation from the treatment actually delivered, so the learning target itself must be chosen. A platform may care about recommendation welfare in the current mediated workflow, treatment learning for a future direct-control regime, or anytime-valid uncertainty for one of those targets. These objectives need not agree. We formalize this objective-choice problem, identify the direct-control regime in which recommendation and treatment objectives collapse, and show by example that recommendation welfare can strictly exceed every learner-measurable treatment policy when downstream actors use private information. For finite-context square-IV problems we propose BRACE, a parameter-free phase-doubling algorithm that performs IV inversion only after matrix certification and otherwise returns full-range but honest structural intervals. BRACE delivers simultaneous policy-value validity, fixed-gap identification of the operationally optimal recommendation policy, and fixed-gap identification of the structurally optimal treatment policy under contextual homogeneity and invertibility. We complement the theory with a finite-context empirical benchmark spanning direct control, mediated present-versus-future tradeoffs, weak identification, homogeneity failure, and rectangular overidentification. The experiments show that safety appears as regret on easy problems, as abstention and wide valid intervals under weak identification, as a reason to prefer recommendation welfare under homogeneity failure, and as tighter structural uncertainty when extra instruments are available. For rich contexts, we also derive an orthogonal score whose conditional bias factorizes into compliance-model and outcome-model errors, clarifying what must be stabilized for anytime-valid semiparametric IV inference.

Figures & tables

ScenarioMain lessonRepresentative finding
Direct-control equivalenceWhen the action–treatment gap disappears, REC and TRT collapse.The best policy has value 0.7750.775 under both objectives; ThompsonBounded on REC and 2SLS-epsilon-decay on TRT both recover it.
Easy strong IVSafety is visible mainly as regret when identification is easy.At the default horizon, BRACE-REC regret is 167.25167.25 , BRACE-REC-FAST is 65.3565.35 , and Actual-UCB is 0.950.95 .
Private-signal operational advantageREC can be the correct deployable target even when TRT is inferior by construction.RECERT deploys REC value 1.01.0 , while its parallel TRT estimate remains 0.50.5 and is not deployed.
Workflow redesign / future controlPresent-channel deployment and future direct-control welfare can point in different directions.RECERT deploys REC value 0.690.69 while estimating TRT value 0.900.90 and abstaining on structural deployment.
Weak IV, small gapUnder weak identification, the distinctive safe behavior is abstention rather than low regret.BRACE variants abstain, while unsafe TRT baselines have wrong-non-abstain rates from about 0.100.10 to 0.700.70 .
Homogeneity failureREC remains robust, but TRT should be weakened or withheld.RECERT keeps deployed REC value 0.950.95 ; 2SLS-style TRT rules deploy the wrong structural policy with value 0.450.45 .
Table 1: Representative outcomes from the finite-context simulation benchmark. Figures are drawn from the executable study described in this section, using 1010 seeds per scenario and each scenario’s default horizon ( 20482048 or 40964096 ).
Figure 1: Core empirical comparisons. Left: in the direct-control, private-signal, workflow-redesign, and homogeneity-failure cases, RECERT matches a strong recommendation baseline while Actual-UCB breaks once recommendation and treatment incentives diverge. Right: in representative TRT settings, the relevant comparison is not just error rate but the split between correct deployment, abstention, and wrong deployment. BRACE abstains in the hard cases, unsafe baselines can fail in different ways, and strong-IV settings still allow correct structural deployment.
Figure 2: Selective structural-uncertainty comparison. Weak square-IV settings produce wide honest intervals, while rectangular overidentification and the rescued-IV design materially tighten them. The appendix shows the full inference panel; this figure isolates the design comparison that matters most for the paper’s main claim about certification and extra instruments.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Recommendation-track summary plots. Left: mean estimated REC value. Right: mean operational regret. Together they show that many methods agree on the final REC answer in easy environments, while differing sharply in how much operational welfare they spend during learning.
Figure 4: Treatment-track summary plots. Left: mean estimated TRT value. Right: abstention rate. The combination makes the main BRACE tradeoff visible: structural caution can preserve validity but often appears as delayed or missing deployment.
Figure 5: Structural risk and uncertainty validity. Left: wrong-non-abstain rate for TRT algorithms. Right: coverage of the best-policy TRT value in the inference track. The left panel highlights where unsafe methods make incorrect structural decisions; the right panel shows that the interval procedures remain conservative in the benchmark.
Figure 6: Inference diagnostics. Certified-share and interval-width plots together show how the benchmark separates strong-ID, weak-ID, and overidentified environments. Wide intervals in weak-ID cases are a feature of the safety story, not a bug in the implementation.
Figure 7: RECERT behavior. Left: deployed REC value for the joint extension. Right: TRT abstention rate for the same method. This is the compact graphical version of the paper’s intended operational stance: strong recommendation deployment together with structural caution.
Figure 8: TRT interval width for RECERT. The joint method is operationally useful precisely because its treatment side is interpreted as a certificate and diagnostic, not as a compulsory deployment rule.
More in this topic

Contextual Bandits

Contextual bandits learn which action to take from features of each situation when only the chosen action’s reward is observed.

See all 152 papers

Explore similar work

CardsList
Jul 5, 2026

Fixed-Confidence Best-Arm Identification for Causal Mediation Analysis

This paper studies the problem of identifying the treatment that maximizes the expected natural direct potential outcome (NDPO), which captures the potential outcome of an intervention while excluding the pathway transmitted through a mediator that researchers may wish to remove from evaluation. We first establish population-level identification of the expected NDPO in a causal bandit setting using observable interventional distributions. We then develop a fixed-confidence best-arm identification (BAI) algorithm based on the Track-and-Stop (TaS) framework, employing a cutting-set method to solve the resulting semi-infinite optimization problem. The proposed algorithm achieves sample-efficient identification with a high-probability correctness guarantee. We prove that it satisfies δδ-correctness and asymptotic optimality. Finally, we validate the approach through empirical evaluations on a large-scale real-world advertising dataset (IPinYou).

Harsh Shrivastava, Yuta Kawakami, Junpei Komiyama +1

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE · RIKEN AIP, Tokyo, Japan

Multi-Armed BanditsCausal Effect EstimationCausal Mediation AnalysisBest-Arm IdentificationCausal Inference
AskPDF
May 13, 2026

IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learning

The instrumental-variables (IV) setting is standard for partial identification of causal effects when unobserved confounding makes point identification impossible. Existing approaches face methodological bottlenecks: closed-form bound estimands are required -- e.g., Balke-Pearl equations in binary IV -- and even when available, designing accurate estimators requires manual effort tailored to each estimand. While direct Bayesian inference of the causal effects, instead of the bounds, circumvents these challenges, it is often computationally intensive and suffers from high prior sensitivity or under-dispersed posteriors. As a remedy, we introduce IV-ICL, an amortized Bayesian in-context learning method that learns the marginal posterior distribution of the causal effects directly and derives bounds as its quantiles. Unlike standard variational inference that optimizes exclusive KL divergence, amortized Bayesian inference minimizes the expected inclusive KL, a mass-covering objective. We empirically observe that optimizing inclusive KL can recover the entire identified set across diverse data-generating processes, while exclusive-KL (e.g. with variational inference) of the same Bayesian formulation collapses onto a single mode and fails to cover the identified set. We evaluate IV-ICL on synthetic and semi-synthetic IV benchmarks and show it produces intervals that are more reliably valid and more informative compared to efficient semi-parametric, Bayesian, and plug-in baselines, at 20-500x lower inference time. Beyond methodology, we propose a procedure to convert randomized controlled trials into IV benchmarks with provably preserved ground-truth causal effects that enables a more realistic evaluation of partial-identification methods.

Vahid Balazadeh, Hamidreza Kamkari, Medha Barath +2

University of Toronto · Vector Institute · MIT CSAIL +1

Causal Effect EstimationParameter IdentifiabilityAmortized InferencePartial IdentificationCausal IdentifiabilityIn-Context Learning+2
AskPDF
May 30, 2026

Bandit Simulation for Average Reward Inference

Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inference on their performance remains an open challenge. After deploying bandits, a natural question is whether one can construct a confidence interval for its mean reward and assess whether it reliably outperforms a baseline policy. The total reward achieved in any single bandit deployment is random, and deploying a bandit twice on the same population typically yields different reward trajectories due to stochastic rewards. Standard statistical inference methods cannot be used because bandit algorithms introduce complex dependencies in the collected data, which violate the i.i.d. assumption underlying many classical approaches. Moreover, existing inference methods for adaptively collected data only apply to estimands that do not depend on the data-collection algorithm (such as the mean reward under a fixed action). We propose Bandit Simulation for Inference (BSI), a framework that fits a simulator of the bandit environment from observed data--either on-policy or off-policy--and uses it to estimate the mean reward under any evaluation policy, including adaptive blackbox algorithms. BSI formally propagates uncertainty in the estimated simulator parameters into the confidence interval construction. Furthermore, for BSI to be valid, it requires only weak exploration assumptions on the behavior policy and avoids importance weighting. We prove that BSI yields asymptotically valid confidence intervals, and demonstrate empirically that it maintains nominal coverage in settings where standard off-policy evaluation methods fail.

Samya Praharaj, Chih-Yu Chang, Koulik Khamaru +1

Rutgers University · Imperial College London

Multi-Armed BanditsSimulation-Based InferenceOff-Policy Evaluation
AskPDF