RSD-Poker: Structure-Adaptive and Shift-Robust Risk-Utility Certification for Residual Policies in Imperfect-Information Games

Miaobo Hu; Shuhao Hu; Xiaobo Guo; Xin Wang; Bokun Wang; Peng Zhang; Daren Zha; Jun Xiao

cs.AISep 27, 2026

RSD-Poker: Structure-Adaptive and Shift-Robust Risk-Utility Certification for Residual Policies in Imperfect-Information Games

Authors: Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Peng Zhang, Daren Zha, Jun Xiao

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Abstract

Residual policy adaptation provides a lightweight way to modify a strong reference policy, but a shared scale and a fixed subgroup partition can hide heterogeneous degradation and become fragile when the deployment mixture of information states changes. We introduce RSD-Poker, a structure-adaptive and shift-robust certification framework that freezes a bank of residual families and scales, learns a policy-visible partition on an independent structure split, and freezes that partition before calibration labels are joined. Each candidate-group pair receives a weighted simultaneous upper certificate for anchor-relative risk and a lower certificate for weak-response utility. A robust group-to-candidate map is then selected over a predeclared uncertainty set of deployment group proportions. Under independent calibration units drawn from each frozen group's law, a candidate bank and partition fixed before calibration, and invariant within-group conditionals, the selected map satisfies its declared mixture-robust risk budget and utility certificate with probability at least $1-ζ_{risk}-ζ_{util}$ . The information contract supports both a teacher-backed transform and a teacher-free observation-only student. The retained deterministic 24-state audit remains an exact replay diagnostic: empirical-zero selects $α=0.08$ , raising the weak-response proxy from 4.2082 to 4.2889 with $0/12$ held-out threshold crossings. On stratified held-out states, the learned-partition dual selector raises weak utility from 4.4074 under global dual certification to 4.4936 and lowers held-out violation from 0.0215 to 0.0078; its mixture-robust variant reaches violation 0.0059. Across five observation-only checkpoints, risk-calibrated residuals attain weak utility $4.3659\pm0.0177$ and violation rate $0.0178\pm0.0057$ .

Figures & tables

Figure 1: RSD-Poker information flow for a single residual family. Top: calibration builds the smoothed anchor from the cached teacher, forms candidate traces over the scale grid, seals them before calibration oracle scores are joined, and freezes the selected scale. Bottom: the frozen policy is replayed on held-out observations, and strong, weak, and fidelity audits are joined only after the held-out trace is frozen. Snowflakes mark freeze boundaries; no held-out quantity feeds back into calibration.

Method	Weak EV $\uparrow$	Strong $\Delta$ $\uparrow$	Viol. $\downarrow$	UCB $\downarrow$	LCB $\uparrow$	KL $\downarrow$
Anchor	4.2113	0.0000	0.0000	0.0569	0.6118	0.0136
Fixed-0.08	4.2921	-0.0023	0.0059	n/a	n/a	0.0107
Empirical-Zero	4.3436	-0.0035	0.0117	0.0648	n/a	0.0119
Mean-Strong	4.3982	-0.0051	0.0449	n/a	n/a	0.0190
KL-Budget	4.3517	-0.0038	0.0234	n/a	n/a	0.0089
Global Risk-Only	4.3916	-0.0048	0.0293	0.0695	n/a	0.0144

Table 1: Primary stratified held-out comparison. Violation is a state fraction; UCB and LCB are risk and normalized-utility certificates. Best values in each column are bold and second-best values are underlined; ties share the mark. Candidate maps are in Table 10 ; n/a denotes a certificate not produced by that selector.

Scale	Weak EV	Strong $\Delta$	C count	H count	H $\Delta$
0.00	4.2082	0.0000	0/12	0/12	0.0000
0.04	4.2491	-0.0009	0/12	0/12	-0.0001
0.08	4.2889	-0.0021	0/12	0/12	-0.0004
0.12	4.3274	-0.0035	2/12	2/12	-0.0011
0.16	4.3648	-0.0052	5/12	3/12	-0.0019

Table 2: Deterministic replay scale audit. Weak EV (higher) and strong delta use 24 states; C/H counts use 12 calibration/held-out states. Bold weak EV is best among zero-calibration-count points; counts are threshold diagnostics for the empirical-zero baseline.

Appendix figures & tables40 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$s_{i},\mathcal{A}(s_{i})$	Observed state and its ordered legal action set.
$\pi_{T},\pi_{0}$	Cached teacher distribution and smoothed distributional anchor.
$\delta_{\phi}(s,a),\alpha$	Frozen residual direction and scalar scale.
$Q_{w},Q_{s}$	Weak- and strong-response action values in the scoring partition.
$E_{g}(\pi,s)$	Distribution-weighted response-model utility for pool $g$ .
$d_{b}(s)$	Strong utility relative to the same-state anchor for candidate $b$ .

Appendix

Table 3: Notation and evaluation units.

Step	Operation
1	Construct disjoint training, structure, calibration, and test splits; fix the legal-action order, thresholds, budgets, and partition class.
2	Fit residual families on training records and freeze the candidate bank $\mathcal{B}$ , parameters, scales, and normalization.
3	Learn the observable partition on structure records; freeze its rule, confidence weights, and deployment-mixture uncertainty set.
4	For each $b\in\mathcal{B}$ , construct the candidate distribution for every calibration observation; seal actions, distributions, group labels, and costs.
5	Join calibration risk and utility scores by state ID; compute weighted simultaneous $U_{b,h}$ and $L_{b,h}$ , including empty-group fallbacks.
6	Maximize worst-case utility over $\mathcal{Q}_{\rho}$ subject to the robust risk budget; freeze the selected map $m^{\star}$ . Record fixed-group and empirical-zero baselines separately.

Appendix

Table 4: Structure-adaptive certification, robust selection, and held-out replay.

ID	Split	Hero	Board	Pot	Stack	Pos.	History
0000	C	J	Q	4	16	BB	C–R
0001	H	Q	K	5	17	BTN	C
0002	C	K	A	6	18	BB	C
0003	H	A	J	7	19	BTN	C–R
0004	C	J	Q	8	20	BB	C
0005	H	Q	K	9	21	BTN	C

Appendix

Table 5: Complete observed-state inventory. C/H denotes calibration/held-out. Every row has the same three-action legal vocabulary.

Field	T	S	Evaluation use
Board and public history	yes	yes	Context and replay.
Own private rank	yes	yes	Acting-player information.
Legal-action mask	yes	yes	Support and validity.
Cached teacher distribution	yes	no	Offline anchor or training target.
Opponent private cards	no	no	Evaluator partition where applicable.
Weak/strong action values	no	no	Post-trace dot products and deltas.

Appendix

Table 6: Field visibility for the two policy interfaces. T denotes access inside the teacher-backed transform; S denotes student inference.

Quantity	Unit	Failure treatment
Fixture weak/strong score	Distinct state–policy row	Missing score has an explicit missing status.
Threshold count	Eligible distinct states	Report numerator and eligible denominator together.
Action legality	All emitted attempts	Illegal attempts remain in the denominator.
Retry rate	Attempts or episodes, declared	Retain retries even when the final action succeeds.
Opponent score	Hands or paired match blocks	Keep ties and state/deck pairing explicit.
Training variability	Independent training seeds	Resample seeds or report seed dispersion separately.

Appendix

Table 7: Aggregation units and failure treatment.

Parameter	Residual student	Matched control
Inference fields	Observations and legal mask	Same fields
Adapter rank	16	16
Optimizer	AdamW	AdamW
$(\beta_{1},\beta_{2})$	$(0.9,0.95)$	$(0.9,0.95)$
Replicate seeds	3	3
Training seed IDs	$\{17,29,43\}$	$\{17,29,43\}$

Appendix

Table 8: Student configuration for the three-seed evaluation.

Field	Study setting
Independent checkpoints per arm	5
Matched factors	Backbone, tokenizer, training states, initialization family, optimizer, update budget, and evaluation episodes
Backbone / tokenizer identities	Qwen3.5-2B / native Qwen3.5 tokenizer
Training seed / checkpoint identifiers	$\{11,23,37,53,71\}$ / rsd5-s11/s23/s37/s53/s71-u12000 , ctl5-s11/s23/s37/s53/s71-u12000
Training / structure / calibration / test counts	$24{,}576$ / $2{,}048$ / $512$ / $512$
Learning rate / batch size / update count	$1.6\times 10^{-4}$ / $32$ / $12{,}000$

Appendix

Table 9: Replication fields for the five-checkpoint observation-only study.

Selector	Candidate or ordered group map
Anchor	anchor
Fixed-0.08	fixed@0.08
Empirical-Zero	fixed@0.12
Mean-Strong	student@0.16
KL-Budget	linear@0.08
Global Risk-Only	MLP@0.12

Appendix

Table 10: Frozen residual candidates in the stratified comparison. The number after @ is the residual scale.

Declared $\varepsilon$	Method	Held-out violation	Coverage	No-cert.	Splits
0.10	Global Risk-Only	0.0308	0.975	3/200	200
0.10	Group Risk-Only	0.0199	0.990	8/200	200
0.10	Group Dual-Certified	0.0138	0.995	8/200	200

Appendix

Table 11: Risk-calibration validity under repeated independent splits. Coverage is the fraction of splits whose held-out violation rate is at most the declared budget; each row uses the same split generator, candidate bank, and held-out denominator. Lower violation and fewer no-certificate outcomes are preferred; coverage is higher-is-better. Best values are bold.

Endpoint	Spearman $\rho$	Candidate count	Evaluation unit
NashConv	0.731	96	solver profile
Best-response value	0.684	96	solver profile
Paired opponent EV	-0.612	96	completed hand
Opponent score	-0.657	96	completed hand

Appendix

Table 12: Association between state-level violation risk and strategic endpoints across frozen candidates and checkpoints.

$n_{C}$	Cert. rate	Median $\alpha^{\star}$	Weak EV	Held-out viol.	Risk UCB
32	0.530	0.04	4.2547	0.0176	0.1682
64	0.680	0.04	4.2619	0.0195	0.1249
128	0.820	0.08	4.3214	0.0215	0.0961
256	0.930	0.12	4.3836	0.0234	0.0794
512	0.980	0.16	4.4397	0.0254	0.0698

Appendix

Table 13: Calibration-size scaling, 200 independent splits per row. Larger certification rate and weak EV, and smaller violation and UCB, are preferred; bold marks the best value across the sweep.

$\tau$	$\varepsilon$	$\zeta$	$\alpha^{\star}$	Weak EV	Held-out viol.	Risk UCB
0.025	0.05	0.05	0.04	4.2837	0.0215	0.0486
0.050	0.05	0.05	0.08	4.3372	0.0176	0.0471
0.050	0.10	0.05	0.12	4.4085	0.0234	0.0793
0.100	0.20	0.10	0.16	4.4487	0.0117	0.1187

Appendix

Table 14: Risk-budget sensitivity. Each row uses a fixed candidate family and reports the selected policy under the stated severity, risk, and confidence parameters. Bold marks the largest weak EV; risk values across different severity thresholds describe different events.

Residual	Weak EV	Held-out viol.	Risk UCB	Teacher KL	Cost
Fixed rule	4.3446	0.0293	0.0884	0.0108	1.00 $\times$
Linear	4.3728	0.0254	0.0816	0.0121	1.02 $\times$
MLP	4.4117	0.0195	0.0678	0.0156	1.05 $\times$
Student residual	4.4389	0.0156	0.0569	0.0149	1.07 $\times$

Appendix

Table 15: Residual-family ablation under the same risk-calibration rule. Weak EV is higher-is-better; violation, UCB, KL, and relative cost are lower-is-better. Best values are bold.

Method	Weak EV $\uparrow$	Violation $\downarrow$	NashConv $\downarrow$
No residual	$4.2207\pm 0.0248$	$0.0549\pm 0.0106$	$0.1048\pm 0.0091$
Fixed residual	$4.3108\pm 0.0213$	$0.0346\pm 0.0082$	$0.0809\pm 0.0068$
Risk-calibrated residual	$4.3659\pm 0.0177$	$0.0178\pm 0.0057$	$0.0615\pm 0.0053$
Method	Opp. score $\uparrow$	Legal (%) $\uparrow$	p95 ms $\downarrow$
No residual	$0.5114\pm 0.0088$	$99.18\pm 0.21$	$113.7\pm 4.2$
Fixed residual	$0.5421\pm 0.0074$	$99.49\pm 0.16$	$109.3\pm 3.5$

Appendix

Table 16: Teacher-free learned-policy evaluation. Entries are mean

\pm

standard deviation across five independent training seeds. Arrows mark preferred directions; best means are bold. Both panels use the same arms.

Evaluation distribution	Weak EV	Strong $\Delta$	Violation rate	Opp. score
Matched	4.4468	-0.0042	0.0117	0.5690
Position shift	4.4117	-0.0069	0.0234	0.5582
Rank shift	4.3976	-0.0083	0.0293	0.5538
History shift	4.3839	-0.0097	0.0332	0.5496
Pot-pressure shift	4.3608	-0.0132	0.0449	0.5431
Opponent shift	4.3746	-0.0108	0.0371	0.5364

Appendix

Table 17: Risk transfer under structured held-out distribution shifts. Utility, strong delta, and opponent score are higher-is-better; violation is lower-is-better. Best values are bold.

Partition rule	Leaves	$n_{\min}$	Cert. rate	Weak EV	Held-out viol.	Worst-group viol.
Global	1	512	0.975	4.4074	0.0215	0.0312
Manual observable groups	4	126	0.960	4.4468	0.0117	0.0234
Random partition	4	120	0.945	4.4327	0.0176	0.0312
Learned, no certificate penalty	7	56	0.935	4.4769	0.0137	0.0312
Learned, certificate-aware	4	116	0.990	4.4936	0.0078	0.0156

Appendix

Table 18: Partition ablation under fixed candidate and calibration contracts. Higher certification rate and weak EV and lower held-out and worst-group violation are preferred; best values are bold. Leaves and

n_{\min}

are descriptive.

Depth	$n_{\min}$	Groups	Cert. rate	Utility LCB	Worst-group viol.	Map time (ms)
1	248	2	0.980	0.7164	0.0195	0.34
2	116	4	0.990	0.7257	0.0156	0.68
3	71	7	0.970	0.7221	0.0195	1.43
4	42	10	0.940	0.7149	0.0312	3.08

Appendix

Table 19: Group-complexity and support sweep for the learned partition. Higher certification rate and utility LCB and lower worst-group violation and map time are preferred; best values are bold.

$\rho$	Nominal risk	Worst-case risk UCB	Robust utility LCB	Held-out viol.	Weak EV
0.00	0.0457	0.0508	0.7257	0.0078	4.4936
0.05	0.0438	0.0516	0.7236	0.0078	4.4898
0.10	0.0415	0.0524	0.7209	0.0059	4.4827
0.20	0.0389	0.0551	0.7152	0.0059	4.4695

Appendix

Table 20: Mixture-robust selection as the deployment group radius varies. Lower risk, risk UCB, and violation and higher utility LCB and weak EV are preferred; best values are bold.

Risk evaluator	Utility evaluator	Weak EV	Held-out viol.	Risk UCB	Strategic endpoint
Strong response	Weak response	4.4936	0.0078	0.0508	0.5843
Weak response	Strong response	4.3716	0.0332	0.0756	0.5607
Strong response	Strong response	4.4418	0.0059	0.0489	0.5794
Weak response	Weak response	4.5148	0.0469	0.0887	0.5482

Appendix

Table 21: Risk-evaluator and utility-evaluator role ablation. Higher weak EV and strategic endpoint (opponent score) and lower violation and risk UCB are preferred; best values are bold. The first row is the proposed assignment.

Families	Scales	$K$	Cert. rate	Utility LCB	Search time (ms)
1	5	5	0.995	0.7034	0.41
2	5	10	0.995	0.7169	0.78
4	5	20	0.990	0.7257	1.56

Appendix

Table 22: Candidate-bank size and residual-family scaling. Higher certification rate and utility LCB and lower search time are preferred; best values are bold.

Range	Utility LCB	Selected map	Weak EV	Held-out viol.
$[3.0,5.0]$	0.7257	l.08/m.12/m.16/s.16	4.4936	0.0078
$[2.8,5.2]$	0.6891	l.08/m.12/m.16/s.16	4.4936	0.0078
$[2.5,5.5]$	0.6478	l.08/m.12/m.16/s.16	4.4936	0.0078
$[2.0,6.0]$	0.6049	f.08/l.12/m.12/s.12	4.4827	0.0059

Appendix

Table 23: Sensitivity to the predeclared utility-normalization range. Higher weak EV and lower violation are preferred; best values are bold. Utility LCBs use different normalizations across rows and are not ranked.

Conditional shift	Weak EV	Strong $\Delta$	Held-out viol.	Opp. score
Matched conditional	4.4827	-0.0032	0.0059	0.5816
Within-group state shift	4.4538	-0.0059	0.0156	0.5729
Opponent-response shift	4.4371	-0.0077	0.0215	0.5608
Adaptive opponent	4.4086	-0.0109	0.0332	0.5487

Appendix

Table 24: Negative-control shifts that change within-group conditionals.

Policy	Weak EV	Teacher KL	Regret	Strong $\Delta$	Count
Top-1	5.3125	5.9214	0.0000	-0.1746	15/24
Anchor	4.2082	0.0134	0.3151	0.0000	0/24
Active query	3.0636	0.1878	1.4200	-0.2953	14/24
Residual (1.00)	4.9138	0.0786	0.0009	-0.0669	14/24
Residual (0.08)	4.2889	0.0105	0.2392	-0.0021	0/24

Appendix

Table 25: Matched policy controls on 24 fixture states. Weak EV is higher-is-better; KL, teacher regret, and threshold counts are lower-is-better. Best values in each of those columns are bold; strong delta retains its sign.

Arm	Count	Weak EV	Strong EV	Expl.	Opp.	ms
Residual	0/36	4.3371	4.2146	0.0318	0.5627	84
No residual	0/36	4.2189	4.2073	0.0524	0.5098	91
Full policy	2/36	4.3026	4.1984	0.0437	0.5386	133
Residual: shift	0/36	4.3094	4.2102	0.0361	0.5483	88

Appendix

Table 26: Three-seed student evaluation. Higher weak/strong EV and opponent score (Opp.) are preferable; lower exploitability (Expl.) and latency are preferable; best values are bold.

Scale	Weak EV	Strong EV	Strong $\Delta$	Min $\Delta$	Max $\Delta$	Count
0.00	4.2082	0.7371	0.0000	0.0000	0.0000	0/24
0.04	4.2491	0.7362	-0.0009	-0.0217	0.0327	0/24
0.08	4.2889	0.7351	-0.0021	-0.0431	0.0636	0/24
0.12	4.3274	0.7336	-0.0035	-0.0643	0.0928	4/24
0.16	4.3648	0.7319	-0.0052	-0.0852	0.1204	8/24

Appendix

Table 27: Full-fixture sensitivity, 24 rows per scale. Utility and delta columns use the fixed response-model units; count is below

-0.05

Scale	Split	Mean $\Delta$	Count	95% interval
0.00	C	0.0000	0/12	[0.0000, 0.0000]
0.00	H	0.0000	0/12	[0.0000, 0.0000]
0.04	C	-0.0017	0/12	[-0.0119, 0.0102]
0.04	H	-0.0001	0/12	[-0.0094, 0.0094]
0.08	C	-0.0037	0/12	[-0.0233, 0.0185]
0.08	H	-0.0004	0/12	[-0.0186, 0.0183]

Appendix

Table 28: Split-specific strong deltas and descriptive 95% percentile intervals. Each C/H row uses 12 states. Scale selection uses C counts only.

Policy	Mean $\Delta$	95% interval	Count
Active query	-0.2953	[-0.4830, -0.1394]	14/24
Anchor	0.0000	[0.0000, 0.0000]	0/24
Residual (0.08)	-0.0021	[-0.0158, 0.0121]	0/24
Residual (1.00)	-0.0669	[-0.1865, 0.0572]	14/24
Top-1	-0.1746	[-0.3650, 0.0188]	15/24

Appendix

Table 29: Full-fixture strong-delta intervals and common-threshold counts. Intervals are descriptive fixed-row bootstrap summaries, 4,000 resamples.

Policy	Query rate	Tokens	Latency (ms)	Tool calls
Top-1	0.0000	1.00	0.1000	0.0000
Anchor	0.0000	1.00	0.1000	0.0000
Active query	0.2917	3.33	0.6833	0.2917
Residual (1.00)	0.0000	2.00	0.3000	0.0000
Residual (0.08)	0.0000	2.00	0.3000	0.0000

Appendix

Table 30: Configured CPU-fixture ledger charges per state. Tokens and latency are accounting constants, with a query-weighted average for active query; they are separate from measured service latency.

ID	Split	Top-1	Anchor	Active query	Res. (1.00)	Res. (0.08)
0000	C	-0.6951 †	0.0000	-0.3759 †	-0.4066 †	-0.0405
0001	H	0.1201	0.0000	-0.1596 †	0.0755	0.0084
0002	C	-0.7451 †	0.0000	-0.3106 †	-0.4032 †	-0.0382
0003	H	-0.3834 †	0.0000	-0.3982 †	-0.2101 †	-0.0191
0004	C	0.4858	0.0000	0.0000	0.3560	0.0468
0005	H	0.4281	0.0000	0.0000	0.3280	0.0455

Appendix

Table 31: Per-state anchor-relative strong delta

d_{i}

on the fixed fixture. A dagger marks a strict threshold crossing

d_{i}<-0.05

; the count row gives crossings out of 24 states. C/H is calibration/held-out.

		Weak EV					Strong EV
ID	Split	T1	A	AQ	R1	R.08	T1	A	AQ	R1	R.08
0000	C	0.2500	-1.0451	-2.7923	-0.3869	-0.9849	-4.5000	-3.8049	-4.1808	-4.2115	-3.8454
0001	H	2.7500	1.5045	-0.3636	2.2588	1.5865	-1.7500	-1.8701	-2.0298	-1.7946	-1.8617
0002	C	6.0000	4.9936	3.7203	5.5989	5.0569	1.0000	1.7451	1.4345	1.3419	1.7069
0003	H	7.7500	6.4943	5.4878	7.2858	6.5788	3.0000	3.3834	2.9852	3.1733	3.3642
0004	C	3.5000	2.5610	2.5610	3.2569	2.6536	-1.0000	-1.4858	-1.4858	-1.1298	-1.4390

Appendix

Table 32: Per-state weak- and strong-response utility on the fixed fixture for Top-1 (T1), anchor (A), active query (AQ), and residual scales 1.00 and 0.08 (R1, R.08). Strong delta in Table 31 subtracts the same-state anchor entry. Mean rows are computed from unrounded records and equal the aggregates of Tables 25 and 27 .

ID	A:F	A:C	A:Raise	R:F	R:C	R:Raise
0000	0.3309	0.0881	0.5810	0.3128	0.0850	0.6022
0001	0.1152	0.1348	0.7499	0.1065	0.1272	0.7663
0002	0.0437	0.3066	0.6497	0.0404	0.2896	0.6700
0003	0.0361	0.3090	0.6549	0.0332	0.2900	0.6768
0004	0.0701	0.0306	0.8993	0.0626	0.0279	0.9095
0005	0.0365	0.0533	0.9102	0.0323	0.0482	0.9195

Appendix

Table 33: Legal-action probabilities for anchor (A) and selected residual (R). F/C/Raise are fold/call/raise; all values are rounded copies of stored probabilities.

Policy	Action accuracy	Log loss	Raise agreement
public prior	0.5039	0.9778	0.0000
frequency calibrated	0.5039	0.9614	0.8155
top1 calibrated	0.5039	2.2948	0.8155

Appendix

Table 34: PHH behavior-label metrics. Action accuracy and raise agreement are higher-is-better; log loss is lower-is-better. All arms use 256 snapshots and 103 raise examples. Best values are bold.

Profile	BR0	BR1	$u_{0}$	NashConv	Exploit.
Uniform	2.0875	2.6597	-0.0781	4.7472	2.3736
1	2.8555	2.3022	-0.8533	5.1577	2.5789
10	0.5747	0.7484	-0.1307	1.3231	0.6615
100	-0.0394	0.1270	-0.0813	0.0876	0.0438
1000	-0.0787	0.0924	-0.0853	0.0136	0.0068

Appendix

Table 35: Exact-Leduc calibration. BR0/BR1 are player best-response values;

u_{0}

is the profile value. NashConv and exploitability are lower-is-better; the final values are best among the listed checkpoints.

Arm / opponent	W/T/L (%)	Legal (%)	Solver EV	BR
Residual / A	54.86 /2.82/ 42.32	99.82	0.0642	0.0960
Residual / B	53.49/2.68/43.83	99.79	0.0617	0.0978
No residual / A	49.58/2.80/47.62	99.21	0.0494	0.1018
Full policy / solver	52.50/2.72/44.78	99.43	0.0558	0.0995

Appendix

Table 36: Independent-opponent evaluation. W/T/L and legality are percentages. Bold compares the two matched opponent-A arms: higher win rate, legality, and solver EV, and lower loss rate and BR.

Arm / opponent	Exploit.	Retry (%)	Tokens	Latency (ms)
Residual / A	0.0318	0.6	118	84
Residual / B	0.0361	0.8	121	88
No residual / A	0.0524	1.1	116	91
Full policy / solver	0.0437	1.8	174	133

Appendix

Table 37: Service costs of the independent-opponent evaluation in Table 36 . All columns are lower-is-better; bold compares the two matched opponent-A arms. Retry is in percent.

Arm	Solver EV	Exploit.	Opp. score	Calls	ms
Residual	4.3847	0.0528	0.5714	2.8	214
No residual	4.2916	0.0719	0.5326	2.7	207
Full policy	4.3372	0.0615	0.5498	3.9	296

Appendix

Table 38: HUNL evaluation, mean over three seeds. Solver EV and opponent score are higher-is-better, exploitability and calls/latency lower-is-better; best values are bold.

Component	Acc. (%)	Cal.	Solver EV	Gap	Cost
Behavior cloning	71.42	0.0847	4.1036	0.0862	1.00 $\times$
Policy distillation	73.18	0.0714	4.1825	0.0718	1.03 $\times$
Residual correction	75.64	0.0498	4.2769	0.0465	1.06 $\times$
Observed-only residual	76.31	0.0416	4.3128	0.0397	1.07 $\times$

Appendix

Table 39: Component evaluation. Accuracy and solver EV are higher-is-better; calibration error (Cal.), exploitability gap (Gap), and relative cost are lower-is-better. Best values are bold.

Endpoint	Residual	No residual	Full policy
Seed-level weak-EV SD	0.0183	0.0272	0.0233
Paired hand-EV interval	$\mathbf{[0.0610,0.0680]}$	$[0.0461,0.0533]$	$[0.0525,0.0595]$
Profile NashConv	0.0642	0.1056	0.0880
Best response, player 0	0.0287	0.0479	0.0392
Best response, player 1	0.0355	0.0577	0.0488
Measured latency p95 (ms)	104.2	113.6	163.7

Appendix

Table 40: Replication endpoints for the three-seed student evaluation. Lower SD, NashConv, best-response value, and latency and a higher hand-EV interval are preferred; best values are bold.

Handle	Record	Manuscript use
S1	Scale sensitivity	Full-fixture five-point frontier.
S2	Held-out scale selection	12/12 split, selected scale, intervals.
S3	Public audit bundle	Observations, teacher labels, five policy traces and per-state scores.
S4	Policy uncertainty	Full-fixture descriptive bootstrap intervals.
S5	PHH player/time metrics	Behavior calibration and label denominators.
S6	Exact-Leduc calibration	Profile values and best-response curve.

Appendix

Table 41: Numerical sources and corresponding manuscript surfaces.

Check	Pass condition
State inventory	24 unique observed keys with matching label keys.
Split closure	12 calibration and 12 held-out keys, no overlap.
Probability support	Finite nonnegative mass on declared legal actions.
Distribution replay	Equation 1 matches stored probabilities within the recorded tolerance.
Action replay	Stored action follows the masked top-action tie rule.
Score aggregation	State scores reproduce the retained policy summaries.

Appendix

Table 42: Reproduction checks and their pass conditions.

Explore similar work

Jun 7, 2026cs.LG

A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control

Selective predictors answer on confident inputs and abstain elsewhere; deploying one safely needs a single finite-sample certificate that simultaneously upper-bounds the selected risk, lower-bounds the acceptance probability

\pacc

above a floor

\pmin

, and lower-bounds the deployment utility. This certificate must be valid under adaptive threshold selection from a finite grid of

m

pairs on

\ncert

samples. We give such a certificate for bounded, possibly non-monotone losses by treating the selected risk directly as a ratio rather than through a Hoeffding-style range bound. The construction couples three confidence bounds: a variance-adaptive empirical-Bernstein bound on the ratio risk, a Clopper--Pearson bound on acceptance, and a two-sided closeness bound on utility. Together they lower-bound the certified policy's utility absolutely and to within

2\gammau

of the best over the \emph{certified set}, both non-vacuous whenever feasible; a regime-scoped third leg matches an external oracle, informative only where the risk margin

\gammar < α

and vacuous at the headline operating points. Relative to the range-only Hoeffding-ratio construction this sharpens the acceptance-floor dependence from

1/\pmin

1/\sqrt{\pmin}

, and a closed-form corollary identifies a per-pair regime in which our risk bound dominates a Hoeffding conformal risk control (Hoeffding--CRC) selective bound. Empirically, on ImageNet (three ResNets) and COCO val 2017 panoptic, the certificate opens a

+22

pp certified-acceptance frontier over Hoeffding--CRC and is

{\approx}10{\times}

tighter than a non-vacuous matched-valid baseline; these gains are regime-scoped, not universal, and absent on ADE20K. The certifier runs in

O(\ncert m)

time.

Xiaoli Yu, Jiamiao Liu

School of Cyber Security and Information Law Chongqing University of Posts and Telecommunications Chongqing 400065, China · Department of Information, Xinqiao Hospital Army Medical University (Third Military Medical University) Chongqing 400037, China

Conformal Risk Control Finite-Sample Certificates Confidence Intervals Certificates Risk

Ask PDF

Sep 11, 2026cs.LG

Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations

Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications. Certification methods can improve robustness against adversarial perturbations by providing lower bounds on expected cumulative rewards. Existing certification methods, however, mainly focus on risk-neutral objectives. In this paper, we extend certification methods to risk-sensitive objectives by establishing lower bounds on the exponential utility of cumulative rewards under

l_{p}

-norm-bounded state adversarial perturbations (

1\leq p <\infty

). By introducing a

\phi

-divergence relaxation of the perturbation set, we formulate the risk-sensitive certification problem as a convex optimization and derive its dual to obtain a tractable approximation of the certified lower bound. We further propose an empirical method that improves certified lower bounds by selecting the training risk-aversion parameter

\beta

independently of the risk level used during evaluation. Experiments on both OpenAI Gym environments and a machine replacement problem show that, compared to risk-neutral training, risk-averse training generally yields policies with higher certified lower bounds, particularly under larger perturbation budgets. Moreover, under both risk-neutral and risk-averse evaluation settings, increasing risk aversion during training leads to non-monotonic certification performance, where certified lower bounds initially improve but eventually decrease due to overly conservative policies.

Tong Li, Saunak Kumar Panda, Yisha Xiang

Department of Industrial and Systems Engineering, University of Houston, 4302 University Dr, Houston, 77004, TX, USA. · Department of Decision and Information Science, University of Houston, 4302 University Dr, Houston, 77004, TX, USA.

Conditional-Value-At-Risk Offline Reinforcement Learning Certification Lower Bounds Convex Optimization

Ask PDF

Aug 16, 2026cs.LG

Conditional Validity for Adaptive Modality Acquisition: When the Policy Chooses Its Own Calibration Group

A multimodal system may begin inference while holding only some of its inputs and may acquire the rest at a cost. With adaptive acquisition, the policy determines which inputs are ultimately observed, so we state risk control conditional on that terminal input pattern. Conditional calibration typically assumes that the grouping map is fixed independently of the calibration sample, a condition that policy-induced grouping does not satisfy. We characterize when pattern-conditional risk control remains valid and give two finite-sample constructions: threshold-free routing with calibration applied at the terminal pattern, and simultaneous validation of complete policy-pattern pairs, which lets calibration data select the deployed policy. A counterexample shows that validity proved for a calibration-independent grouping map need not transfer once the policy makes the terminal group calibration-dependent. We call the resulting method RouteCert. On a clinical electrocardiogram task with a staged, cost-ordered lead protocol, the deployed policy answers 71.2% of held-out patients, with an observed 7.4% disagreement with the cardiologist's diagnosis at 48.8% of the prespecified ordinal cost of acquiring each stage, and all three acquisition stages are validated separately. On masked multimodal benchmarks, validating pointwise at each terminal pattern holds observed worst-pattern selective risk, measured against the full-information reference decision rather than the true label, at 0.034, where a pooled design reaches 0.145 against a 0.10 cap, at a comparable answered fraction (0.350 vs 0.342); under the budget-matched simultaneous comparison, the answered fraction falls to 0.305.

Melika Baghi

Georgia Institute of Technology Atlanta, GA, USA

Conformal Risk Control Acquisition Calibrated Uncertainty

Ask PDF