DECODE: Decomposable Environments for COntrolled Sequential-Decision Evaluation
Authors: Diego Silva Prado, Bruno Caldas de Souza, Tiago Amaral Penteado de Jesus, Ricardo Guimarães Herrmann, Cristiano Breuel, Cristiano Bevitori Maffia De Oliveira
When a sequential decision system falls short of an optimum, its score does not reveal which modifications would improve its decisions. DECODE is a benchmark for testing such modifications through controlled experiments on planning and available information. Its consumer-credit collections generator provides five partially observed environments that share known dynamics matrices and vary a memory parameter and process noise. The experiments change selected inputs or planning rules while keeping the rest of the solver fixed. Each comparison specifies the solver's state inputs, response inputs and planning horizon. On the deterministic base environment, which was selected for a large gap between six-month and one-month planning with the hidden state, extending planning from one to six months increases the scores of two gradient-boosted solvers by 15 to 17 percentage points of the exact optimum, computed with the hidden state and borrower traits. Even at this horizon, the best graded solver, whose repayment response is fitted on logs, remains 15.6 points below that optimum. In a specified intervention sequence, three changes to its state inputs and their evaluation during search lower its score by 6.3 points, and replacing its fitted repayment model with the true response, evaluated with hidden borrower traits, then raises it by 15.6, a net recovery of nine of those points. Because the intermediate effects are conditional on the repayment model and the order of the changes, they do not uniquely assign the gap to separate causes. Within these limits, DECODE makes hypotheses about solver performance testable through specified interventions under known dynamics and explicit information conditions.
Figures & tables
Figure 1: Overview of DECODE.
Figure 2: World generation and policy-value evaluation.
Reference
Information
Calculation
Interpretation
Exact optimum ( σ=0 )
True hidden state and borrower traits
Enumerate channel sequences and optimize discounts by backward dynamic programming
MDP upper bound for the grading objective
One-month optimum ( σ=0 )
True hidden state and borrower traits
Optimize the current month and re-decide monthly
Reference for myopic planning
Filter planner
Gaussian belief from readings, with true borrower traits
Integrate monthly repayment probabilities and chain their marginals
Value of a specified policy, without POMDP optimality
True-state re-planning ( σ>0 )
Current true hidden state and borrower traits
Monthly re-planning, evaluated over shared innovation paths
Normalizer, not an upper bound
Table 1: Policy-value references.
Probe
Information set
Snapshot
Current month’s public state, reading, and offered discount
History
Snapshot set plus the last six months of readings and actions (recorded by channel)
True state
Snapshot set plus the hidden state zt
Contrast
What it measures
Separation, history − snapshot
Increase in fitted predictive R2 from the six-month window
Headroom, true state − history
Increase in fitted predictive R2 from adding the true state
Table 2: Information probes and contrasts.
Solver
Information
Model or policy
Decision rule
Horizon
GBM-lags
Public state, current reading, six months of readings and channels
Two calibrated GBM classifiers
Myopic or sequential, reading and history features frozen during search
1, 6
GBM-flat
Public state and current reading
As GBM-lags
As GBM-lags, without history
1, 6
CE-Seq
Current state and reading, full pre-decision history including account states
Recurrent belief with two calibrated heads
Sequential, future readings masked
6
GBM-cash
Public state and current reading
GBM cash predictor, Tweedie loss
Myopic
1
LLM-agent
Delinquency ageing and balance
Fixed rules written from censored logs
Rules applied monthly
1
Table 3: Graded solvers.
Figure 3: Closed-loop performance on Base.
Figure 4: Memory measurements.
Figure 5: Memory and re-observation.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Term
Meaning
Base, Mem-0.5, Mem-4, Noise-0.1, Noise-0.2
the five released environments, with parameter settings in Table 5
belief
the probability distribution of the hidden state given everything a solver has observed
certainty-equivalent (CE)
chaining monthly repayment probabilities without their cross-month dependence
closed-loop grading
grading in which each solver re-decides every month on the readings its own actions generate
decision point
month six of an evaluation loan, when a solver takes over after six months of public history, with six months left to decide
disclosure
the replacement of one input of a graded solver by a specified reference quantity
Appendix
Table 4: Glossary.
Member
Transition model
Memory δ
Noise σ
Base rate
Checks
Base
linear, deterministic
0
0
0.038
pass, re-anchored
Mem-0.5
action-rotation
0.5
0
0.038
pass, one clause amended
Mem-4
action-rotation
4
0
0.026
pass, re-anchored
Noise-0.1
linear, stochastic
0
0.1
0.038
pass
Noise-0.2
linear, stochastic
0
0.2
0.032
pass, re-anchored
Appendix
Table 5: Members and release checks.
Solver
Change from the row above
Value
Step
95% interval
GBM-lags
none
84.40
exact fatigue
exact per-path fatigue instead of interpolation
83.65
−0.75
[−1.18,−0.34]
filter features
reference-filter features instead of history
82.97
−0.68
[−1.60,+0.22]
rolled belief
filter belief propagated instead of frozen
78.14
−4.83
[−6.29,−3.43]
exact response
true response with hidden traits instead of fitted response
93.75
+15.61
[14.14,17.16]
full disclosure
all four changes together
+9.35
[8.68,10.02]
Appendix
Table 6: Declared intervention sequence on Base.
Configuration
ΔR2 vs history
share
at epoch cap
recurrent (GRU), small
−0.002±0.003
≈0
no stopping log
recurrent (GRU), small, long budget
+0.083±0.002
27.9%
yes
recurrent (LSTM), large
+0.071±0.006
23.7%
yes
recurrent (GRU), large
+0.112±0.002
37.7%
yes
reference filter (fitted)
+0.205
68.8%
information reference (no fit)
+0.298
100%
Appendix
Table 7: Memory value on Mem-4.
configuration
epoch cap
patience
learning rate
attention (this appendix)
30
5
3×10−4
recurrent (Table 7 )
30
5
10−3
recurrent, long budget (Table 7 )
100
15
10−3
member
seeds
memory value
per-seed
Base
2
−0.021
−0.022 , −0.020
Mem-0.5
2
0.000
−0.008 , +0.008
Appendix
Table 8: Attention model budgets and memory values.
Solver
1-month horizon
6-month horizon
GBM-lags
67.78
84.40
GBM-flat
65.59
80.65
CE-Seq
82.28
GBM-cash
66.32
LLM-agent
64.74
random policy
29.11
Appendix
Table 9: Closed-loop values on Base.
Contrast
Points
95% interval
GBM-lags, six months minus one month
+16.62
[15.31,18.02]
GBM-flat, six months minus one month
+15.06
[13.54,16.66]
GBM-lags minus GBM-flat, six months
+3.75
[2.75,4.78]
GBM-lags minus CE-Seq, six months
+2.12
[0.82,3.34]
CE-Seq minus GBM-flat, six months
+1.63
[0.17,3.21]
GBM-lags minus GBM-flat, one month
+2.19
[1.32,3.15]
Appendix
Table 10: Paired contrasts on Base.
Data
Δ AUC
95% interval
Partition range
Fannie Mae spells (real, n=20,677 )
+0.041
[0.031,0.050]
[0.030,0.045]
Base
+0.013
[0.010,0.016]
[0.011,0.014]
Mem-0.5
+0.013
[0.010,0.016]
[0.012,0.017]
Mem-4
+0.010
[0.007,0.013]
[0.008,0.013]
Appendix
Table 11: Predictive gains from pre-entry history.
Department of Pure Mathematics and Mathematical Statistics, University of Cambridge · Department of Electrical and Computer Engineering, University of British Columbia