DECODE: Decomposable Environments for COntrolled Sequential-Decision Evaluation
Organizations: Alpha Synthesis Labs
Abstract
When a sequential decision system falls short of an optimum, its score does not reveal which modifications would improve its decisions. DECODE is a benchmark for testing such modifications through controlled experiments on planning and available information. Its consumer-credit collections generator provides five partially observed environments that share known dynamics matrices and vary a memory parameter and process noise. The experiments change selected inputs or planning rules while keeping the rest of the solver fixed. Each comparison specifies the solver's state inputs, response inputs and planning horizon. On the deterministic base environment, which was selected for a large gap between six-month and one-month planning with the hidden state, extending planning from one to six months increases the scores of two gradient-boosted solvers by 15 to 17 percentage points of the exact optimum, computed with the hidden state and borrower traits. Even at this horizon, the best graded solver, whose repayment response is fitted on logs, remains 15.6 points below that optimum. In a specified intervention sequence, three changes to its state inputs and their evaluation during search lower its score by 6.3 points, and replacing its fitted repayment model with the true response, evaluated with hidden borrower traits, then raises it by 15.6, a net recovery of nine of those points. Because the intermediate effects are conditional on the repayment model and the order of the changes, they do not uniquely assign the gap to separate causes. Within these limits, DECODE makes hypotheses about solver performance testable through specified interventions under known dynamics and explicit information conditions.
Figures & tables
| Reference | Information | Calculation | Interpretation |
|---|---|---|---|
| Exact optimum ( ) | True hidden state and borrower traits | Enumerate channel sequences and optimize discounts by backward dynamic programming | MDP upper bound for the grading objective |
| One-month optimum ( ) | True hidden state and borrower traits | Optimize the current month and re-decide monthly | Reference for myopic planning |
| Filter planner | Gaussian belief from readings, with true borrower traits | Integrate monthly repayment probabilities and chain their marginals | Value of a specified policy, without POMDP optimality |
| True-state re-planning ( ) | Current true hidden state and borrower traits | Monthly re-planning, evaluated over shared innovation paths | Normalizer, not an upper bound |
| Probe | Information set |
|---|---|
| Snapshot | Current month’s public state, reading, and offered discount |
| History | Snapshot set plus the last six months of readings and actions (recorded by channel) |
| True state | Snapshot set plus the hidden state |
| Contrast | What it measures |
| Separation, history snapshot | Increase in fitted predictive from the six-month window |
| Headroom, true state history | Increase in fitted predictive from adding the true state |
| Solver | Information | Model or policy | Decision rule | Horizon |
|---|---|---|---|---|
| GBM-lags | Public state, current reading, six months of readings and channels | Two calibrated GBM classifiers | Myopic or sequential, reading and history features frozen during search | 1, 6 |
| GBM-flat | Public state and current reading | As GBM-lags | As GBM-lags, without history | 1, 6 |
| CE-Seq | Current state and reading, full pre-decision history including account states | Recurrent belief with two calibrated heads | Sequential, future readings masked | 6 |
| GBM-cash | Public state and current reading | GBM cash predictor, Tweedie loss | Myopic | 1 |
| LLM-agent | Delinquency ageing and balance | Fixed rules written from censored logs | Rules applied monthly | 1 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Term | Meaning |
|---|---|
| Base, Mem-0.5, Mem-4, Noise-0.1, Noise-0.2 | the five released environments, with parameter settings in Table 5 |
| belief | the probability distribution of the hidden state given everything a solver has observed |
| certainty-equivalent (CE) | chaining monthly repayment probabilities without their cross-month dependence |
| closed-loop grading | grading in which each solver re-decides every month on the readings its own actions generate |
| decision point | month six of an evaluation loan, when a solver takes over after six months of public history, with six months left to decide |
| disclosure | the replacement of one input of a graded solver by a specified reference quantity |
| Member | Transition model | Memory | Noise | Base rate | Checks |
|---|---|---|---|---|---|
| Base | linear, deterministic | 0 | 0 | 0.038 | pass, re-anchored |
| Mem-0.5 | action-rotation | 0.5 | 0 | 0.038 | pass, one clause amended |
| Mem-4 | action-rotation | 4 | 0 | 0.026 | pass, re-anchored |
| Noise-0.1 | linear, stochastic | 0 | 0.1 | 0.038 | pass |
| Noise-0.2 | linear, stochastic | 0 | 0.2 | 0.032 | pass, re-anchored |
| Solver | Change from the row above | Value | Step | 95% interval |
|---|---|---|---|---|
| GBM-lags | none | 84.40 | ||
| exact fatigue | exact per-path fatigue instead of interpolation | 83.65 | ||
| filter features | reference-filter features instead of history | 82.97 | ||
| rolled belief | filter belief propagated instead of frozen | 78.14 | ||
| exact response | true response with hidden traits instead of fitted response | 93.75 | ||
| full disclosure | all four changes together |
| Configuration | vs history | share | at epoch cap |
|---|---|---|---|
| recurrent (GRU), small | no stopping log | ||
| recurrent (GRU), small, long budget | 27.9% | yes | |
| recurrent (LSTM), large | 23.7% | yes | |
| recurrent (GRU), large | 37.7% | yes | |
| reference filter (fitted) | 68.8% | ||
| information reference (no fit) | 100% |
| configuration | epoch cap | patience | learning rate |
|---|---|---|---|
| attention (this appendix) | 30 | 5 | |
| recurrent (Table 7 ) | 30 | 5 | |
| recurrent, long budget (Table 7 ) | 100 | 15 | |
| member | seeds | memory value | per-seed |
| Base | 2 | , | |
| Mem-0.5 | 2 | , |
| Solver | 1-month horizon | 6-month horizon |
|---|---|---|
| GBM-lags | 67.78 | 84.40 |
| GBM-flat | 65.59 | 80.65 |
| CE-Seq | 82.28 | |
| GBM-cash | 66.32 | |
| LLM-agent | 64.74 | |
| random policy | 29.11 | |
| Contrast | Points | 95% interval |
|---|---|---|
| GBM-lags, six months minus one month | ||
| GBM-flat, six months minus one month | ||
| GBM-lags minus GBM-flat, six months | ||
| GBM-lags minus CE-Seq, six months | ||
| CE-Seq minus GBM-flat, six months | ||
| GBM-lags minus GBM-flat, one month |
| Data | AUC | 95% interval | Partition range |
|---|---|---|---|
| Fannie Mae spells (real, ) | |||
| Base | |||
| Mem-0.5 | |||
| Mem-4 |
Sequential Decision Making
Sequential Decision Making studies how to choose actions over time when outcomes, information, resources, or goals can change.
See all 149 papers