Second-Order Policy Effects as State Transitions: A Source-Linked Benchmark for Policy Simulation
Organizations: The Institute of Energetic Paradigm
Abstract
Policy evaluation often estimates direct benefits and costs while treating the institutional environment as fixed. In practice, a policy changes the system it enters: actors adapt, enforcement capacity shifts, burdens move, and new equilibria form around capture, gaming, compliance theater, irreversibility, and repair costs. We formalize this as second-order policy-effect prediction and present a source-linked benchmark for policy simulation. The benchmark contains 96 named public-policy cases across eight domains and four balanced action classes: implement, modify, pilot, and block. Each case includes source locators and state variables for benefit, capture, gaming, burden shift, instability, uncertainty, irreversibility, distributional risk, and implementation capacity. The runner regenerates method outputs and aggregate results from the case table, and the simulator never reads the expert action target. We report a protocol-based transition-channel audit with recall, precision, F1-style efficiency, and selective top-channel stress diagnostics, so universal channel coverage is not mistaken for field validation. The side-effect simulator achieves mean policy-effect quality of 0.945, compared with 0.838 for the risk-register baseline and 0.879 for the causal-loop baseline. Its advantage is concentrated in side-effect recall and aggregate transition scoring; it does not dominate the best structured baselines on exact policy-action choice. The evidence remains benchmark-based, but supports a bounded claim: transition-state variables make policy simulators more sensitive to downstream institutional effects.
Figures & tables
| Method | Mean | Recall | Match | Exact |
|---|---|---|---|---|
| Baseline policy memo | 0.530 | 0.363 | 0.750 | 0.479 |
| Static cost-benefit | 0.683 | 0.508 | 0.892 | 0.708 |
| Generic LLM critique | 0.770 | 0.663 | 0.917 | 0.750 |
| Causal-loop baseline | 0.879 | 0.785 | 0.983 | 0.948 |
| Risk-register baseline | 0.838 | 0.848 | 0.892 | 0.677 |
| Side-effect simulator | 0.945 | 0.975 | 0.951 | 0.875 |
| Policy | Mean | Match | Exact | Burden |
|---|---|---|---|---|
| Always implement | 0.352 | 0.500 | 0.250 | 0.433 |
| Always modify | 0.493 | 0.667 | 0.250 | 0.641 |
| Always pilot | 0.505 | 0.667 | 0.250 | 0.641 |
| Always block | 0.381 | 0.500 | 0.250 | 0.383 |
| Majority action | 0.415 | 0.500 | 0.250 | 0.553 |
Policy Evaluation
Policy evaluation estimates the outcomes, value, risks, and fairness of decision rules before or after deployment.
See all 55 papers