Bayesian Decision Theory

Prognostic predictions from ML models support human decision making in several fields, including high-stakes ones such as healthcare and the judiciary. Yet, we still lack a clear understanding of how decision makers learn from ML-based decision support (ML-DS). In this paper, we introduce a general computational framework, the 2-Step Agent, to capture how a prediction from an ML model can also be used as a source of useful information about the training data and the causal mechanism that generated it. Our framework models (i) how a prediction from an ML model affects the beliefs of a rational Bayesian agent, and (ii) how this change in beliefs affects the estimation of the causal effect, the downstream decision for a new observation, and the subsequent outcome. In addition to the framework itself, we make three contributions. First, for the multivariate linear-Gaussian setting, we derive a tractable solution for the Bayesian inference problem introduced, i.e. one in which the agent infers information about a population from an ML prediction. Second, we experimentally identify conditions under which ML-DS is beneficial compared to unassisted decision making. Third, we show that a single misaligned prior belief of the user can be sufficient for ML-DS to lead to worse downstream outcomes compared to no decision support even when the prognostic model is well-specified and the agent is perfectly rational. Hence, even under ideal conditions, ML-DS can do more harm than good.

Complexity and limited ability affect how we learn and make decisions under uncertainty. Using finite automata to model belief formation, this paper studies the characteristics of optimal learning behavior in small and big worlds, where the complexity of the environment is low and high, respectively, relative to the cognitive ability of the decision maker. The optimal long-run average payoff and distribution of actions are well approximated by the Bayesian benchmark in very small worlds, but the gap widens as the worlds get bigger. In big worlds, it can be optimal for the decision maker to ignore some states entirely, concentrating limited cognitive resources on a subset of states. This ``optimal ignorance'' is reminiscent of well-documented non-Bayesian learning patterns such as the use of heuristics, correlation neglect, and inattentive learning. The results establish a relationship between complexity, cognitive ability, and the scope for non-Bayesian learning behavior.