cs.LGOct 31, 2026

Beyond Blackwell: Constrained Data Processing Inequalities

Authors: Brendan van Rooyen

Organizations: Epigraph Solutions

Abstract

We study the comparison of statistical experiments from a decision-theoretic standpoint, working with Markov transitions as basis-free linear maps on dual spaces. Our starting observation is that the data-processing inequality, that adding noise to an experiment cannot lower its Bayes risk, is the functoriality of fibrewise minimisation over a presheaf of decision rules: the rules available after an experiment form a presheaf, garbling acts on it by precomposition, and a risk functional is a natural transformation from it to the ordered reals. The argument refers to neither probabilities nor expected values, and it survives replacing the decision presheaf by a sub-presheaf of admissible rules. Any constraint imposed on the strategy that a rule induces fairness, unbiasedness, is automatically of this form, and so yields a data-processing inequality with no counterpart in the unconstrained Blackwell theory. Our main result is their quantitative companion. For constraints closed under post-processing we define a constrained deficiency and prove a constrained randomization theorem: one experiment dominates another on every admissible decision problem exactly when the constrained deficiency is small. The classical theorem is the case of no constraint, where the identity is itself an admissible rule; in general it is not, and the constrained deficiency is an optimisation over the coarsenings of an experiment that render it compliant. Specialising to equalized odds, we exhibit a pair of experiments that unconstrained Bayes risk identifies at a given loss and prior but that the constrained risk separates, giving a one-line certificate of Blackwell-incomparability. We work throughout with finite sets.

Explore similar work

Aug 26, 2026

Comparing Corrupted Constrained Learning Problems

A key result in statistics is the data processing inequality, originally proved by Blackwell (1951) and later refined by DeGroot (1962) in terms of statistical uncertainty. The latter statement claims that the Bayes risk achieved on raw data always undercuts the Bayes risk on a processed version of the same data. This seemingly contradicts empirical findings in machine learning: pre-training, representation learning, feature learning, data augmentation are techniques used to improve performance of machine learning models. We reconcile both worlds by simply accepting that machine learning problems are constrained learning problems: the model class used does not include all measurable functions. We present counterexamples showing that the classical data processing inequality fails to hold in such a setting. Hence, we formulate a generalized data processing inequality, requiring the constrained Bayes risk of a joint distribution (with respect to a loss function and a constrained model class) to lower bound the constrained Bayes risk on the stochastically modified data distribution, regardless of the choice of distribution. We show this inequality to be equivalent to a set containment condition on a specific function set induced by the loss and model class, called the superprediction set. Finally, we exploit our characterization, derive sufficient conditions for this containment and quantify the inequality-gap.
Feb 6, 2025

Targeted Learning for Inference on Data Fairness

As algorithmic decision making has become increasingly ubiquitous even in high-stakes settings, significant effort has been invested in defining, detecting, and eliminating unfair model-driven outcomes. Prior work has largely focused on estimating the fairness properties of a given predictive model, and quantifying uncertainty based on variability in a fixed model's performance across evaluation datasets. Alternatively, we can also ask about uncertainty that the model inherits from its training data; here we evaluate data fairness - fairness in the data generating process used for training. We perform inference on data fairness using targeted learning, a flexible framework for nonparametric inference, and derive estimators for demographic parity, equal opportunity, and conditional mutual information. Simulations show that our data fairness estimators improve confidence-interval coverage and reduce false detections of population unfairness relative to fixed-model inference. We also demonstrate their application to real data.
May 9, 2026

The Pokémon Theorem: Fairness Criteria Can't Catch 'Em All

Debates around algorithmic fairness rarely end. Models considered "fair" by one criterion are soon shown to be unfair by another. We prove that this is unavoidable. Satisfying any finite checklist of fairness criteria never certifies fairness itself, since the model may always violate yet another criterion. We call this the Pokémon theorem: you can't catch them all. In fact, trying to catch all criteria recovers the classical impossibility theorems. Since these violations come from differences between the groups' data, a natural remedy would be to make the data fair by learning fair features. Unfortunately, this, too, is impossible: features that scrub out sensitive group information also scrub out all predictive signal. While exact fairness is out of reach, approximate fairness is not. We derive the Pareto frontier between fairness and predictive signal for approximately fair features, and show how to attain it.