Federated Measurement of Demographic Disparities from Quantile Sketches

Arthur Charpentier; Agathe Fernandes Machado; Olivier Côté; François Hu

stat.MLFeb 21, 2026

Federated Measurement of Demographic Disparities from Quantile Sketches

Authors: Arthur Charpentier, Agathe Fernandes Machado, Olivier Côté, François Hu

Organizations: UQAM, Montréal, Canada · Kyoto University, Japan · Université Laval, Québec, Canada · Milliman, Paris, France ·

Abstract

Many fairness goals are defined at a population level that misaligns with siloed data collection, which remains unsharable due to privacy regulations. Horizontal federated learning (FL) enables collaborative modeling across clients with aligned features without sharing raw data. We study federated auditing of demographic parity through score distributions, measuring disparity as a Wasserstein--Frechet variance between sensitive-group score laws, and expressing the population metric in federated form that makes explicit how silo-specific selection drives local-global mismatch. For the squared Wasserstein distance, we prove an ANOVA-style decomposition that separates (i) selection-induced mixture effects from (ii) cross-silo heterogeneity, yielding tight bounds linking local and global metrics. We then propose a one-shot, communication-efficient protocol in which each silo shares only group counts and a quantile summary of its local score distributions, enabling the server to estimate global disparity and its decomposition, with $O(1/k)$ discretization bias ( $k$ quantiles) and finite-sample guarantees. Experiments on synthetic data and COMPAS show that a few dozen quantiles suffice to recover global disparity and diagnose its sources.

Figures & tables

Panel of Figure 1 — Figure 1: Synthetic Beta. Left : score distributions by group, $U_{2}=0.0076$ . Right : $U_{2}(k)$ with $k\in\{1,2\}$ as a function of $\rho$ .

Panel of Figure 2 — Figure 2: Sensitivity to $k$ (synthetic). Convergence of $U_{2}(k)$ , independence allocation (left) and selection bias (right).

	Global	Silo 1	Silo 2	Silo 3	Silo 4	Silo 5
$n_{\text{AA}}$	3696	535	704	745	814	898
$n_{\text{C}}$	2454	695	536	521	407	295
$\|\Delta\bar{z}\|$	1.6357	0.4483	0.9784	1.0978	0.9667	0.5654
$U_{2}$	0.7609	0.0748	0.2634	0.3073	0.2355	0.0745
$H_{2}$	0.0077	0.0017	0.0045	0.0044	0.0035	0.0012
$W_{2}$	1.7813	0.5479	1.0324	1.1319	1.0104	0.6326

Table 1: COMPAS , simulated silos with selection bias. African-American (AA) vs. Caucasian (C) scores, with

d=5

silos.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

\widehat{F}_{j,s,k}(x):=\frac{1}{k}\sum_{\ell=1}^{k}\mathbf{1}\{\widehat{q}_{j,s,\ell}\leq x\}.

Appendix

Algorithm 1 One-shot federated audit from

k

quantiles.

\widehat{G}_{k,p}:=\sum_{s\in\mathcal{S}}\widehat{\alpha}_{s}\cdot\frac{1}{k}\sum_{\ell=1}^{k}\big|\widehat{q}^{\times}_{s,\ell}-\widehat{q}^{\times*}_{\ell}\big|^{p}.

Appendix

Table 5

Panel of Figure 3 — Figure 3: COMPAS: score distributions and Wasserstein barycenter. Left: (beta-)kernel density estimates of the jittered score $Z$ by group. Middle: empirical CDFs $F_{\mathrm{AA}}$ and $F_{\mathrm{C}}$ together with the Wasserstein barycenter distribution (dashed), obtained by inverting the barycenter quantile $Q^{\star}$ . Right: group quantile functions $Q_{\mathrm{AA}}$ and $Q_{\mathrm{C}}$ and their Wasserstein barycenter $Q^{\star}=\alpha_{\mathrm{AA}}Q_{\mathrm{AA}}+\alpha_{\mathrm{C}}Q_{\mathrm{C}}$ (dashed).

Random assignment (independence)
metric	Global	Silo 1	Silo 2	Silo 3	Silo 4	Silo 5
$\|\Delta\bar{z}\|$	1.6357	1.5583	1.5732	1.7513	1.7481	1.5675
$U_{2}$	0.7609	0.6642	0.7069	0.8824	0.8652	0.7296
$H_{2}$	0.0077	0.0068	0.0071	0.0092	0.0088	0.0073
$W_{2}$	1.7813	1.6769	1.7313	1.9099	1.8858	1.7404
Negative selection bias

Appendix

Table 2: COMPAS: random vs. selection-biased silo allocations. Global and per-silo summary statistics for African-American (AA) vs. Caucasian (C) scores under three regimes: random assignment (independence), negative selection bias, and positive selection bias (Gaussian-copula assignment with fixed group-wise margins

(N_{\ell,g})

). Rows report the mean-score gap

|\Delta\bar{z}|

and distributional distances (

U_{2}

H_{2}

W_{2}

Panel of Figure 4 — Figure 4: COMPAS: original score distributions and Wasserstein barycenter. Left: histograms of score $Z$ by group. Middle: empirical CDFs $F_{\mathrm{AA}}$ and $F_{\mathrm{C}}$ together with the Wasserstein barycenter distribution (dashed), obtained by inverting the barycenter quantile $Q^{\star}$ . Right: group quantile functions $Q_{\mathrm{AA}}$ and $Q_{\mathrm{C}}$ and their Wasserstein barycenter $Q^{\star}=\alpha_{\mathrm{AA}}Q_{\mathrm{AA}}+\alpha_{\mathrm{C}}Q_{\mathrm{C}}$ (dashed).

Figure 5: COMPAS: convergence in $k$ . Mean absolute error $\mathrm{MAE}(k)=\mathbb{E}\,|\widehat{U}_{2}(k)-U_{2}|$ as a function of the number of quantiles $k$ (sent per group and per silo) on a log scale, for several numbers of silos $d$ and for different allocation regimes (random vs. selection bias). Across regimes, $\widehat{U}_{2}(k)$ stabilizes quickly, with diminishing returns beyond a few dozen quantiles.

Figure 6: COMPAS: fast stabilization of $\widehat{U}_{2}(k)$ . Boxplots of the federated estimator $\widehat{U}_{2}(k)$ over repeated allocations (Monte Carlo scenarios), for a fixed number of silos $d$ and varying $k$ . The dashed horizontal line is the centralized benchmark $U_{2}$ computed on the pooled data with a fine grid. Even for moderate $k$ , the distribution of $\widehat{U}_{2}(k)$ concentrates near $U_{2}$ .

Figure 7: COMPAS: reliability profile. Probability of meeting a relative-error target

p_{\mathrm{ok}}(k)=\mathbb{P}\!\left(\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}\leq\tau\right)

(here

\tau=1\%

) as a function of

k

(on a log scale), for several

d

and allocation regimes. This complements MAE curves by showing how quickly the bulk of runs reaches the target accuracy.

Figure 8: COMPAS: minimal sketch size for a target reliability. Heatmap of $k_{95}$ , the smallest number of quantiles $k$ such that $p_{\mathrm{ok}}(k)\geq 95\%$ (with $\tau=1\%$ ), as a function of the number of silos $d$ and the allocation regime. Selection bias requires larger sketches to achieve the same reliability.

Panel of Figure 9 — Figure 9: COMPAS: error versus selection intensity. ( $k=10$ top and $k=20$ bottom ) Relative error $\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}$ versus the realized score–silo dependence, measured by $|\mathrm{cor}_{Z}(Z,A)|$ (Spearman correlation). Stronger selection (higher dependence) tends to increase estimation error at fixed sketch size, consistent with more heterogeneous within-silo score distributions.

Figure 10: COMPAS: centralized discretization vs. federated estimation. Comparison of the centralized discretized target $U_{2}(k)$ (computed from pooled data on a $k$ -point grid) and the federated estimator $\widehat{U}_{2}(k)$ (mean over scenarios, with variability bands). The horizontal reference is $U_{2}$ computed on a fine grid. The closeness of the curves for moderate $k$ indicates that discretization and sketch resolution are the main drivers, beyond federation noise.

Figure 11: COMPAS: accuracy–communication trade-off. Relative error $\mathbb{E}\!\left[\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}\right]$ versus a simple communication proxy $\text{budget}=2d(k+1)$ scalars (two groups, $d$ silos, and $(k+1)$ quantiles per group and silo). For fixed $d$ , increasing $k$ quickly improves accuracy; for fixed $k$ , larger $d$ may increase noise through smaller local samples, highlighting a practical trade-off between federation granularity and sketch resolution.

Random assignment (independence)
metric	Global	Silo 1	Silo 2	Silo 3	Silo 4	Silo 5
$\|\Delta\bar{z}\|$	0.1338	0.1245	0.1458	0.1410	0.1366	0.1203
$U_{2}$	0.0029	0.0025	0.0034	0.0035	0.0032	0.0022
$H_{2}$	0.0021	0.0019	0.0024	0.0023	0.0021	0.0017
$W_{2}$	0.1821	0.1705	0.1989	0.1945	0.1869	0.1611
Negative selection bias

Appendix

Table 3: Adult: random vs. selection-biased silo allocations. Global and per-silo summary statistics for AA vs. C scores under three regimes: random allocation (independence), negative selection bias, and positive selection bias (Gaussian-copula assignment with fixed group-wise margins). Rows report the mean-score gap

|\Delta\bar{z}|

and distributional distances (

U_{2}

H_{2}

W_{2}

Figure 12: Adult: convergence in $k$ . Mean absolute error $\mathrm{MAE}(k)=\mathbb{E}\,|\widehat{U}_{2}(k)-U_{2}|$ as a function of the number of quantiles $k$ (sent per group and per silo) on a log scale, for several numbers of silos $d$ and for different allocation regimes.

Figure 13: Adult: stabilization of $\widehat{U}_{2}(k)$ . Boxplots of $\widehat{U}_{2}(k)$ over repeated allocations for a fixed $d$ and varying $k$ . The dashed horizontal line is the centralized benchmark $U_{2}$ .

Figure 14: Adult: reliability profile.

p_{\mathrm{ok}}(k)=\mathbb{P}\!\left(\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}\leq\tau\right)

(here

\tau=1\%

) as a function of

k

(log scale), for several

d

and allocation regimes.

Figure 15: Adult: minimal sketch size for a target reliability. Heatmap of $k_{95}$ , the smallest $k$ such that $p_{\mathrm{ok}}(k)\geq 95\%$ (with $\tau=1\%$ ), as a function of $d$ and the allocation regime.

Panel of Figure 16 — Figure 16: Adult: error versus selection intensity. ( $k=10$ top, $k=20$ bottom) Relative error $\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}$ versus $|\mathrm{cor}_{Z}(Z,A)|$ (Spearman).

Figure 17: Adult: centralized discretization vs. federated estimation. Centralized $U_{2}(k)$ (pooled data on a $k$ -grid) versus federated $\widehat{U}_{2}(k)$ (mean over scenarios with variability bands). The horizontal reference is $U_{2}$ computed on a fine grid.

Figure 18: Adult: accuracy–communication trade-off. $\mathbb{E}\!\left[\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}\right]$ versus $\text{budget}=2d(k+1)$ .

Random assignment (independence)
metric	Global	Silo 1	Silo 2	Silo 3	Silo 4	Silo 5
$\|\Delta\bar{z}\|$	0.2175	0.2442	0.2071	0.2223	0.1948	0.2202
$U_{2}$	0.0036	0.0043	0.0036	0.0036	0.0030	0.0037
$H_{2}$	0.0064	0.0085	0.0065	0.0066	0.0063	0.0065
$W_{2}$	0.2453	0.2712	0.2402	0.2478	0.2204	0.2477
Negative selection bias

Appendix

Table 4: Law: random vs. selection-biased silo allocations. Global and per-silo summary statistics for AA ( black ) vs. C ( white ) scores under three regimes: random assignment (independence), negative selection bias, and positive selection bias (Gaussian-copula assignment with fixed group-wise margins). Rows report the mean-score gap

|\Delta\bar{z}|

and distributional distances (

U_{2}

H_{2}

W_{2}

Figure 19: Law: convergence in $k$ . Mean absolute error $\mathrm{MAE}(k)=\mathbb{E}\,|\widehat{U}_{2}(k)-U_{2}|$ as a function of the number of quantiles $k$ (sent per group and per silo) on a log scale, for several numbers of silos $d$ and for different allocation regimes.

Figure 20: Law: stabilization of $\widehat{U}_{2}(k)$ . Boxplots of $\widehat{U}_{2}(k)$ over repeated allocations for a fixed $d$ and varying $k$ . The dashed horizontal line is the centralized benchmark $U_{2}$ .

Figure 21: Law: reliability profile.

p_{\mathrm{ok}}(k)=\mathbb{P}\!\left(\frac{|\widehat{U}_{2}(k)-U_{2}|}{U_{2}}\leq\tau\right)

(here

\tau=1\%

) as a function of

k

(log scale), for several

d

and allocation regimes.

Figure 22: Law: minimal sketch size for a target reliability. Heatmap of $k_{95}$ , the smallest $k$ such that $p_{\mathrm{ok}}(k)\geq 95\%$ (with $\tau=1\%$ ), as a function of $d$ and the allocation regime.

Group Fairness

Group fairness measures and reduces disparities in how algorithms predict, rank, allocate resources, and make decisions for different groups.

See all 140 papers

Multi-Sensitive Distributional Fairness: Joint Wasserstein Audits and Shapley Allocation
Oct 13, 2026Agathe Fernandes Machado, Charlotte Jamotton, Arthur CharpentierGroup Fairness Intersectional Fairness
FAIRVAR: Fair Federated Learning via Variance Regularization
Aug 16, 2025Zahra Kharaghani, Ali Dadras, Tommy LöfstedtAlgorithmic Fairness Federated Learning
Department of Computing Science, Umeå University, Umeå, Sweden · Department of Mathematics and Mathematical Statistics, Umeå University, Umeå, Sweden
Assessing the Impacts of Imperfect Datasets on Client Selections in Federated Learning
Aug 3, 2026Yuan-Heng Tsai, Li-Hsing Yen, Yan-Wei ChenAlgorithmic Fairness Client Selection in Federated Learning
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan.

Oct 13, 2026J/K move · Enter open · S save

Multi-Sensitive Distributional Fairness: Joint Wasserstein Audits and Shapley Allocation

Agathe Fernandes Machado, Charlotte Jamotton, Arthur Charpentier

Group Fairness Intersectional Fairness Algorithmic Auditing Shapley Value Attribution Wasserstein Distance

Open paper AskPDF arXiv

Fairness audits with several sensitive attributes are often performed one coordinate at a time. This can miss purely intersectional disparities: a score may be independent of each sensitive coordinate while depending on their joint profile. We study distributional demographic parity for a real-valued score using one-dimensional Wasserstein geometry. We distinguish two complementary constructions. Conditional and sequential audits characterize joint parity through their zero sets, but their positive values do not generally decompose a fixed joint Wasserstein disparity. By contrast, full-cell conditional quantile functions define a Hilbert-valued random variable whose variance is the joint disparity. Conditional expectations of this quantile process induce a monotone information game: every reveal order yields an exact martingale decomposition of the same joint disparity, while Shapley values provide an order-free attribution across sensitive attributes. Under independent sensitive attributes, the game reduces to a Hoeffding-ANOVA decomposition and Shapley splits each interaction energy equally among its members; under dependence, orthogonality is lost and explicit formulas reveal suppression and redundancy effects that marginal audits can hide. We give fixed-grid estimators and permutation calibration, and illustrate the framework on synthetic examples and ACSIncome. The resulting framework separates three questions that are often conflated: whether joint distributional parity holds, how a fixed joint disparity decomposes along an information path, and how that disparity can be attributed across sensitive coordinates without privileging an ordering.

Abstract

Group Fairness

Explore similar work

Multi-Sensitive Distributional Fairness: Joint Wasserstein Audits and Shapley Allocation

FAIRVAR: Fair Federated Learning via Variance Regularization

Assessing the Impacts of Imperfect Datasets on Client Selections in Federated Learning