Organizations: 1DBA-BIDA (Business Intelligence and Data Analytics) Program, Westcliff University, USA · Department of Computer Science and Engineering, Northern University Bangladesh, Bangladesh · 3Washington University of Science and Technology, Alexandria, Virginia, USA · 4Stanton University, Los Angeles, CA, USA
Abstract
Deploying ML models in regulated decision-making (credit underwriting, fraud detection, loan approval) requires demonstrating fairness and robustness to auditors without exposing model weights or customer data. We address this attestation problem for U.S. bank supervision under SR 11-7 and OCC 2011-12 guidance. We present a chunked zero-knowledge circuit design that attests an aggregated fairness statistic - the demographic-parity gap - on committed, nonce-sampled batches of real 2022 HMDA mortgage data, and demonstrate it end-to-end: 32,768 rows, 32 independently verified zkSNARK proofs, aggregated attested gap within 0.0029 of the true held-out value, per-chunk proving under 4 seconds. We also demonstrate extensibility by attesting a second control on the identical architecture - expected calibration error at 10 bins - with all 32 chunks verified, per-chunk proving at about 14.7 seconds, and attested ECE within 0.00037 of the plaintext value on the same committed rows. We compare this design against two alternatives - a flat summation circuit, which overflows past a few thousand rows, and a tree-reduction circuit, numerically exact but intractable to compile - and find the chunked design is the only one that reached this scale. We discovered and root-caused a genuine data-quality failure (a sentinel-code outlier distorting circuit proving and the fairness statistic itself) and resolved it with a published preprocessing specification. We also propose a fuller mapping from SR 11-7 and OCC 2011-12 control language to zero-knowledge statements (nine control elements spanning soundness, calibration, robustness, and drift), a nonce-based sampling protocol resisting bank-side cherry-picking, and a threat model for deployment, proposed as design work, not implemented results. Two controls, one model class, one task are demonstrated end-to-end; the rest is scoped and left as future work.
External evaluations are becoming increasingly central to the governance of AI systems. In practice, however, independent auditors often have limited access to deployed models and must rely on query-based interactions. Most existing fairness evaluation methods assume static datasets and fixed-sample statistical tests, making them poorly suited to real-world auditing scenarios in which evidence must be collected sequentially under query constraints. In this work, we formulate fairness auditing as a tolerance-aware sequential hypothesis-testing problem under limited model output access. We develop a sequential generalized likelihood-ratio framework that allows auditors to accumulate evidence from a finite audit pool and stop once sufficient support for compliance or violation has been obtained. The framework is instantiated for decision-based Statistical Parity and Equal Opportunity audits, and extended to score- and logit-based proxy audits when richer observables are available. Our results show that both the fairness metric and the level of model access significantly affect audit efficiency, and that the benefits of richer output information are not uniform across auditing settings. In particular, richer outputs can substantially reduce the number of queries required for some fairness metrics and operating regimes, while offering limited gains in near-threshold cases. This work provides a practical statistical framework for sequential fairness auditing under realistic deployment constraints.
Ioannis Pitsiorlas, Martha V. Sourla, Marios Kountouris
Privacy-preserving machine learning auditing protocols allow auditors to assess models for properties such as accuracy or fairness, without revealing their internals or training data. This makes them especially attractive for auditing models deployed in sensitive domains such as healthcare or finance. For these protocols to be meaningful in real-world audit settings, though, their guarantees must reflect how the model will behave once deployed, rather than merely certifying its behavior during an audit. Existing security definitions often miss this mark: most certify model behavior only on a fixed audit dataset, without ensuring that the same guarantees generalize to other datasets drawn from the same distribution. As we show, this gap allows a model provider to attack many cryptographic model certification (CMC) schemes built on secure zero knowledge proofs (ZKP) by carefully engineering training data, resulting in models that exhibit benign behavior during an audit, but pathological behavior in practice. For example, we empirically demonstrate that an attacker can certify that a model achieves over 99% accuracy on an audit dataset, but less than 30% accuracy on fresh samples from the same distribution. To address this gap, we formalize rigorous cryptographic security notions tailored to CMC frameworks, introduce a generic protocol template, and prove that it satisfies these requirements. Our results thus offer both cautionary evidence about existing approaches and constructive guidance for designing secure, privacy-preserving ML auditing protocols.
Carter Luck, Olive Franzese-McLaughlin, Elisaweta Masserova +3
Machine learning systems deployed for credit, hiring, and resource distribution are increasingly subject to regulatory oversight from policies such as the EU AI Act and GDPR. Current fairness governance practices rely on observational fairness metrics, post-hoc explainability, and immutable audit logs, but provide limited support for causal attribution and efficient evidentiary verification. We introduce Causal Evidentiary Governance (CEG), a framework in which regulated institutions commit to a versioned directed acyclic graph (DAG) that partitions causal pathways into allowable and disallowed groups. The Causal Harm Rate measures prediction variation attributable to disallowed causal pathways. Each decision is accompanied by a signed Decision-Evidence Packet (DEP), cryptographically binding the prediction to a digest of the published DAG and path-specific attributions. DEP digests can be appended to a Merkle tree to enable logarithmic-cost inclusion proofs. We validate CEG through a two-layer empirical methodology using demographic summaries from four years of PMA credit supervisory data to construct 10,000 synthetic credit applicants across four strategic DAG counterfactuals. Causal Harm Rate isolates injected causal effects more clearly than demographic parity or equalized odds. Cross-model validation and ablation studies assess robustness. Evaluation on the German Credit dataset shows that harm associated with specific causal pathways can be substantially understated by associational fairness metrics. Finally, a proof-of-concept implementation demonstrates operationally plausible throughput and highlights relevant performance tradeoffs.