cs.CRNov 16, 2026

Verifiable Random Sampling of Private Data Without a Trusted Single Party

Authors: Muhammed Karamuk, Mikołaj Glinka, Ruben Wolff

Organizations: Forest AI

Abstract

Problem: When a party supports claims about a private dataset by disclosing a sample, auditors cannot tell whether it was drawn at random or "cherry-picked" to hide flaws in the rest of the data. Solution: The data holder commits to the hashes of every item in its dataset and registers the commitment with independent servers. Selection then waits on a decentralized randomness beacon: a network that jointly emits a fresh random value every few seconds, outside the control of the data holder and the auditors. The value emitted two beacon rounds after the commitment picks the sample. Each server returns a signed receipt, so anyone can later check that the revealed sample is exactly the one that value prescribes. Selection from the committed set stays unbiased as long as the beacon's threshold holds and one required server is honest, and a pre-agreed name, checked with the servers, stops repeated or abandoned draws. We call this protocol NoCherry. Applications: Any audit that spot-checks a private corpus: private AI benchmarks and held-out test sets, dataset quality claims, code and security audits, test-fairness review, vulnerability handling, and compliance sampling. Auditors need not trust the data holder to hand over a representative sample of the committed data, since the protocol proves it, and data holders need not expose their entire dataset.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Apr 12, 2026

Differentially Private Verification of Distribution Properties

A recent line of work initiated by Chiesa and Gur and further developed by Herman and Rothblum investigates the sample and communication complexity of verifying properties of distributions with the assistance of a powerful, knowledgeable, but untrusted prover. In this work, we initiate the study of differentially private distribution property verification. After all, if we do not trust the prover to help us with verification, why should we trust it with our sensitive sample? We map a landscape of differentially private verification of properties of distributions. In the non-private case it is known that one-round private-coin protocols can have substantially lower complexity than public-coin (AM) protocols. In contrast, the possibility for improvement in differentially private interactive proofs depends on the privacy parameter regime and model. Drawing on connections between privacy and replicability and privacy amplification techniques in the literature we show: 1. There exists a reduction from any one-round (ε,δ)(\varepsilon,δ)-differentially private private-coin protocol to a differentially private AM protocol for the parameter regime ε=O(1/s)\varepsilon = O(1/\sqrt{s}) and δ=O(1/s5/2)δ= O(1/s^{5/2}) with approximately the same privacy and sample and communication complexities. In the local model, this is relaxed to ε=O(1/log⁡s)\varepsilon = O(1/\sqrt{\log s}) 2. However, when the privacy guarantee is very relaxed (ε∈Ω(log⁡s)\varepsilon \in Ω(\log s)), private coins indeed reduce sample and communication complexities. We also obtain a computationally efficient Merlin-Arthur proof for privately testing whether samples are drawn from a product distribution and prove that its sample complexity is optimal up to a polylogNpolylog N factor by reducing uniformity testing to independence testing with Boolean attributes and appealing to known lower bounds on sample complexity for private uniformity testing.
Sep 11, 2026

Canaries in the Bank: Auditing User-Level Privacy in Private Evolution

Private Evolution (PE) generates high-fidelity synthetic data in federated settings without exposing users' raw data. It aggregates clipped user votes over a shared candidate bank into a differentially private histogram, with noise calibrated to the worst-case user contribution. However, it is unclear whether an adversary can realize this worst-case privacy loss while following the PE protocol. We introduce a protocol-aware empirical audit in which the server commits to a single shared candidate bank and replaces roughly 1% of its entries with probes derived from a known, non-private canary. We evaluate eight attacks, including an unchanged-bank baseline, exact copies, plausible paraphrases, and high-entropy synthetic nonces. Experiments on Yelp and Sentiment140 show that natural-text attacks remain substantially below the theoretical DP bound, while nonce-based attacks yield considerably stronger bounds and come closest to the mechanism's privacy ceiling. These results quantify the gap between formal worst-case privacy and leakage achievable through protocol-valid candidate-bank manipulation.
Oct 7, 2026

Shared and structured inputs undermine collective random choice by reasoning AI agents

Random selection is widely used in resource allocation and auditing, making reliable implementation essential for AI-agent systems. Behavioural tests across six reasoning models uncovered threshold and divisibility rules used in identifier-based choices. For threshold-following GPT-6 Sol and Gemini 3.8 Flash, single-agent measurements prospectively predicted correlated participation under shared identifiers and biased participation under distinct identifiers with common timestamp bits. Changing dates, formats and identifier labels revealed when these predictions held. Explicit instructions to randomize independently reduced but did not eliminate shared-input correlation. To test implications for oversight, we asked four models to select customer requests randomly for human review. GPT-6 Sol approached the target rate while selecting predictably from identifiers; the others rarely selected requests. All four closely followed supplied random draws. These findings expose collective and audit vulnerabilities that selection rates alone miss, making input-dependent bias, correlation and predictability central targets for agent evaluation.