stat.MLOct 18, 2026

Strictly Replicable Realizable Learning: Unknown Marginals, Finite-Class Interpolation, and the Cost of Shared Randomness

Authors: Guangjian Zhang

Abstract

Strict replicability requires two independent training samples, processed with the same random seed, to produce exactly the same hypothesis. We study realizable Boolean learning under unknown and known instance marginals. For unknown marginals, we obtain the worst-case sample complexity Θ(((log⁡M)2/ρ2+log⁡(1/δ))/ε)Θ(((\log M)^2/ρ^2+\log(1/δ))/\varepsilon) for classes of cardinality at most MM, throughout a fixed small-parameter range. This resolves the distribution-free finite-class sample-complexity question up to universal constants and rules out a general ρ−1ρ^{-1} dependence. The upper bound uses a truncated-power distribution with a hard empirical-error budget. For known marginals, a control-variate learner uses O(((1+log⁡M)3/2/ρ+log⁡(1/δ))/ε)O(((1+\log M)^{3/2}/ρ+\log(1/δ))/\varepsilon) samples, and explicit uniform-marginal instances establish the matching class-size exponent. On a product of dd finite threshold chains of length kk, we determine the exact rate Θ((dklog⁡(1+d/(kρ))+log⁡(1/δ))/ε)Θ((dk\log(1+\sqrt d/(kρ))+\log(1/δ))/\varepsilon). We also characterize deterministic learning, the very-small-ρρ regime for arbitrary finite classes, and every cardinality-replicability regime for finite classes of VC dimension one. A VC-covering-number bound yields the fixed-accuracy exponent 3/23/2 independently of cardinality. Finally, we separate statistical complexity from implementation costs: we prove a sharp fixed-block trade-off, short-seed existence bounds, and a shared rejection-filter reduction from optimal learning of the full Boolean class to efficient public sampling of ordinary Bernoulli product distributions. Detailed proofs include the geometric lower bounds, the discrete posterior-kernel comparison, and the finite-bit constructions.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 18, 2026

The Optimal Sample Complexity of Pointwise Replicable Agnostic Learning

Pointwise replicability asks that two independently trained predictors, using the same random seed, agree at each fixed query with high probability. We determine its optimal agnostic sample complexity on finite and countable domains. For every nonempty binary class of finite VC dimension d≥1d\ge1, the required number of independent examples is Θ ⁣(dρ2α2+log⁡(1/β)α2)Θ\!\left(\frac{d}{ρ^2α^2}+\frac{\log(1/β)}{α^2}\right), uniformly over independent accuracy, failure, and disagreement parameters α,β,ρ∈(0,1/10)α,β,ρ\in(0,1/10). The learner returns one binary function, may be improper, and uses unrestricted shared randomness. Its sample budget is fixed before observing data, and the replicability guarantee includes queries of zero marginal mass. The main construction combines an ensemble of empirical-risk minimizers with a smooth, signal-dependent risk decomposition. A single shared phase selects the final binary predictor. Record times certify its weak-signal risk, while a uniform bound protects the strong-signal risk at that same selected time. A phase-comparison theorem controls the actual shared-seed execution under simultaneous changes in the marginals and risk coefficients. A rare repair step then makes the confidence cost additive. Together with matching lower bounds, this removes the remaining logarithmic overhead and parameter restrictions in the general countable-domain sample-complexity problem. This extended version also retains the earlier explicit constructions and their proofs, including proper learning for VC-one classes, the full finite-atom class, a faster realizable bound, and counterexamples that separate restrictions of particular algorithms from restrictions of the learning problem.
Feb 23, 2026

The Sample Complexity of Replicable Realizable PAC Learning

In this paper, we consider the problem of replicable realizable PAC learning. We construct a particularly hard learning problem and show a sample complexity lower bound with a close to (log⁡∣H∣)3/2(\log|H|)^{3/2} dependence on the size of the hypothesis class HH. Our proof uses several novel techniques and works by defining a particular Cayley graph associated with HH and analyzing a suitable random walk on this graph by examining the spectral properties of its adjacency matrix. Furthermore, we show an almost matching upper bound for the lower bound instance, meaning if a stronger lower bound exists, one would have to consider a different instance of the problem.
Apr 12, 2026

Replicable Composition

Replicability requires that algorithmic conclusions remain consistent when rerun on independently drawn data. A central structural question is composition: given kk problems each admitting a ρρ-replicable algorithm with sample complexity nn, how many samples are needed to solve all jointly while preserving replicability? The naive analysis yields O~(nk2)\widetilde{O}(nk^2) samples, and Bun et al. (STOC'23) observed that reductions through differential privacy give an alternative O~(n2k)\widetilde{O}(n^2k) bound, leaving open whether the optimal O~(nk)\widetilde{O}(nk) scaling is achievable. We resolve this open problem and, more generally, show that problems with sample complexities n1,…,nkn_1,\ldots,n_k can be jointly solved with O~(∑ini)\widetilde{O}(\sum_i n_i) samples while preserving constant replicability. Our approach converts each replicable algorithm into a perfectly generalizing one, composes them via a privacy-style analysis, and maps back via correlated sampling. This yields the first advanced composition theorem for replicability. En route, we obtain new bounds for the composition of perfectly generalizing algorithms with heterogeneous parameters. As part of our results, we provide a boosting theorem for the success probability of replicable algorithms. For a broad class of problems, the failure probability appears as a separate additive term independent of ρρ, immediately yielding improved sample complexity bounds for several problems. Finally, we prove an Ω(nk2)Ω(nk^2) lower bound for adaptive composition, establishing a quadratic separation from the non-adaptive setting. The key technique, which we call the phantom run, yields structural results of independent interest.