Strictly Replicable Realizable Learning: Unknown Marginals, Finite-Class Interpolation, and the Cost of Shared Randomness
Abstract
Strict replicability requires two independent training samples, processed with the same random seed, to produce exactly the same hypothesis. We study realizable Boolean learning under unknown and known instance marginals. For unknown marginals, we obtain the worst-case sample complexity for classes of cardinality at most , throughout a fixed small-parameter range. This resolves the distribution-free finite-class sample-complexity question up to universal constants and rules out a general dependence. The upper bound uses a truncated-power distribution with a hard empirical-error budget. For known marginals, a control-variate learner uses samples, and explicit uniform-marginal instances establish the matching class-size exponent. On a product of finite threshold chains of length , we determine the exact rate . We also characterize deterministic learning, the very-small- regime for arbitrary finite classes, and every cardinality-replicability regime for finite classes of VC dimension one. A VC-covering-number bound yields the fixed-accuracy exponent independently of cardinality. Finally, we separate statistical complexity from implementation costs: we prove a sharp fixed-block trade-off, short-seed existence bounds, and a shared rejection-filter reduction from optimal learning of the full Boolean class to efficient public sampling of ordinary Bernoulli product distributions. Detailed proofs include the geometric lower bounds, the discrete posterior-kernel comparison, and the finite-bit constructions.
Figures & tables
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Result | Guarantee and role | Statement or complete argument |
|---|---|---|
| Foundational results | ||
| Anchored obstruction | Unknown-marginal lower bound, with accuracy failure separated from the replication denominator. | Theorems 3.2 and C.2 |
| Compact geometry | Self-contained compact Brunn–Minkowski input for output-cell expansion. | Appendix B |
| Gibbs learner | Finite-class Gibbs upper bound and its KL–generalization analysis. | Propositions C.5 and C.3 |
| Poisson–ball learner | Full Boolean class upper bound, ball coupling, truncation, and fallback. | Propositions C.7 and C.4 |
| Known full Boolean class | Matching known-marginal rate and its contrast with the unknown-marginal rate. | Sections 3.6 and C.6 |
Sample Complexity
Sample complexity studies how many data points, queries, or observations are needed to learn, estimate, test, or sample to a target accuracy.
See all 102 papers