cs.LGOct 30, 2026

New Lower Bounds for Private PAC Learning

Authors: Prathamesh Dharangutte, Amartya Sanyal

Organizations: Department of Computer Science, University of Copenhagen

Abstract

We study realizable PAC learning under approximate differential privacy, and give a framework for proving lower bounds on its sample complexity. We construct classes with identical VC and Littlestone dimensions but different private sample complexities at common privacy and accuracy parameters, showing that these dimensions do not determine the sample complexity, even for improper learners. To address this we introduce the Independent Class Packing (ICP) parameter, the largest total sample complexity of classes in G\mathcal{G} that can be packed independently into a class H\mathcal{H}, and prove that every (ε,δ)(\varepsilon, δ)-DP, (α,β)(α, β)-PAC learner for H\mathcal{H}, possibly improper, uses Ω(ICP(H,G)/(αε))Ω\left(\mathrm{ICP}(\mathcal{H}, \mathcal{G})/(α\varepsilon)\right) examples. Using this framework we prove tight lower bounds for axis-aligned rectangles, union of intervals and linear subspaces and improved lower bounds for halfspaces and convex polygons. Finally we also show that for product classes, and more generally when the same classes pack into H\mathcal{H} and cover its domain so that target labels can be used directly, ICP characterizes private PAC sample complexity up to logarithmic factors.

Figures & tables

Explore similar work

CardsList
  1. Nearly Linear Private Learning of Littlestone Classes

    Oct 16, 2026Shay Moran, Chao YanDifferential PrivacyPAC Learning

  2. Private Component-by-Component Learning

    Oct 4, 2026Dvir Karni, Eliad TsfadiaBoolean Function LearningPrivacy-Preserving ML

  3. Private online learning and prediction for Littlestone classes

    Oct 5, 2026Amartya SanyalPrivacy-Preserving MLDifferential Privacy