cs.LGJul 12, 2026

The VC dimension of partial concept classes via Radon's theorem

Authors: Grigory IvanovAttila JungMárton Naszódi

Abstract

Following Alon, Hanneke, Holzman, and Moran (FOCS 2021), we define a partial concept class (PCC) as a family of partial functions f:V{0,1,}f: V\to\{0,1,\ast\}; equivalently, its concepts partition the ground set into black (f1(1)f^{-1}(1)), grey (f1()f^{-1}(\ast)), and white parts (f1(0)f^{-1}(0)). Its VC dimension is defined by shattering sets on which the value \ast is not taken. We study two geometric PCCs in real Banach spaces, both with a margin δ>0δ>0: expanded half-spaces, where the grey part is a strip of width at least δδ adjacent to a half-space, and expanded balls, where the grey part is an annulus of width δδ around a unit radius ball. Our main results are dimension-free upper bounds on the VC dimension of the PCC of expanded balls in Lp\parenthμL_p\parenthμ, 1p<1\le p<\infty, including the non-Euclidean and algorithmically particularly relevant case 1d\ell^d_1. These bounds depend on the margin and on the radii, but not on the ambient dimension or the underlying measure space. These are extensions of the work of Bourneuf, Charbit, and Thomassé (FOCS 2025) who studied the PCC of expanded balls in Euclidean space, that is, 2d\ell_2^d. We also prove lower bounds on the VC dimension that match the upper bounds in terms of the margin parameter δδ. Finally, we derive a Dense Neighborhood Lemma in LpL_p-spaces, again extending the known Euclidean results. Our method relies on the linearization of the distance through a map into a space of non-trivial Rademacher type, and then the use of a balanced signed-sum estimate, or a no-dimensional Radon theorem. The arguments rely on ideas from functional analysis that are clearly explained for the non-expert in that field.

Explore similar work

Aug 31, 2026math.LO

A Borel Concept Class of VC Dimension One with a Non-PAC Consistent Learner in ZFC

The fundamental theorem of statistical learning states that, under suitable measurability assumptions, finite Vapnik--Chervonenkis (VC) dimension guarantees that every proper consistent learning rule is probably approximately correct (PAC). Blumer, Ehrenfeucht, Haussler, and Warmuth showed, assuming the Continuum Hypothesis, that the "well-behavedness" condition of the concept class cannot be omitted: they constructed a concept class of Borel sets of VC dimension one admitting a consistent learning rule that is not PAC. We show that the Continuum Hypothesis is unnecessary. Working in Zermelo--Fraenkel set theory with the Axiom of Choice (ZFC) alone, we construct a concept class of Borel sets on [0,1][0,1] of VC dimension one and a proper consistent learning rule that is not PAC. More precisely, for a suitable Borel probability measure and target concept, the rule has true risk one at every sample size on a set of samples of outer probability one. Consequently, finite VC dimension and Borel measurability of the individual concepts do not suffice to guarantee that every proper consistent learning rule is PAC. The result shows, with no need of extra set-theoretical assumptions, that the additional regularity assumption in the fundamental theorem cannot in general be omitted.
Mateus Jesus de Arruda Campos, Gabriel Fernandes, Vinicius de Oliveira Rodrigues
May 19, 2026stat.ML

Contradiction Graphs Determine VC Dimension

We study the contradiction graphs associated with binary concept classes. For a class H{0,1}XH \subseteq \{0,1\}^X, the order-mm contradiction graph Gm(H)G_m(H) has as vertices the HH-realizable labeled sequences of length mm, with two vertices adjacent when the two sequences assign opposite labels to some common domain point. Our main result is that the single graph Gm(H)G_m(H) determines the threshold predicate VCdim(H)m\mathrm{VCdim}(H)\ge m. Consequently, the full sequence (Gm(H))m1(G_m(H))_{m \ge 1} determines the exact VC dimension and, in particular, detects finite versus infinite VC dimension, answering a question posed by Alon et al. (2024).
Jesse Campbell, Daniel Ibaibarriaga, Lev Reyzin
Mar 7, 2026cs.LG

Margin in Abstract Spaces

Margin-based learning, exemplified by linear and kernel methods, is one of the few classical settings where generalization guarantees are independent of the number of parameters. This makes it a central case study in modern highly over-parameterized learning. We ask what minimal mathematical structure underlies this phenomenon. We begin with a simple margin-based problem in arbitrary metric spaces: concepts are defined by a center point and classify points according to whether their distance lies below rr or above RR. We show that whenever R>3rR>3r, this class is learnable in \emph{any} metric space. Thus, sufficiently large margins make learnability rely only on the triangle inequality, without any linear or analytic structure being necessary. Our first main result extends this phenomenon to concepts defined by bounded linear combinations of distance functions, and reveals a sharp threshold: there exists a universal constant such that whenever the margin is larger than this constant, the class is learnable in every metric space, while below it there exist metric spaces where it is not learnable at all. We then ask whether margin-based learnability can always be explained via an embedding into a linear space -- that is, reduced to linear classification in some Banach space through a kernel-type construction. We answer this negatively by demonstrating a margin learnable class that cannot be embedded into any Banach space in which linear classification with margins is learnable.
Yair Ashlagi, Roi Livni, Shay Moran +1