cs.LGAug 13, 2026

Exponential Convex Calibration Dimension for the Multi-Label Jaccard Measure

Authors: Mingyuan Zhang

Organizations: Independent Researcher

Abstract

The per-instance Jaccard score, or intersection over union (IoU), is standard in multi-label classification and binary segmentation. With ss labels, its loss matrix has 2s2^s outcomes and reports. Under the convention Jac(,)=1\mathrm{Jac}(\varnothing,\varnothing)=1, we prove that the Jaccard score, shifted-loss, and ordinary loss matrices are nonsingular and that the loss columns have affine dimension 2s12^s-1. The proof combines a finite MinHash Gram representation with Boolean Möbius inversion. For exact calibration, we prove 2s1CCdim(LJac)2s12^{s-1} \leq \mathrm{CCdim}(L^{\mathrm{Jac}}) \leq 2^s-1. The lower bound uses a factorially weighted distribution with 2s1+12^{s-1}+1 supported outcomes and Bayes-optimal reports. Consequently, every exactly calibrated convex surrogate requires exponentially many prediction coordinates. We also give two polynomial-dimensional approximation guarantees with explicit regret transfers. A new F1F_1-to-Jaccard transfer turns an existing (s2+1)(s^2+1)-dimensional F1F_1 surrogate into a polynomial-time rule with asymptotic Jaccard regret at most 3223-2\sqrt{2}. For any α>0α>0 and 0<ρ<10<ρ<1, a MinHash square-loss surrogate attains Jaccard-regret floor αα uniformly over arbitrary conditional label distributions. With probability at least 1ρ1-ρ, the direct construction has dimension O((s2+slog(1/ρ))/α2)O((s^2+s\log(1/ρ))/α^2), while a signed variant has dimension O((s+log(1/ρ))/α2)O((s+\log(1/ρ))/α^2). Thus zero-regret calibration requires exponential dimension, whereas every fixed additive regret tolerance admits polynomial prediction dimension.

Explore similar work

Aug 9, 2026cs.LG

Exact Rank and Convex Calibration Dimension Lower Bounds for the Multi-Label F1 Loss

The instance-wise F1F_1 measure is a central performance measure for multi-label classification. For a problem with ss labels, it defines a 2s×2s2^s\times 2^s loss matrix. Previous work exhibited s2+1s^2+1-coordinate affine and shifted low-rank representations and used them to construct quadratic-dimensional convex calibrated surrogates. We determine the exact rank. Under the convention F1(,)=1F_1(\varnothing,\varnothing)=1, the F1F_1 score matrix, the shifted loss matrix, and the unshifted loss matrix all have rank s2s+2s^2-s+2, while the column-affine dimension of the loss is s2s+1s^2-s+1. The proof factors the nonempty score matrix through subset-incidence matrices and a positive-definite Cauchy matrix. Exact rank does not, by itself, lower-bound the dimension of an arbitrary convex calibrated surrogate. We therefore analyze the Bayes geometry of F1F_1 directly. We construct a distribution for which precisely all supersets of a fixed core label set are Bayes optimal, and show that the corresponding active loss columns, restricted to the witness support, have affine dimension hnhn, where n=ss/3n=s-\lfloor s/3\rfloor and h=(ss/3)1/21h=\lceil(s\lfloor s/3\rfloor)^{1/2}\rceil-1. Applying the feasible-subspace lower bound for convex calibration dimension gives CCdim(LF1)(233o(1))s2.\operatorname{CCdim}(L^{F_1}) \ge \left(\frac{2}{3\sqrt{3}}-o(1)\right)s^2. Together with the quadratic upper bound, this establishes CCdim(LF1)=Θ(s2)\operatorname{CCdim}(L^{F_1})=Θ(s^2).
Mingyuan Zhang
May 27, 2026cs.LG

Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learning

Many real-world classification tasks require predicting multiple labels per instance, necessitating the optimization of complex evaluation metrics such as the FF-measure and Jaccard index. While the Empirical Utility Maximization (EUM) framework is natural for these population-level metrics, existing theoretical results are largely limited to asymptotic Bayes-consistency. In this paper, we develop principled learning algorithms for optimizing a broad class of generalized metrics within the EUM framework, grounded in the stronger notion of HH-consistency. Our key contribution is the design of novel surrogate loss functions for multi-label learning that admit provable HH-consistency bounds, enabling optimization with non-asymptotic guarantees tailored to the hypothesis class and finite samples. Crucially, we prove these combinatorially formulated surrogates decompose exactly, operating in strictly O(l)O(l) time without approximations. Building on this foundation, we introduce MMO (Multi-Label Metric Optimization), a new family of algorithms for optimizing generalized linear-fractional metrics. We validate our approach through extensive experiments, demonstrating robust scalability and superior performance over state-of-the-art continuous baselines on large-scale datasets (MS-COCO, Reuters-21578) in high-sparsity, deep learning regimes. Our results offer both theoretical rigor and practical effectiveness for general multi-label metric optimization.
Mehryar Mohri, Yutao Zhong
Jul 20, 2026cs.LG

The Label Complexity of Class-Conditional Coverage under Distribution Shift

Conformal prediction certifies that a classifier's prediction sets cover the truth, and that certificate is marginal. Many recognition benchmarks build distribution shift into evaluation, placing disjoint conditions in the training and test splits. Under that shift the certificate stays reassuring while per class coverage fails silently: on a real cross subject skeleton benchmark marginal coverage holds near ninety percent while the worst class is covered about seventy percent and ten of sixty classes fall below eighty percent. This class specific undercoverage stays hidden behind a single reassuring marginal number. Once the shift acts jointly on covariates and labels, the target class conditional score law is unidentified, so no label free method is at once per class valid and efficient uniformly over target laws consistent with the observed source joint distribution and target covariate marginal. The per class labels needed to recover every class threshold to a given tolerance grow as the inverse square of that tolerance and the logarithm of the class count, with matching bounds for classwise threshold procedures. Pseudo labels do not shortcut it: the best prediction powered estimator gains at most a small constant factor where coverage collapses. Across three real shifts and an image corruption benchmark, source label calibration recovers much of the gap while marginal coverage holds, and stops once it breaks.
Weijia Han, Lisha Qu