cs.LGAug 18, 2025

Hierarchical Conformal Classification

Authors: Floris den Hengst, Inès Blin, Majid Mohammadi, Syed Ihtesham Hussain Shah, Taraneh Younesian

Organizations: Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · Sony Computer Science Laboratories - Paris, Paris, France

Abstract

Conformal prediction (CP) is a powerful framework for quantifying uncertainty in machine learning models, offering reliable predictions with finite-sample coverage guarantees. When applied to classification, CP produces a prediction set of possible labels that is guaranteed to contain the true label with high probability, regardless of the underlying classifier. However, standard CP treats classes as flat and unstructured, ignoring domain knowledge such as semantic relationships or hierarchical structure among class labels. This paper presents hierarchical conformal classification (HCC), an extension of CP that incorporates class hierarchies into both the structure and semantics of prediction sets. We formulate HCC as a constrained optimization problem whose solutions yield prediction sets composed of nodes at different levels of the hierarchy, while maintaining coverage guarantees. To address the combinatorial nature of the problem, we formally show that a much smaller, well-structured subset of candidate solutions suffices to ensure coverage while upholding optimality. An empirical evaluation on three new benchmarks consisting of audio, image, and text data highlights the advantages of our approach, and a user study shows that annotators significantly prefer hierarchical over flat prediction sets.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Cluster Frequency Conformal Prediction for Local Coverage

    May 24, 2026Tomer Lavi, Bracha Shapira, Nadav RappoportConformal PredictionMulticlass Classification

  2. Softmax is not Enough (for Adaptive Conformal Classification)

    Feb 23, 2026Navid Akhavan Attar, Hesam Asadollahzadeh, Ling Luo +1Conformal PredictionUncertainty Quantification

  3. Conformalized Credal Regions for Classification with Ambiguous Ground Truth

    Nov 7, 2024Michele Caprio, David Stutz, Shuo Li +1Conformal PredictionImprecise Probability