cs.LGNov 4, 2026

Partially Interpretable Decision Trees

Authors: Ludovic Schwartz, Federico Berton, Tim van Erven

Organizations: University of Amsterdam

Abstract

Partially interpretable models extend simple interpretable models with the option to defer complicated inputs to an accurate but uninterpretable black-box model. By deferring more inputs, the model gets more accurate; by deferring fewer, it gets more interpretable. Existing procedures to train such partially interpretable models build on linear models or decision rules. We propose three algorithms for decision trees: a partially interpretable extension of the existing Branches algorithm, which performs exact search to optimize the tree; a conversion algorithm that we run on top of standard CART, which is computationally more efficient; and a generalized CART whose splitting criterion takes into account the option to defer while growing the tree. Perhaps surprisingly, the performance of the three algorithms is similar across many realworld datasets. We can explain this through a formal bound on the potential gain of smart tree construction, provided the black box accuracy is sufficiently homogeneous across the domain. We also construct a synthetic dataset where the accuracy of the black box is deliberately inhomogeneous. There, our deferral-aware generalization of CART does outperform regular CART and matches the performance of exact search. Finally, we evaluate an application to concept bottleneck models. Partially interpretable decision trees can replace uninterpretable neural networks after the bottleneck. On undeferred inputs, the model is fully interpretable, and deferring preserves accuracy.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again

    Jun 30, 2026Zakk Heile, Hayden McTavish, Margo Seltzer +1Ensemble LearningLearning to Defer

  2. Experiments with Optimal Model Trees

    Mar 17, 2025Sabino Francesco Roselli, Eibe FrankInterpretable MLDecision Tree Learning

  3. Temporally Interpretable Differentiable Decision Trees

    Oct 7, 2026Eisuke Hirota, Aarav Sane, Rohan PalejaInterpretable MLInterpretability in RL