cs.LGJul 30, 2026

Search Strategies for Optimal Classification and Regression Trees

Authors: Jacobus G. M. van der LindenMim van den BosEmir Demirović

Organizations: Delft University of Technology

Abstract

Optimal decision trees (ODTs) are compact, interpretable machine learning models that globally optimize a given objective, but their scalability remains challenging. While recent work has proposed a variety of search strategies to improve scalability, the precise contribution of each strategy remains unclear. To address this gap, we introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones. This provides a common lens through which to understand and compare different strategies, which we use to empirically investigate the effect of 18 search strategies. Compared to the state of the art, the best strategy in our evaluation achieves significantly better anytime performance for classification, and improves runtime by more than an order of magnitude for regression.

Explore similar work

CardsList
  1. Experiments with Optimal Model Trees

    Mar 17, 2025Sabino Francesco Roselli, Eibe FrankDecision TreesInterpretable Models