Improving the Intrinsic Interpretability of Differentiable Decision Trees through Euclidean Feature-Concentration Regularisation

Qiqi Yang

cs.LGOct 29, 2026

Improving the Intrinsic Interpretability of Differentiable Decision Trees through Euclidean Feature-Concentration Regularisation

Authors: Qiqi Yang

Abstract

Differentiable decision trees (DDTs) support end-to-end optimisation and explicit decision paths, but dense oblique splits can obscure the role of individual input features. We propose the Euclidean-Concentrated Differentiable Decision Tree (EU-DDT), which regularises the relative absolute feature weights at each internal node using their normalised squared Euclidean distance from the uniform distribution. This encourages dominance by a small number of features while retaining probabilistic soft routing. Across five matched random seeds, EU-DDT reduces the mean effective number of features from 8.94 to 1.54 on Wine and from 45.28 to 9.06 on Digits. Mean test accuracy changes from 100.0% to 93.9% on Wine and from 92.3% to 94.6% on Digits, illustrating the dataset-dependent trade-off between concentration and predictive performance. We also use EU-DDT to parameterise a PPO actor with an MLP critic. Two random-seed runs per environment show a consistent qualitative increase in concentration while preserving effective task performance. In representative CartPole-v1 and MiniPacman runs, EU-DDT reaches 100% final evaluation success, while mean node concentration increases from 0.1272 to 0.7179 and from 0.0437 to 0.7237, respectively. These results support EU regularisation as a mechanism for reducing node-level feature complexity; the reinforcement-learning evidence remains preliminary because only two seeds are evaluated.

Figures & tables

Parameter	Setting
Comparison models	DDT / L1-DDT / L2-DDT / EU-DDT / Neural Network
Input scaling	Wine: standardisation; Digits: MinMax scaling to $[-1,1]$
Tree depth	Wine: 3; Digits: 6
Internal nodes / leaves	Wine: 7 / 8; Digits: 63 / 64
Optimizer	Adam
Batching	Full batch

Table 6-1: Supervised-Learning Training Parameters and Model Configuration

Dataset	Model	Maximum weight share $\uparrow$	Top-1/Top-2 margin $\uparrow$	Effective features $\downarrow$	Relative sparsity $\uparrow$	Mean active-node $\overline{C}_{\mathcal{A}}\uparrow$
Wine	DDT	0.1797	0.0296	8.94	0.0681	0.0403
Wine	DDT+L1	0.3272	0.0985	5.23	0.4374	0.1668
Wine	DDT+L2	0.1894	0.0334	8.49	0.0835	0.0468
Wine	EU-DDT	0.7804	0.5906	1.54	0.8659	0.6962
Digits	DDT	0.0381	0.0022	45.28	0.0726	0.0069
Digits	DDT+L1	0.1175	0.0360	30.13	0.3386	0.0554

Table 6-2: Five-Seed Mean Node-Weight Statistics for Wine and Digits

Dataset	DDT	DDT+L1	DDT+L2	EU-DDT	Neural Network
Wine five-seed mean accuracy	1.000	1.000	1.000	0.939	1.000
Digits five-seed mean accuracy	0.923	0.949	0.960	0.946	0.969

Table 6-3: Five-Seed Mean Test Accuracy on Wine and Digits

Panel of Figure 6 — Figure 6-1: Comparison of Absolute Feature Weights at Selected Internal Nodes on Digits.

Parameter	CartPole-v1	MiniPacman
Representative seed	3	1
State dimension	4	14
Action dimension	2	4
Tree depth	3	4
Internal nodes / leaves	7 / 8	15 / 16
PPO updates	60	60

Table 7-1: Reinforcement-Learning Experimental Configuration

CartPole-v1
Metric	DDT	L1	L2	EU
Final success rate $\uparrow$	100%	90%	46.7%	100%
Concentration $\uparrow$	0.1272	0.3153	0.0798	0.7179
Maximum share $\uparrow$	0.4661	0.6031	0.4114	0.8220
Top-1/Top-2 margin $\uparrow$	0.1648	0.3562	0.1272	0.7236
k90 $\downarrow$	3.29	2.86	3.71	1.71

Table 7-2: Reinforcement-Learning Results for Representative Random Seeds

Panel of Figure 7 — Figure 7-1: Independent-Evaluation Success Rate and Legacy DDT Weight-Diagnostic Curves for Representative Runs on CartPole-v1 and MiniPacman.

Interpretable ML

Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.

See all 310 papers

Oct 7, 2026

Temporally Interpretable Differentiable Decision Trees

Interpretability offers a solution to safe autonomy by providing transparency into an agent's underlying decision-making model. Within sequential-decision making tasks, differentiable decision trees (DDTs) are one approach to such interpretability, maintaining automatic-differentiable policies while providing humans with a discrete tree-based visualization. Nonetheless, current implementations of DDTs are not well-suited for sequential-decision making domains, as there exists an inherent mismatch between a tree's single-timestep behavior and a human's multi-timestep planning. Our work thus introduces time as a new dimension of interpretability, coined as temporal interpretability, and demonstrates how temporal abstractions via action chunking improve it. We achieve this by first introducing two novel policy gradient algorithms that incorporate action chunking. Additionally, to maintain parameter-efficient trees, we develop an information-theoretic tree restructuring algorithm that modifies the tree during training. Across four simulation environments, we find that warm-starting action chunked DDTs from a distilled action chunked policy is the most effective way to obtain temporally interpretable trees: they match neural network policies in three of the four domains while using up to 80

\%

fewer parameters. Our code is available at https://github.com/ei5uke/temp-interp.

Eisuke Hirota, Aarav Sane, Rohan Paleja

Department of Computer Science, Purdue University · Thomas Lord Department of Computer Science, University of Southern California

Interpretable ML Interpretability in RL Reinforcement Learning Policy Gradient Methods

Ask PDF

Aug 5, 2026

ArborEnum: Decision Tree Rashomon Sets over Continuous Features

The Rashomon effect describes the phenomenon that many models can achieve nearly equivalent performance on the same learning task, with significant ramifications for robustness, feature importance, and customizability. These use cases motivate the computation of Rashomon sets: the set of all models whose regularized loss is near-optimal. Decision trees are one of the few model classes for which Rashomon sets can be fully enumerated, but this computation has always been conditional on a binarization of the original data, either restricting which splits each tree is allowed to make or substantially increasing the complexity of an already difficult combinatorial problem. We introduce the first algorithm that exactly enumerates decision-tree Rashomon sets while exploiting the ordered structure of continuous features. We further develop a relaxation for approximate enumeration and an anytime algorithm that progressively refines the set of candidate thresholds, producing increasingly detailed approximations that converge to the continuous-feature Rashomon set. Experiments show that coarse binarization can miss many trees, important features, and predictive multiplicity; our algorithms achieve orders-of-magnitude speedups over existing enumeration methods, with approximations providing further speedups while maintaining near-perfect recall.

Zakk Heile, Hayden McTavish, Margo Seltzer +1

Department of Computer Science, Duke University, Durham, USA · Department of Computer Science, University of British Columbia, Vancouver, Canada

Decision Tree Learning

Ask PDF

Jul 28, 2026

dtControl2+ $\varepsilon$ : Trading Optimality for Explainability in MDPs via Decision Trees

Over the past decade, decision trees have been used to represent controllers (a.k.a. policies) in an explainable way, with dtControl2 as a current state-of-the-art tool. However, for systems that are large or have many corner cases, even such representations tend to be too complex and not human-comprehensible. Unfortunately, reducing the size of the decision tree is not straightforward, as missing just a single crucial case might result in an incorrect controller. We tackle this issue in the setting of Markov decision processes, extending dtControl2 by "

\varepsilon

" functionality: Given an allowed imprecision

\varepsilon \geq 0

, we construct a smaller decision tree, distilling the essence of the controller, while still guaranteeing its

\varepsilon

-optimality. This enables us to provide tunably simpler explanations, omitting a controllable amount of detail. Our tool constructs decision trees that are orders of magnitude smaller than the state of the art.

Tereza Kinská, Jan Křetínský, Tobias Meggendorfer +2

Masaryk University, Brno, Czech Republic · Technical University of Munich, Munich, Germany · Lancaster University Leipzig, Leipzig, Germany +1

Markov Decision Processes Markov Models Decision Tree Learning Interpretability in RL

Ask PDF

Abstract

Interpretable ML

Explore similar work

Temporally Interpretable Differentiable Decision Trees

ArborEnum: Decision Tree Rashomon Sets over Continuous Features

dtControl2+ε\varepsilonε: Trading Optimality for Explainability in MDPs via Decision Trees

dtControl2+ $\varepsilon$ : Trading Optimality for Explainability in MDPs via Decision Trees