cs.LGNov 16, 2026

Explaining Cross-Label Decision Structure in Multi-Output Tree Ensembles

Authors: Sylvio Barbon Junior, Hiago F. F. Oliveira, Rafael Figueira Gonçalves, Jessica Fernandes Lopes, Ricardo Cerri

Organizations: University of Trieste, Trieste, Italy · Institute of Mathematics and Computer Sciences, University of São Paulo, São Carlos, Brazil · SENAI, Londrina, Brazil

Abstract

A multi-output tree ensemble predicts several binary labels through decision paths that it largely shares across those labels, yet the attribution methods commonly used to explain such models treat each output in isolation. We propose multi-sink Decision Predicate Graphs (ms-DPGs), which extend decision predicate graphs with one sink per label: each leaf contributes to the sink of every label its in-bag positive count for that label. On this representation we define Shared Predicate Flow (SPF), the share of a label's decision mass that passes through conditions active for both labels of a pair, and a contracted view of the graph in which label-specific lifts indicate whether a shared condition raises the probability of both labels or of one at the expense of the other. Controlled experiments show that SPF captures shared decision conditions rather than mere feature overlap. On nine multi-label benchmarks, native multi-output forests share substantially more decision structure than Binary Relevance on eight, including five of the six on which the two are at predictive parity, even at equal hyperparameters, whereas per-label TreeSHAP rankings register the change of architecture without showing whether more is shared. Because a native forest routes every label through the same splits, SPF describes the fitted model rather than the data, and on eight of the nine benchmarks it rises when the feature-label signal is removed; a high SPF is therefore not evidence of shared predictive information. ms-DPG offers a global, predicate-level account of how a multi-output ensemble reuses decision conditions across its labels.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Nov 13, 2026

Where Tree Ensembles Decide: Structural Local Explanation with Decision Predicate Graphs

Tree ensembles are among the most accurate models for tabular data, yet the computation behind a single prediction is spread over hundreds of decision rules. Feature attributions and counterfactuals explain such predictions through the influence of features or a change of the input, leaving open which of the thresholds applied by the trees a prediction responds to, while structural explanations compress these rules and lose the trees behind them. To address this gap, we propose a local Decision Predicate Graph (local DPG), which gathers the rules executed for a sample into a verifiable record of the computation, with a class support that reproduces every explained prediction. From these rules we select the critical predicate, the executed predicate whose crossing is estimated to move the model most strongly toward a chosen class, which offers the user a concrete threshold and the trees that share it, without an external explainer. Across 16 datasets and four ensemble families, it separates differently predicted neighbours better than random and nearest-threshold predicates. A single crossing recovers the true class in 46-69% of wrong predictions, against 29-50% when the same rule is applied to the feature ranked first by TreeSHAP, a measure close to the selection criterion, and on planted faults it finds the faulty threshold in 46-71% of the resulting errors, against 6-21% at random. Although training-free edits of these predicates did not reduce the error on unseen data, the critical predicate complements attributions and counterfactuals by naming where a prediction can be changed.
Feb 26, 2026

Fair feature attribution for multi-output prediction: a Shapley-based perspective

In this article, we provide an axiomatic characterization of feature attribution for multi-output predictors within the Shapley framework. While SHAP explanations are routinely computed independently for each output coordinate, the theoretical necessity of this practice has remained unclear. By extending the classical Shapley axioms to vector-valued cooperative games, we establish a rigidity theorem showing that any attribution rule satisfying efficiency, symmetry, dummy player, and additivity must necessarily decompose component-wise across outputs. Consequently, any joint-output attribution rule must relax at least one of the classical Shapley axioms. This result identifies a previously unformalized structural constraint in Shapley-based interpretability, clarifying the precise scope of fairness-consistent explanations in multi-output learning. Numerical experiments on a biomedical benchmark illustrate that multi-output models can yield computational savings in training and deployment, while producing SHAP explanations that remain fully consistent with the component-wise structure imposed by the Shapley axioms.
May 14, 2026

Woodelf++: A Fast and Unified Partial Dependence Plot Algorithm for Decision Tree Ensembles

Partial Dependence Plots (PDPs) visualize how changes in a single feature affect the average model prediction. They are widely used in practice to interpret decision tree ensembles and other machine learning models. Joint-PDPs extend this idea to pairs of features, revealing their combined effect. Partial Dependence Interaction Values (PDIVs) measure feature interactions. The Any-Order-PDIVs task computes these interactions for every feature subset across all rows of the dataset. We introduce Woodelf++, a unified and efficient approach for computing all these useful explainability tools on decision tree ensembles, building on Woodelf, an algorithm for efficient SHAP computation. By deriving suitable metrics over pseudo-Boolean functions, Woodelf++ can compute PDPs (exact and approximate), Joint-PDPs, and Any-Order-PDIVs in a unified framework. Our method delivers substantial complexity improvements over the state of the art, including an exponential gain for Any-Order-PDIVs. Additionally, we introduce and efficiently compute Full PDPs, which leverage the model's split thresholds to faithfully capture its behavior across all possible feature values. Woodelf++ is implemented in pure Python and supports GPU acceleration. On a dataset with 400,000 rows, Woodelf++ computes PDP and Joint-PDP up to 6x faster than the state of the art and up to five orders of magnitude faster than scikit-learn. For Any-Order-PDIVs, the gap is even larger: Woodelf++ computes all interaction values in 5 minutes, while the state of the art is estimated to require over 1,000,000 years.