Interpreting Random Forests: Comparing Surrogate Models and Rule Extraction Methods
Organizations: Institute of Medical Biometry and Statistics, University of Luebeck, Lübeck, Germany · Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany · CAIMed, Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine, Göttingen
Abstract
Random forests (RFs) often achieve high prediction performance, but their ensemble structure can be difficult to interpret. Surrogate models and rule extraction methods offer interpretable approximations, yet guidance on selecting an appropriate method for a specific research question remains limited. We compared 12 methods for interpreting RFs, with a decision tree as a prediction-oriented reference, across 11 simulation settings and 67 benchmark datasets for regression and classification. Evaluation addressed prediction and structural similarity to the RF, model complexity, prediction performance, and runtime. Simulations additionally assessed explainability using known effect and noise variables. No method performed best across all objectives. For regression, SLIM provided the strongest general performance for prediction similarity or performance on test data, while a linear model offered a simpler global interpretation of linear effects when performance was comparable. For classification, inTrees achieved the strongest overall benchmark performance, with CTree and OC-CART providing useful alternatives. ART consistently produced the most compact representations. NH and SIRUS offered strong structural similarity for regression, while MRT most closely reproduced RF variable usage in the classification simulations, illustrating that prediction-oriented rankings do not necessarily identify the best structural explanation. Outcome-type restrictions, runtime limits, and method-specific failures reduced result availability for several approaches and should be considered alongside performance when selecting a method. Method selection should therefore be guided by the outcome type, anticipated data structure, and intended interpretability objective. Prediction similarity alone is insufficient for explaining an RF and should be considered jointly with structural similarity and model complexity.
Figures & tables
| Method | Information used | Final representation | Construction and selection criteria | Stopping and complexity control | Prediction components | Final prediction |
|---|---|---|---|---|---|---|
| Methods using the oracle coaching approach | ||||||
| LM | Variables and RF predictions | Global linear regression model | Least-squares fitting. Forward selection can reduce the variable set | Variable selection stops when the selection criterion no longer improves | Intercept and linear variable terms | Intercept plus the sum of variable values multiplied by their coefficients |
| OC-CART | Variables and RF predictions | Single tree with constant terminal-node predictions | Variable and split point selected jointly by maximizing impurity reduction | Node-size and other growth constraints. No post-pruning in this study | Terminal-node mean or majority class | Prediction of the terminal node reached by the observation |
| GUIDE | Variables and RF predictions | Single tree with local linear main effects models | Variable selected using residual-sign association and interaction tests. Split point minimizes the combined residual sum of squares of the child-node models | Minimum improvement criterion, node-size constraints, and a limit on tree-growth levels. Variable-selection tests do not require significance for continued growth | Linear model within each terminal node | Prediction from the local model in the terminal node reached by the observation |
| SLIM | Variables and RF predictions | Single tree with local main effects models | Variable and split point selected jointly by minimizing the combined residual sum of squares of the child-node models | Node-size constraints and a limit on tree-growth levels in this study. A minimum improvement criterion also controls splitting | Main effects model within each terminal node | Prediction from the local model in the terminal node reached by the observation |
| MOB | Variables and RF predictions | Single tree with intercept-only node models | Variable selected by parameter instability tests. Split point minimizes the combined residual sum of squares around the child-node means | No significant instability after multiplicity correction, or growth constraints | Mean RF prediction within each terminal node | Prediction of the terminal node reached by the observation |
| Name | Abbreviation | Type | Publication | Implementation | Used version | Regression | Classification |
|---|---|---|---|---|---|---|---|
| Artificial Representative Tree | ART | S | ( Laabs et al., 2024a ) | R package timbR on GitHub https://github.com/imbs-hl/timbR | V3.3.2 | yes | yes |
| Conditional Inference Tree | CTree | S | ( Hothorn et al., 2006 ) | R package partykit on CRAN | V1.2-26 | yes | yes |
| Classification and Regression Tree | decision tree (reference model) | - | ( Breiman et al., 1984 ) | R package ranger on CRAN | V0.17.0 | yes | yes |
| Generalized, Unbiased Interaction Detection and Estimation | GUIDE | S | ( Loh, 2002 ) | R implementation on GitHub https://github.com/slds-lmu/mbt_comparison | Downloaded 18.01.2026 | yes | no |
| Interpretable Trees | inTrees | R | ( Deng, 2019 ) | R package inTrees in CRAN archive | V1.4 | yes | yes |
| Model-Based Recursive Partitioning | MOB | S | ( Zeileis et al., 2008 ) | R package partykit on CRAN | V1.2-26 | yes | no |
| Data set | N (original) | N (after QC) | P (original) | P (after QC) | outcome | type | Source | Used in publication of |
| BreastCancer | 699 | 683 | 11 | 10 | classif (2) | bench | UCI: https://doi.org/10.24432/C5HP4Z | CTree |
| Digit recognition | 5200 | 5200 | 7 | 7 | classif (10) | sim | Simulation from ( Breiman et al., 1984 ) | decision tree |
| Digit recognition noisy | 5200 | 5200 | 24 | 24 | classif (10) | sim | Simulation from ( Breiman et al., 1984 ) | CART algorithm |
| GlaucomaMVF | 170 | 153 | 67 | 67 | classif (2) | bench | R package ipred V0.9.15 | CTree |
| XOR-problem | 200 | 200 | 20 | 20 | classif (2) | sim | Simulation from ( Deng, 2019 ) | inTrees |
| Soybean | 683 | 562 | 36 | 36 | classif (19) | bench | R package mlbench V2.1.7 | CTree |
| Method | Outcome | Source | Details |
|---|---|---|---|
| ART | regr | bench | runtime above 24 h for data sets Allstate (10 times), Buzzinsocialmedia_Twitter (10 times), nyc-taxi (7 times) |
| ART | classif | bench | no variables selected for data set heart_disease_s (4 times) |
| NH | regr | bench | constraints are inconsistent, no solution for data set bike_sharing (8 times); negative length vectors are not allowed for data sets Allstate (10 times), Brazilian_houses (10 times), Journals (10 times), Mercedes_Benz (10 times), Moneyball (10 times), Ozone (10 times), Servo (10 times), abalone (10 times), black_friday (10 times), nyc-taxi (10 times), socmob (10 times); quadratic function is not positive definite for data sets Buzzinsocialmedia_Twitter (10 times), diamonds (10 times), house_16H (10 times), house_sales (10 times) |
| SLIM | regr | bench | no applicable method for ‘predict’ applied to an object of class for data sets Journals (4 times), Ozone (2 times), machine (3 times), prostate (7 times) |
| ruleFit | regr | bench | runtime above 24 h for data sets Buzzinsocialmedia_Twitter (10 times), nyc-taxi (10 times) |
| ruleFit | classif | sim | multinomial or binomial class has 1 or 0 observations for data set Digits recognition (5 times) |
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 329 papers