Symbolic Branch Networks: Tree-Inherited Neural Models for Interpretable Multiclass Classification
Organizations: Independent Researcher, Germany
Abstract
Symbolic Branch Networks (SBNs) are neural models whose architecture is derived directly from an ensemble of decision trees. Each root-to-parent-of-leaf decision path is mapped to a hidden neuron, while the matrices W1 (feature-to-branch) and W2 (branch-to-class) encode the structural relationships induced by the teacher ensemble. This provides an explicit correspondence between teacher branches and neural components, enabling feature-level and branch-level interpretation while retaining the flexibility of gradient-based optimization. The primary configuration studied in this work trains the teacher-derived feature-to-branch matrix W1 through gradient-based learning while keeping the branch-to-class matrix W2 fixed; this configuration is denoted SBN1,0. Across 29 multiclass tabular datasets, this configuration improves upon the teacher ensemble on 28 of 29 datasets, increasing mean accuracy from 84.0% to 87.9%. For reference, XGBoost and TabPFN achieve mean accuracies of 87.3% and 91.3%, respectively, on the same datasets. A one-sided Wilcoxon signed-rank test on dataset-level mean accuracies indicates a statistically significant improvement of SBN1,0 over the teacher ensemble (p=4.71x10-7). Beyond predictive performance, SBN provides an explicit structural basis for interpretation. Global feature importance can be derived from the teacher-induced feature-to-branch connectivity, while individual predictions can be traced to selected branch-neurons, their associated features, and their class contributions. An analysis across ten independently constructed teacher ensembles shows consistent predictions for a selected instance but only partial stability in its local feature rankings.
Figures & tables
| Configuration | Interpretation | ||
|---|---|---|---|
| Fixed | Fixed | Both matrices fixed | |
| Fixed | Trained | Only trained | |
| Trained | Fixed | Primary SBN configuration | |
| Trained | Trained | Both parameter matrices trained |
| Task type | # datasets | # samples (range) | # features (range) | # classes |
|---|---|---|---|---|
| Multiclass (main study) | 29 | – | – | – |
| Binary (Appendix) | 30 | – | – |
| Parameter | Default value | Search values |
|---|---|---|
| n_estimators | 100 | 10, 50, 100, 200 |
| max_depth | 6 | 3, 6, 9 |
| learning_rate | 0.3 | 0.03, 0.1, 0.3 |
| Dataset | TeacherSBN | ||||
|---|---|---|---|---|---|
| analcatdata_authorship | 0.978 0.009 | 0.989 0.009 | 0.991 0.009 | 0.993 0.008 | 0.989 0.011 |
| balance-scale | 0.840 0.024 | 0.890 0.025 | 0.927 0.042 | 0.951 0.020 | 0.947 0.026 |
| car | 0.933 0.011 | 0.982 0.012 | 0.984 0.008 | 0.999 0.002 | 0.999 0.002 |
| cmc | 0.514 0.028 | 0.542 0.032 | 0.539 0.037 | 0.545 0.043 | 0.538 0.038 |
| cnae-9 | 0.914 0.016 | 0.902 0.019 | 0.912 0.020 | 0.921 0.021 | 0.924 0.018 |
| connect-4 | 0.689 0.007 | 0.763 0.007 | 0.753 0.008 | 0.814 0.050 | 0.818 0.003 |
| Comparison | Wilcoxon statistic | -value |
|---|---|---|
| vs. TeacherSBN | 371 | |
| vs. TeacherSBN | 367 | |
| vs. TeacherSBN | 417 | |
| vs. TeacherSBN | 414 |
| Dataset | TabPFN | XGBoost | |
|---|---|---|---|
| analcatdata_authorship | 1.000 0.000 | 0.983 0.007 | 0.993 0.008 |
| balance-scale | 0.984 0.009 | 0.857 0.030 | 0.951 0.020 |
| car | 0.998 0.003 | 0.988 0.007 | 0.999 0.002 |
| cmc | 0.561 0.026 | 0.529 0.034 | 0.545 0.043 |
| cnae-9 | 0.937 0.019 | 0.897 0.022 | 0.921 0.021 |
| connect-4 | 0.850 0.002 | 0.844 0.013 | 0.814 0.050 |
| Comparison | Wilcoxon statistic | -value |
|---|---|---|
| vs. TabPFN | 1 | |
| vs. XGBoost | 156 |
| Feature | Interpretation |
|---|---|
| COMPACTNESS | Overall compactness of the vehicle silhouette. |
| CIRCULARITY | Degree to which the silhouette approaches a circular shape. |
| DISTANCE_CIRCULARITY | Circularity measured from distances between the centroid and boundary. |
| RADIUS_RATIO | Ratio between characteristic radii of the silhouette. |
| PR.AXIS_ASPECT_RATIO | Aspect ratio defined with respect to the principal axis. |
| MAX.LENGTH_ASPECT_RATIO | Aspect ratio derived from the maximum silhouette length. |
| Feature | Runs in top 3 | Runs in top 5 | Median rank | Mean rank |
|---|---|---|---|---|
| HOLLOWS_RATIO | 9 | 10 | 1 | 1.60 |
| MAX.LENGTH_RECTANGULARITY | 7 | 9 | 2 | 2.67 |
| COMPACTNESS | 4 | 6 | 2.5 | 2.83 |
| CIRCULARITY | 2 | 4 | 5 | 4.20 |
| MAX.LENGTH_ASPECT_RATIO | 2 | 4 | 5 | 6.71 |
| KURTOSIS_ABOUT_MINOR | 2 | 3 | 3 | 3.00 |
| No. of | No. of | Total | Hidden neurons | sparsity (%) | sparsity (%) | ||||
|---|---|---|---|---|---|---|---|---|---|
| Dataset | classes | feats | samples | min | max | min | max | min | max |
| analcatdata_authorship | 4 | 70 | 841 | 563 | 627 | 88.89 | 89.53 | 38.40 | 39.88 |
| balance-scale | 3 | 4 | 625 | 331 | 362 | 2.26 | 3.75 | 18.17 | 20.04 |
| car | 4 | 21 | 1728 | 781 | 896 | 54.19 | 55.96 | 44.27 | 45.51 |
| cmc | 3 | 9 | 1473 | 485 | 504 | 28.59 | 34.16 | 13.62 | 17.04 |
| cnae-9 | 9 | 856 | 1080 | 3376 | 3556 | 95.41 | 95.78 | 46.24 | 49.11 |
| Component | Balance-scale | MNIST |
|---|---|---|
| Training time (s) | ||
| Teacher training | ||
| Architecture construction | ||
| Student training | ||
| Total pipeline | ||
| Class-prediction inference | ||
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | TeacherSBN | SBN_0_0 | SBN_0_1 | SBN_1_0 | SBN_1_1 |
|---|---|---|---|---|---|
| Internet-Advertisements | 0.950 0.011 | 0.931 0.018 | 0.933 0.019 | 0.921 0.023 | 0.922 0.023 |
| PhishingWebsites | 0.959 0.003 | 0.957 0.003 | 0.955 0.003 | 0.971 0.003 | 0.970 0.004 |
| adult | 0.808 0.004 | 0.820 0.005 | 0.819 0.003 | 0.803 0.006 | 0.791 0.046 |
| bank-marketing | 0.819 0.013 | 0.861 0.005 | 0.860 0.005 | 0.839 0.008 | 0.846 0.006 |
| banknote-authentication | 0.993 0.005 | 1.000 0.000 | 0.999 0.002 | 1.000 0.000 | 1.000 0.000 |
| blood-transfusion-service | 0.625 0.053 | 0.685 0.034 | 0.685 0.033 | 0.685 0.036 | 0.683 0.031 |
| Comparison | Wilcoxon statistic | -value |
|---|---|---|
| vs. TeacherSBN | 368 | |
| vs. TeacherSBN | 323 | |
| vs. TeacherSBN | 341 | |
| vs. TeacherSBN | 314 |
| Dataset | feats | samples | min | max | min | max |
|---|---|---|---|---|---|---|
| Hidden neurons | sparsity (%) | |||||
| Internet-Advertisements | 1558 | 3279 | 2858 | 3552 | 96.56 | 97.11 |
| PhishingWebsites | 30 | 11055 | 1776 | 1920 | 58.86 | 61.30 |
| adult | 108 | 48842 | 8424 | 8505 | 79.91 | 81.89 |
| bank-marketing | 51 | 45211 | 5152 | 5215 | 66.92 | 69.74 |
| banknote-authentication | 4 | 1372 | 155 | 184 | 13.04 | 20.60 |
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 329 papers