On the Limits of Interpretable Machine Learning in Quintic Root Classification
Authors: Rohan Thomas, Majid Bani-Yaghoub
Organizations: Division of Computing, Analytics & Mathematics, School of Science and Engineering, University of Missouri-Kansas City, 5100 Rockhill Rd., Kansas City, Missouri 64110, USA
Can Machine Learning (ML) autonomously recover interpretable mathematical structure from raw numerical data? We aim to answer this question using the classification of real-root configurations of polynomials up to degree five as a structured benchmark. We tested an extensive set of ML models, including decision trees, logistic regression, support vector machines, random forest, gradient boosting, XGBoost, symbolic regression, and neural networks. Neural networks achieved strong in-distribution performance on quintic classification using raw coefficients alone (84.3% + or - 0.9% balanced accuracy), whereas decision trees perform substantially worse (59.9% + or - 0.9%). However, when provided with an explicit feature capturing sign changes at critical points, decision trees match neural performance (84.2% + or - 1.2%) and yield explicit classification rules. Knowledge distillation reveals that this single invariant accounts for 97.5% of the extracted decision structure. Out-of-distribution, data-efficiency, and noise robustness analyses indicate that neural networks learn continuous, data-dependent geometric approximations of the decision boundary rather than recovering scale-invariant symbolic rules. This distinction between geometric approximation and symbolic invariance explains the gap between predictive performance and interpretability observed across models. Although high predictive accuracy is attainable, we find no evidence that the evaluated ML models autonomously recover discrete, human-interpretable mathematical rules from raw coefficients. These results suggest that, in structured mathematical domains, interpretability may require explicit structural inductive bias rather than purely data-driven approximation.
Figures & tables
Model
Description
Decision Trees (CART)
Interpretable models that produce explicit if-then rules, allowing direct examination of learned structure.
Logistic Regression
Linear classifier to test whether linear decision boundaries suffice for this problem.
Support Vector Machines (SVM)
RBF kernel to explore whether kernel methods could find separating boundaries.
Random Forest
Ensemble of decision trees to assess whether aggregation improves performance.
Gradient Boosting
Sequential ensemble method for comparison.
XGBoost
Optimized gradient boosting implementation.
Table 1: Overview of Machine Learning models used in this study to evaluate their interpretability
Polynomial
Accuracy (%)
Observations
Quadratic
97.3±4.2
Discovered discriminant rule (100% in 2/5 trials)
Cubic
94.4±1.6
Discovered highly accurate but complex heuristic
Quartic
47.3±5.8
Consistently failed; relied on simple linear heuristic
Quintic
53.5±2.9
Failed to find interpretable rules; relied on complex heuristics
Table 2: Summary of Symbolic Regression Performance by Polynomial Degree
Table 4: Performance comparison of neural networks and decision trees when additional feature “crit8” added.
Metric
Value
Interpretation
NN Test Balanced Accuracy
87.0% ± 1.2%
Robust feature learning
Tree Test Fidelity
98.9% ± 0.1%
Successful logic transfer
Tree Standalone Accuracy
84.6% ± 1.2%
Generalizable rule extraction
Table 5: Quantitative results for the Knowledge Distillation process from Neural Networks (NN) to Decision Trees. The high fidelity indicates that the interpretable tree successfully approximates the neural model’s decision logic.
Feature Set
Neural Networks
Decision Trees
Raw coefficients
84.3% ± 0.9%
59.9% ± 0.9%
+ Sturm sequences
85.2% ± 1.1%
62.1% ± 4.1%
+ Newton sums
83.0% ± 1.3%
63.0% ± 4.2%
+ Critical points
89.3% ± 0.7%
84.2% ± 1.2%
+ Hybrid symbolic
80.5% ± 1.4%
61.4% ± 4.0%
All 63 combined
87.0% ± 1.2%
68.2% ± 4.5%
Table 6: Comparative performance across different mathematical feature sets for quintic polynomials. Bold values indicate the optimal feature configuration, where Critical points provide the most significant gain in interpretability and accuracy.
Figure 1: Behavioral divergence between neural approximation (Top Row) and discrete mathematical execution (Bottom Row). Error bars indicate 95% Confidence Intervals. Left: Invariant decision trees maintain perfectly stable, scale-invariant bounds out-of-distribution, whereas neural approximations decay. Middle: Trees lock in boundaries with mere dozens of samples, while networks require thousands. Right: Both models degrade identically under noise, reflecting the fundamental alteration of the true polynomial roots.