Neural learners for classification and regression are often criticized as black boxes. We propose a probabilistic decomposition of neural learners that accounts for heterogeneous predictability across the feature space. A training procedure combining the EM algorithm, stochastic gradient descent, and maximum likelihood estimation is developed to jointly learn the predictable component and an input-dependent reliability score. We show that the optimal decomposed learner recovers the response distribution within the predictable component, while the reliability score characterizes the predictability of individual observations. We further extend the framework to incorporate localized feature transformations. Experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed framework.
Figures & tables
Figure 1: (a) Labeled dataset across the feature space. (b) Accuracy of a neural classifier in different regions. (c) High-reliability instances (in green, with p(z=1∣x)>0.5 ) identified by our model.
Figure 2: Overall architecture of our framework.
Figure 3: (a) Labeled training dataset in feature space. (b) True reliability p(z=1∣x) . (c) True predictable component distribution p(y=1∣x,z=1) .
Figure 4: (a) Estimated reliability on test data. (b) Estimated predictable component on test data. (c) Theoretical (green) and learned (blue) local feature transformations. (d) Classification accuracy on test data with different reliability thresholds.
Figure 5: (a) Classification accuracy with different reliability thresholds. (b) Classification accuracy versus reliability thresholds with x2 removed. (c) Classification accuracy versus reliability thresholds with x4 added. (b) Predictation RMSE with different reliability thresholds.
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.