Interpretable ML

ML: Machine Learning

Interpretable machine learning aims to make predictions understandable enough for people to inspect, challenge, and use, especially in high-stakes decisions and scientific discovery. Approaches include inherently transparent decision trees, rule sets, additive models, and symbolic formulas, as well as concept- and prototype-based models that connect predictions to human-readable ideas or examples. Other methods explain complex models after training with feature attributions, local surrogates, or analyses of internal representations, while newer work also builds in uncertainty, domain knowledge, and opportunities for human intervention. A central challenge is balancing predictive accuracy with explanations that are faithful, stable under changes in data, and meaningful to the people relying on them; evaluation increasingly tests these qualities rather than treating a plausible explanation as sufficient.

Latest papers

310 papers26 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

Oct 2, 2024

MARS: A neurosymbolic approach for interpretable drug discovery

Background: Neurosymbolic (NeSy) artificial intelligence describes the combination of logic or rule-based techniques with neural networks. Compared to neural approaches, NeSy methods often possess enhanced interpretability, which is particularly promising for biomedical applications like drug discovery. However, no clear guidelines exist to assess the biological plausibility of model interpretations. Methods: To assess interpretability in the context of drug discovery, we devise a novel prediction task, called drug mechanism-of-action (MoA) deconvolution, with an associated, tailored knowledge graph (KG), MoA-net. We then develop the MoA Retrieval System (MARS), a NeSy approach for drug discovery which leverages logical rules with learned rule weights. Results: Using MARS' interpretable features alongside domain knowledge, we find that MARS and other NeSy approaches on KGs are susceptible to reasoning shortcuts, in which the prediction of true labels is driven by ``degree-bias'' rather than the domain-based rules. Subsequently, we demonstrate ways to identify and mitigate this. Thereafter, MARS achieves performance on par with current state-of-the-art models while producing model interpretations aligned with known MoAs. Conclusion: Through MARS, we showcase the novel task of computational MoA deconvolution. Our results emphasize the importance of using interpretable models, like NeSy ones, for applications in drug discovery. Specifically, by identifying and mitigating reasoning shortcuts, MARS MoA predictions which are biologically meaningful and, therefore, more reliable for downstream drug discovery research.

Lauren Nicole DeLong, Yojana Gadiya, Paola Galdi +2

School of Informatics, University of Edinburgh, 10 Crichton St, Edinburgh, EH8 9AB, United Kingdom. · Cancer Research UK Scotland Institute, Switchback Road, Glasgow, G61 1BD, United Kingdom. · Enveda Therapeutics, 5700 Flatiron Parkway, Boulder, 80301, Colorado, United States of America.

Interpretable ML AI-Assisted Scientific Research Neuro-Symbolic Reasoning Shortcut Learning Drug Discovery

Ask PDF

Sep 1, 2024

Interpretable Clustering: A Survey

In recent years, much of the research on clustering algorithms has primarily focused on enhancing their accuracy and efficiency, frequently at the expense of interpretability. However, as these methods are increasingly being applied in high-stakes domains such as healthcare, finance, and autonomous systems, the need of transparent and interpretable clustering outcomes has become a critical concern. This is not only necessary for gaining user trust but also for satisfying the growing ethical and regulatory demands in these fields. Ensuring that decisions derived from clustering algorithms can be clearly understood and justified is now a fundamental requirement. To address this need, this paper provides a comprehensive and structured review of the current state of explainable clustering algorithms, identifying key criteria to distinguish between various methods. These insights can effectively assist researchers in making informed decisions about the most suitable explainable clustering methods for specific application contexts, while also promoting the development and adoption of clustering algorithms that are both efficient and transparent. For convenient access and reference, an open repository organizes representative and emerging interpretable clustering methods under the taxonomy proposed in this survey, available at https://hulianyu.xyz/Interpretable-Clustering-Repository

Lianyu Hu, Mudi Jiang, Junjie Dong +2

College of Information Science and Engineering, Henan University of Technology, China · School of Software, Dalian University of Technology, China · Xinchang Power Supply Company, State Grid Corporation of China, China

Clustering Interpretable ML

Ask PDF

Jul 30, 2024

MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network

Cardiac arrhythmia, a condition characterized by irregular heartbeats, often serves as an early indication of various heart ailments. With the advent of deep learning, numerous innovative models have been introduced for diagnosing arrhythmias using Electrocardiogram (ECG) signals. However, recent studies solely focus on the performance of models, neglecting the interpretation of their results. This leads to a considerable lack of transparency, posing a significant risk in the actual diagnostic process. To solve this problem, this paper introduces MambaCapsule, a deep neural networks for ECG arrhythmias classification, which increases the explainability of the model while enhancing the accuracy.Our model utilizes Mamba for feature extraction and Capsule networks for prediction, providing not only a confidence score but also signal features. Akin to the processing mechanism of human brain, the model learns signal features and their relationship between them by reconstructing ECG signals in the predicted selection. The model evaluation was conducted on MIT-BIH and PTB dataset, following the AAMI standard. MambaCapsule has achieved a total accuracy of 99.54% and 99.59% on the test sets respectively. These results demonstrate the promising performance of under the standard test protocol.

Yinlong Xu, Zitai Kong, Yixuan Wu +5

State Key Laboratory of Transvascular Implantation Devices and TIDRI, Zhejiang University, Hangzhou, China · School of Public Health, Zhejiang University and Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence, Hangzhu, China · State Key Laboratory of Transvascular Implantation Devices of The Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University, Hangzhou, China +3

Interpretable ML Electrocardiogram Classification

Ask PDF

Jan 18, 2024

Probabilistic Truly Unordered Rule Sets

Rule set learning has recently been frequently revisited because of its interpretability. Existing methods have several shortcomings though. First, most existing methods impose orders among rules, either explicitly or implicitly, which makes the models less comprehensible. Second, due to the difficulty of handling conflicts caused by overlaps (i.e., instances covered by multiple rules), existing methods often do not consider probabilistic rules. Third, learning classification rules for multi-class target is understudied, as most existing methods focus on binary classification or multi-class classification via the

one-versus-rest" approach. To address these shortcomings, we propose TURS, for Truly Unordered Rule Sets. To resolve conflicts caused by overlapping rules, we propose a novel model that exploits the probabilistic properties of our rule sets, with the intuition of only allowing rules to overlap if they have similar probabilistic outputs. We next formalize the problem of learning a TURS model based on the MDL principle and develop a carefully designed heuristic algorithm. We benchmark against a wide range of rule-based methods and demonstrate that our method learns rule sets that have lower model complexity and highly competitive predictive performance. In addition, we empirically show that rules in our model are empirically

independent" and hence truly unordered.

Lincen Yang, Matthijs van Leeuwen

LIACS, Leiden University Niels Bohrweg 1, 2333 CA, Leiden, The Netherlands

Interpretable ML Multiclass Classification Classification Rule-Based Classification

Ask PDF

Nov 29, 2023

Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-Fields

The current trend in the literature on Time Series Classification is to develop increasingly accurate algorithms by combining multiple models in ensemble hybrids, representing time series in complex and expressive feature spaces, and extracting features from different representations of the same time series. As a consequence of this focus on predictive performance, the best time series classifiers are black-box models, which are not understandable from a human standpoint. Even the approaches that are regarded as interpretable, such as shapelet-based ones, rely on randomization to maintain computational efficiency. This poses challenges for interpretability, as the explanation can change from run to run. Given these limitations, we propose the Bag-Of-Receptive-Field (BORF), a fast, interpretable, and deterministic time series transform. Building upon the classical Bag-Of-Patterns, we bridge the gap between convolutional operators and discretization, enhancing the Symbolic Aggregate Approximation (SAX) with dilation and stride, which can more effectively capture temporal patterns at multiple scales. We propose an algorithmic speedup that reduces the time complexity associated with SAX-based classifiers, allowing the extension of the Bag-Of-Patterns to the more flexible Bag-Of-Receptive-Fields, represented as a sparse multivariate tensor. The empirical results from testing our proposal on more than 150 univariate and multivariate classification datasets demonstrate good accuracy and great computational efficiency compared to traditional SAX-based methods and state-of-the-art time series classifiers, while providing easy-to-understand explanations.

Francesco Spinnato, Riccardo Guidotti, Anna Monreale +1

Scuola Normale Superiore, Pisa, Italy · University of Pisa, Pisa, Italy · ISTI-CNR, Pisa, Italy

Time Series Classification Interpretable ML Multiscale Time-Series Modeling Interpretable Time-Series Classification

Ask PDF

Nov 26, 2023

Topology-enhanced machine learning for speech signal processing

In artificial-intelligence-aided signal processing, existing deep learning models often exhibit a black-box structure. Here, conceptually beyond spectral analysis, we demonstrate that topological methods not only effectively capture intrinsic and complex structural information but can also enhance neural networks. We provide a transparent methodology, TopCap, to capture topological features inherent in time series for basic machine learning. Compared to prior approaches, we obtain descriptors that probe finer information such as the vibration of a time series. Notably, in classifying voiced and voiceless consonants, TopCap achieves an accuracy consistently standing in comparison with neural network models. Moreover, by integrating TopCap features into those neural networks, our approach improves upon state-of-the-art methods in terms of robustness against noise, as well as accuracy, stability, convergence of loss function, and interpretability.

Pingyao Feng, Qingrui Qu, Haiyu Zhang +4

Department of Mathematics, Southern University of Science and Technology, Shenzhen, China. · Department of Mathematics, North Carolina State University, Raleigh, NC, USA. · School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen), Shenzhen, China. +1

Interpretable ML Speech Processing Audio Classification Topological Data Analysis

Ask PDF

Nov 3, 2023

Distributionally Robust Regularization of Sparse Integer Programming Trained Learning Models

Building explainable machine learning models is crucial for human users to be able to interpret the proposed statistical relationship obtained from the training data. Mixed-integer optimization formulations are often used to train such models with explicit sparsity constraints, aiming to strike the right trade-off between sparsity and prediction accuracy. Existing methods to find the right choice of sparsity -- e.g., via cross-validation -- are computationally expensive. For convex model training formulations, recent advances in distributionally robust optimization (DRO) provide strong generalization while sidestepping this computational burden. We describe an extension of such regularization via DRO to mixed-integer sparse programs, providing statistical guarantees as a function of an associated sparsity parameter of the formulation. We illustrate the use of this approach in the case of building explainable binary classification models using sets of feature value rules.

Sanjeeb Dash, Soumyadip Ghosh, Joao Goncalves +1

IBM Research, Yorktown Heights, NY 10598

Rule-Based Classification Interpretable ML Distributionally Robust Optimization Mixed-Integer Linear Programming

Ask PDF

Apr 21, 2023

Granular-ball computing: an efficient, robust, and interpretable adaptive multi-granularity representation and computation method

To overcome the limitations of point-based inputs, overly fine computation and limited adaptability in existing artificial intelligence methods, Guoyin Wang and Shuyin Xia proposed granular-ball computing as a new artificial intelligence learning paradigm. Unlike traditional clustering, which mainly performs macro-level grouping, granular-ball computing uses differently sized hyperspheres, termed granular balls, as mesoscopic representation units; rectangles and ellipsoids can serve as approximate balls in low-dimensional spaces. It adaptively fits arbitrary data distributions, replacing traditional artificial intelligence computation based on fine-grained point inputs or single-granularity modeling and establishing a new theoretical paradigm for artificial intelligence based on granular balls. It aims to build an end-to-end multigranular artificial intelligence framework that improves the efficiency, robustness, and interpretability of existing methods. Recently, this theory has advanced rapidly and yielded representative results, yet it still lacks a unified model for systematic summarization. Accordingly, this article first proposes a general representation model of granular-ball computing within a unified descriptive framework and systematically reviews its fundamental ideas and advances in granular-ball computing across granular-ball supervised learning, granular-ball unsupervised learning, approximate granular-ball representation and computation, granular-ball deep learning based on latent-space granulation, granular-ball graph learning, and granular-ballinterdisciplinary research. Further, it identifies open challenges and outlines future research directions.

Shuyin Xia, Guoyin Wang, Xinbo Gao +2

Chongqing Key Laboratory of Computational Intelligence, Key Laboratory of Cyberspace Big Data Intelligent Security, Ministry of Education, and Key Laboratory of Big Data Intelligent Computing, Chongqing University of Posts and Telecommunications, 400065, Chongqing, China · College of Computer and Information Science, the National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing 401331, China

Interpretable ML Adaptive Computation

Ask PDF

Jul 31, 2022

Rule Extraction in Machine Learning: Chat Incremental Pattern Constructor

Rule extraction is a central problem in interpretable machine learning because it seeks to convert opaque predictive behavior into human-readable symbolic structure. This paper presents Chat Incremental Pattern Constructor (ChatIPC), a lightweight incremental symbolic learning system that extracts ordered token-transition rules from text, enriches them with definition-based expansion, and constructs responses by similarity-guided candidate selection. The system may be viewed as a rule extractor operating over a token graph rather than a conventional classifier. I formalize the knowledge base, definition expansion, candidate scoring, repetition control, English-rule heuristics, and response construction mechanisms used by ChatIPC. I further situate the method within the literature on rule extraction, decision tree induction, association rules, interpretable machine learning, and sequence construction. The updated C++ code implementation of ChatIPC is also reviewed in detail: it parses an embedded dictionary, normalizes lexical keys, caches definition tokens and part-of-speech tags, computes Jaccard scores on bitsets, applies heuristic linguistic bonuses, and persists the knowledge base with a versioned binary format. The paper emphasizes mathematical formulation and algorithmic clarity, and it provides pseudocode for the learning, scoring, and construction algorithms.

Caleb Princewill Nwokocha

Interpretable ML Incremental Learning Text Generation

Ask PDF

Oct 28, 2020

Deep Learning for Individual Heterogeneity

This paper integrates deep neural networks (DNNs) into structural models to increase flexibility and capture rich heterogeneity while preserving interpretability. Economic (or scientific or domain-restricted) structure and machine learning are complements in empirical modeling, not substitutes: DNNs provide the capacity to learn complex, nonlinear heterogeneity, while the structure ensures the estimates remain interpretable and suitable for decision-making and policy analysis. We start with a standard parametric structural model and then enrich its parameters into fully flexible functions, which are estimated using a DNN with the model structure built in. We illustrate our framework with an application to demand estimation in consumer choice. We show that by enriching a demand model we can capture rich heterogeneity exploit it to create personalized pricing. Optimization is not possible without structure, but cannot be heterogeneous without machine learning. The same lessons apply to precision dosing, adaptive treatment, educational testing, and other targeting settings. We provide theoretical justification for our proposed methodology: nonasymptotic bounds and a novel and general influence function for feasible inference via double machine learning, so that the latter can be easily applied in numerous new contexts. These results may be of interest in other contexts as they generalize prior work.

Max H. Farrell, Tengyuan Liang, Sanjog Misra

Department of Economics, UC Santa Barbara · Booth School of Business, University of Chicago

Interpretable ML

Ask PDF