Differentiable Tree

Differentiable trees represent a burgeoning area of research aiming to combine the interpretability of traditional decision trees with the flexibility and power of neural networks. Current efforts focus on developing efficient algorithms for training these models, including architectures based on transformers, tensor product representations, and novel tree structures optimized for specific tasks like feature selection and symbolic regression. This approach holds significant promise for improving the interpretability and scalability of machine learning models across diverse applications, from biological tree inference to reinforcement learning and high-dimensional data analysis. The resulting models offer a balance between accuracy and explainability, addressing limitations of both purely symbolic and purely neural methods.

Papers

October 7, 2024

A Review of Artificial Intelligence based Biological-Tree Construction: Priorities, Methods, Applications and Trends
Zelin Zang, Yongjie Xu, Chenrui Duan, Jinlin Wu, Stan Z. Li, Zhen Lei
Artificial Intelligence Financial Application NCD Method Priority User Differentiable Tree

July 2, 2024

Terminating Differentiable Tree Experts
Jonathan Thomm, Michael Hersche, Giacomo Camposampiero, Aleksandar Terzić, Bernhard Schölkopf, Abbas Rahimi
Transformer Layer Differentiable Tree Individual Coniferous Tree

May 23, 2024

Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning
Hector Kohler, Quentin Delfosse, Riad Akrour, Kristian Kersting, Philippe Preux
Reinforcement Learning Inherent Interpretability Interpretable Way Interpretable Policy Differentiable Tree Programmatic Policy

December 17, 2023

Discretionary Trees: Understanding Street-Level Bureaucracy via Machine Learning
Gaurab Pokharel, Sanmay Das, Patrick J. Fowler
Machine Learning Long Term Cooperation Decision Policy Differentiable Tree

October 28, 2023

End-to-end Feature Selection Approach for Learning Skinny Trees
Shibal Ibrahim, Kayhan Behdin, Rahul Mazumder
Feature Selection Ensemble Learning Tree Ensemble LightGBM Model Joint Feature Interpretable Tree Differentiable Tree

July 23, 2023

NCART: Neural Classification and Regression Tree for Tabular Data
Jiaqi Luo, Shixin Xu
Deep Learning Model Decision Tree Tabular Data Interpretable Neural Network Regression Tree Neural Classifier Neural Tree Differentiable Tree

June 1, 2023

Differentiable Tree Operations Promote Compositional Generalization
Paul Soulos, Edward Hu, Kate McCurdy, Yunmo Chen, Roland Fernandez, Paul Smolensky, Jianfeng Gao
Compositional Generalization Differentiable Tree Tree Transformer Structure to Structure

April 18, 2023

Differentiable Genetic Programming for High-dimensional Symbolic Regression
Peng Zeng, Xiaotian Song, Andrew Lensen, Yuwei Ou, Yanan Sun, Mengjie Zhang, Jiancheng Lv
Gradient Based Symbolic Regression Genetic Programming Differentiable Tree Genetic Programming Tree

May 19, 2022

Flexible Modeling and Multitask Learning using Differentiable Tree Ensembles
Shibal Ibrahim, Hussein Hazimeh, Rahul Mazumder
Multi Task Learning Multitask Learning Tree Ensemble Flexible Model Differentiable Tree