stat.MLOct 30, 2024

Very fast Bayesian Additive Regression Trees on GPU

Authors: Giacomo Petrillo

Organizations: Department of Statistics, Computer Science, Applications “Giuseppe Parenti” (DISIA), University of Florence, Italy.

Abstract

Bayesian Additive Regression Trees (BART) is a nonparametric Bayesian regression technique based on an ensemble of decision trees. It is part of the toolbox of many statisticians. The overall statistical quality of the regression is typically higher than other generic alternatives, and it requires less manual tuning, making it a good default choice. However, it is a niche method compared to natural competitors such as XGBoost, due to the longer running time, making sample sizes above 10000-100000 a nuisance. We present a GPU implementation of BART, up to 500x faster than the previous state of the art, making BART fast enough to be a viable option on large datasets, and showing that BART parallelizes better than its non-Bayesian counterparts. We also find that, to be as practical as tree boosting on large datasets, BART would need a scalable way to infer the number of trees, which we do not provide. Our implementation is available in the Python package bartz.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. CRAFT-GBM: Selecting Best Features for Gradient-Boosted Machine Learning

    Nov 9, 2026Basil Kuok, Yi Zhou, Yao Xuan +2Gradient-Boosted Decision TreesFeature Selection