Gradient-Boosted Decision Trees

Also known as GBDT

Overview

Gradient-boosted decision trees build an additive predictor by fitting each new tree to the loss or residual errors left by earlier trees; systems such as XGBoost, LightGBM, and CatBoost make this practical with histogram-based splits and efficient training. Researchers extend them with custom objectives, quantile and probabilistic prediction, constraints, calibration, and methods for privacy, interpretability, and faster inference. They remain strong on many tabular and structured forecasting tasks, but broad benchmarks show no model family wins everywhere, and deep or foundation models can be preferable in some settings. Key challenges include reliable performance under distribution shift, trustworthy explanations, costly hyperparameter tuning, and avoiding misleading results from weak validation designs.

Latest papers

CardsList

64 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.1% of all new papers.

  1. Probabilistic Wind Power Forecasting with Tree-Based Machine Learning and Weather Ensembles

    Feb 13, 2026Max Bruninx, Diederik van Binsbergen, Timothy Verstraeten +2Gradient-Boosted Decision TreesEnsemble Learning

  2. Selecting Hyperparameters for Tree-Boosting

    Feb 5, 2026Floris Jan Koster, Fabio SigristGradient-Boosted Decision TreesBayesian Optimization

  3. Gradient Boosted Mixed Models: Flexible Estimation of Mean and Variance Components for Clustered Data

    Oct 31, 2025Mitchell L. Prevett, Francis K. C. Hui, Zhi Yang Tho +2Gradient-Boosted Decision TreesHeteroscedastic Regression

  4. Verifying Glitches in Decision Tree Ensembles

    Jul 19, 2025Satyankar Chandra, Harsh Chaurasia, Ashutosh Gupta +3Ensemble LearningGradient-Boosted Decision Trees