Gradient-Boosted Decision Trees

Also known as GBDT

Gradient-boosted decision trees build an additive predictor by fitting each new tree to the loss or residual errors left by earlier trees; systems such as XGBoost, LightGBM, and CatBoost make this practical with histogram-based splits and efficient training. Researchers extend them with custom objectives, quantile and probabilistic prediction, constraints, calibration, and methods for privacy, interpretability, and faster inference. They remain strong on many tabular and structured forecasting tasks, but broad benchmarks show no model family wins everywhere, and deep or foundation models can be preferable in some settings. Key challenges include reliable performance under distribution shift, trustworthy explanations, costly hyperparameter tuning, and avoiding misleading results from weak validation designs.

Latest papers

56 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Affine-invariant Cubic Newton with Weak Learners
Oct 24, 2026Nikita Zozoulenko, Daniel Falkowski, Thomas Cass +1Curvature-Aware Optimization Gradient-Boosted Decision Trees
Imperial College London · Stockholm University · University of St. Gallen
TreeWalker: Partial Evaluation for Grouped Tree-Ensemble Inference
Oct 2, 2026Durmus Karatay, Richard NewmanEfficient Inference Inference-Time Optimization
Shopify
Comparing a gradient boosting algorithm to the GOES FDC for wildfire detection
Oct 1, 2026Asaf Vanunu, Boaz Nadler, Arnon KarnieliGradient-Boosted Decision Trees Wildfire Detection
The Albert Katz International School for Desert Studies, The Jacob Blaustein Institutes for Desert Research, Sde Boker Campus, Ben-Gurion University of the Negev, Midreshet Ben-Gurion 84990, Israel · The Goldman Sonnenfeldt School of Sustainability and Climate Change, Ben-Gurion University of the Negev, Beer Sheva 8410501, Israel · The Remote Sensing Laboratory, French Associates Institute for Agriculture and Biotechnology of Drylands, The Jacob Blaustein Institutes for Desert Research, Sde Boker Campus, Ben-Gurion University of the Negev, 84990, Israel +1
Fixing a Model That Learned Worse Cancer Means Lower Risk: Monotonic Constraints in Bladder Cancer Recurrence Prediction
Oct 1, 2026Saram Abbas, David Thomas, Naeem Soomro +3Counterfactual Evaluation Gradient-Boosted Decision Trees
School of Engineering, Newcastle University, Newcastle upon Tyne, UK · Electronics and Computer Science, University of Southampton, Southampton, UK · Department of Urology, Freeman Hospital, Newcastle upon Tyne, UK +1
Machine Learning for Invisible Dark Boson Searches at the Electron-Ion Collider
Sep 18, 2026Rojae Mighty, Ankush Reddy KanugantiHigh-Energy Physics Classification
Department of Physics and Astronomy, Stony Brook University, Stony Brook, NY 11794-3800, USA · Physics Department, Brookhaven National Laboratory, Upton, NY 11973-5000, USA
Are Gradient Boosting Models Suitable for Intermittent Demand Forecasting?
Sep 13, 2026Vladislav Kislinskii, Mazhar HameedTime Series Forecasting Gradient-Boosted Decision Trees
Gisma University of Applied Sciences, Potsdam, Germany
Predictors and Orchestrators: Parsimonious Machine Learning within an Agentic AI Harness for Multi-Horizon Karst Aquifer Forecasting
Sep 6, 2026Pramod Lekhak, Chetan Sharma, Hakan Başağaoğlu +2Time Series Forecasting Spatiotemporal Forecasting
School of Civil and Environmental Engineering & Construction Management, University of Texas at San Antonio, San Antonio, TX 78249, USA · Civil Engineering Department, Jodhpur Institute of Engineering and Technology, Jodhpur-342802, Rajasthan, India · Edwards Aquifer Authority, 900 E. Quincy St., San Antonio, TX 78215, USA
Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners
Sep 3, 2026David Milec, Spyridon Samothrakis, Michael Fairbank +1Gradient-Boosted Decision Trees Game-Playing Agents
Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic · School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom · Maastricht University, the Netherlands
KellyBoost: Growth-Optimal Portfolio Construction with Gradient-Boosted Trees
Aug 24, 2026Jiayu LiPortfolio Optimization Quantitative Finance
Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization
Aug 21, 2026Md Rezwanul Islam, Wael MohammedTime Series Forecasting Model Collapse
Field Nation LLC, Minneapolis, MN, USA
Leaf Values as Coordinates: Exact Contrastive Explanation for Gradient-Boosted Ensembles
Aug 19, 2026Emanuele LuzioCounterfactual Explanations Algorithmic Recourse
Independent Researcher
Convolution Smoothed Quantile Regression for XGBoost
Aug 15, 2026Mandy Yao, Meredith FranklinQuantile Regression Gradient-Boosted Decision Trees
University of Toronto · Department of Statistical Sciences, University of Toronto
FQTree: Fine-grained Quantization and Hardware Generation of Boosted Decision Trees
Aug 12, 2026Zhiqiang Que, Chang Sun, Haiyang Wang +6Gradient-Boosted Decision Trees Quantization-Aware Training
University of Bristol, UK · California Institute of Technology, USA · Imperial College London, UK
XGBoost "is all you need": the case of forecasting transmitted heat energy in District Heating Systems
Aug 11, 2026Milan ZdravkovićGradient-Boosted Decision Trees Energy-Efficient ML
Faculty of Mechanical Engineering University of Niš ul. Aleksandra Medvedeva 14, 18000 Niš, Serbia
Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study
Aug 7, 2026Yushu Zou, Ye Li, Johra Moosa +3Epidemic Forecasting Gradient-Boosted Decision Trees
Public Health Ontario, Toronto, Canada · Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Canada · Department of Laboratory Medicine and Pathobiology, University of Toronto, Canada +1
Double Descent in Gradient Boosting Decision Trees via Split-Candidate Scaling
Aug 4, 2026Ryuichi KanohDouble Descent Gradient-Boosted Decision Trees
1The University of Electro-Communications · 2National Institute of Informatics
Conditioning Tree-Based Diffusions and Flows for Probabilistic Tabular Regression
Jul 30, 2026Silas KoemenFlow Matching Gradient-Boosted Decision Trees
Independent Researcher
TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees
Jul 29, 2026James ChapmanGradient-Boosted Decision Trees Multi-View Learning
Spectrally Deconfounded Gradient Boosting
Jul 10, 2026Andrea Nava, Peter Bühlmann, Fabio SigristSpectral Regularization Gradient-Boosted Decision Trees
Seminar for Statistics, ETH Zürich, Zürich, Switzerland · IFZ, Lucerne University of Applied Sciences and Arts, Rotkreuz, Switzerland
path_boost: A Python Package for Interpretable Graph-Level Prediction using Path-Based Gradient Boosting
Jul 8, 2026Claudio Meggio, Johan Pensar, Riccardo De BinGradient-Boosted Decision Trees Graph Representation Learning
Department of Mathematics, University of Oslo, Oslo, 0371, Oslo, Norway.
Beyond Gradient-Based Attacks: Adversarial Robustness and Explainability Stability in Cybersecurity Classifiers
Jul 2, 2026Mona Rajhans, Vishal KhawareyGradient-Boosted Decision Trees SHAP Feature Attribution
Palo Alto Networks, Santa Clara, CA, USA · Quicken Inc., Menlo Park, CA, USA
CW-B: Class Weighted Boosting Framework for Imbalance Resilient Multi Class Cardiac Phenotyping
Jun 29, 2026Sijia Li, Xiaoyu Tan, Chen Zhan +3Class-Imbalanced Learning Gradient-Boosted Decision Trees
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China · Tencent YouTu Lab, Shanghai 200232, China · Department of Cardiology, Zhongshan Hospital, Fudan University, Shanghai Institute of Cardiovascular Diseases, Shanghai 200032, China +1
Gradient boosting with vector-valued leafs
Jun 28, 2026David CortesGradient-Boosted Decision Trees
EMA-FS: Accelerating GBDT Training via Gain-Informed Feature Screening
Jun 24, 2026Yan SongGradient-Boosted Decision Trees Tabular ML
PayPal, Inc.
Gradient boosting for extremes: sampling theory and application to insurance
Jun 12, 2026Stéphane Lhaut, Olivier LopezGradient-Boosted Decision Trees Heavy-Tailed Distributions
1CREST, CNRS, Ecole polytechnique, Groupe ENSAE-ENSAI, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France
ScoreStop: Gradient-based early stopping using functional score tests
Jun 1, 2026Oliver J. Hines, Christian L. HinesGradient-Boosted Decision Trees Early Stopping
Columbia University, NY, USA · Alan Turing Institute, London, UK
Practical Anonymous Two-Party Gradient Boosting Decision Tree
May 26, 2026Chenyu Huang, Fan Zhang, Minxin Du +6Secure Multi-Party Computation Gradient-Boosted Decision Trees
Tencent · Hong Kong Polytechnic University · Chinese University of Hong Kong +1
Trajectory-Based Difficulty Scoring for Reliable Learning on Tabular Data
May 23, 2026Tomer Lavi, Bracha Shapira, Nadav RappoportGradient-Boosted Decision Trees Tabular ML
Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel.
Hinge Regression Trees and HRT-Boost: Newton-Optimized Oblique Learning for Compact Tabular Models
May 22, 2026Hongyi Li, Jun Xu, Hong YanGradient-Boosted Decision Trees Decision Tree Learning
School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, 518055, China and with the Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments, Shenzhen, 518055, China · Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong
Pocket Foundation Models: Distilling TFMs into CPU-Ready Gradient-Boosted Trees
May 18, 2026Aditya Tanna, Nassim Bouarour, Mohamed Bouadi +2Tabular Foundation Models Gradient-Boosted Decision Trees
Lexsi Labs

Oct 24, 2026J/K move · Enter open · S save

Affine-invariant Cubic Newton with Weak Learners

Nikita Zozoulenko, Daniel Falkowski, Thomas Cass, Lukas Gonon

Imperial College London · Stockholm University · University of St. Gallen

Curvature-Aware Optimization Gradient-Boosted Decision Trees

Open paper AskPDF arXiv

Modern gradient boosting algorithms are based on Newton's method, which excels empirically but lacks global convergence guarantees. While recent work on Gradient-Regularized Newton (GRN) boosting achieved a global

\mathcal{O}(1/γ^{12}k^2)

rate, its analysis was not able to exploit the natural Hessian geometry and suffers from a heavy

γ^{12}

dependence on a worst-case `weak gradient edge'

γ

. In this paper, we tackle these theoretical drawbacks by generalizing the Affine-Invariant Cubic Newton (AICN) scheme to convex optimization with weak learners. Our analysis is based on average cosine values

\overlineΘ_k

along the optimization path and directional improvements

δ_k

in the Hessian-induced norm. We show that the weak-AICN update reduces to a damped weak-Newton step with an explicit stepsize determined by

δ_k

. Under semi-strong self-concordance, we establish a global

\mathcal{O}(1/\overlineΘ_k^4 k^2)

rate, significantly improving upon the weak-GRN bound. We further prove a local linear rate, global linear convergence for Hessian-dominated losses, cosine angle lower bounds, and present weak-AdaAICN, which adaptively tunes the cubic regularization. Finally, we demonstrate the broader applicability of this framework beyond gradient boosting by showing that Newton coordinate descent emerges naturally as a special case of optimization with weak learners.