Authors: Mulah Moriah, Franck Vermet, Pierre Ailliot, Juliette Legrand
Organizations: Euro-Institut d’Actuariat Jean Dieudonné (EURIA), UBO, Brest, France. · Univ Brest, CNRS UMR 6205, Laboratoire de Mathématiques de Bretagne Atlantique, Brest, France.
We propose SAMBoost (Spatially Aware Mesh Boosting), a gradient boosting method that combines tree-based covariate modelling with a stochastic partial differential equation (SPDE) latent spatial field through a single penalised objective. At each boosting iteration, the predictor is decomposed into separate covariate and spatial components. The method is motivated by applications in which the spatial effect is not a nuisance to be removed but a quantity of interest that must be recovered alongside the covariates. In insurance, for example, practitioners need both flexible nonlinear covariate effects and an interpretable geographic risk surface that can be reported on its own. Existing approaches often involve trade-offs, offering flexible covariate modelling but no explicit spatial field, rigorous spatial inference with a fixed additive covariate structure, or a combination of both at computational costs that limit practicality for large datasets. SAMBoost addresses these goals within the boosting loop, and the underlying mesh produces a sparse precision structure whose cost is largely independent of sample size. On simulations and on an open benchmark, SAMBoost matches or improves on the predictive accuracy of joint Gaussian process and tree boosting approaches. It is also faster and recovers the spatial field and covariate effects more accurately. When covariate effects include interactions, it outperforms additive Bayesian spatial models. Applied to a flood insurance portfolio with 4.9 million records, SAMBoost trains on the full training set without subsampling and runs about sixteen times faster than a Bayesian reference, with a consistent spatial risk surface. A public Python implementation is available.
Figures & tables
Figure 1: Simulation domains and data-generating processes.
AUC ↑
Log loss ↓
Brier ↓
rspa↑
rmsespa↓
rcov↑
rmsecov↓
Time (s)
SAMBoost
0.689±
0.020
0.637±
0.012
0.223±
0.005
0.951 ±
0.021
0.221 ±
0.035
0.749±
0.040
0.476 ±
0.033
0.8 ±
0.4
GPBoost
0.678±
0.021
0.645±
0.011
0.227±
0.005
0.926±
0.027
0.268±
0.037
0.707±
0.057
0.557±
0.065
80.7±
49.7
INLA
0.699 ±
0.019
0.629 ±
0.011
0.220 ±
0.005
0.951 ±
0.020
0.281±
0.082
0.829 ±
0.027
0.528±
0.083
40.8±
39.5
Table 1: Simulation 1 (classification). Prediction and dissociation metrics on the test set, reported as mean ± standard deviation over 30 seeds. Best value per column in bold.
Figure 2: Recovered spatial fields in Simulation 1 for a representative seed, namely the seed for which the spatial RMSE of SAMBoost is the median over the 30 replications.
RMSE ↓
rmsespa↓
rspa↑
rcov↑
rmsecov↓
Time (s)
SAMBoost
0.596 ±
0.020
0.160 ±
0.018
0.938±
0.020
0.895 ±
0.013
0.318 ±
0.020
6.6 ±
3.1
GPBoost
0.610±
0.020
0.207±
0.034
0.888±
0.042
0.866±
0.030
0.371±
0.032
111.9±
26.0
INLA
0.754±
0.024
0.207±
0.069
0.947 ±
0.016
0.625±
0.020
0.593±
0.032
72.5±
25.7
Table 2: Simulation 2 (regression with confounding). Prediction and dissociation metrics on the test set, reported as mean ± standard deviation over 30 seeds. Best value per column in bold.
Figure 3: Recovered spatial fields in Simulation 2 for a representative seed, namely the seed for which the spatial RMSE of SAMBoost is the median over the 30 replications. The cross marks the hotspot centre and the diagonal line marks the river corridor.
Covariate recovery
Spatial recovery
Covariate function
Method
rcov↑
rmsecov↓
rspa↑
rmsespa↓
Simple Fcov,simple∗
SAMBoost
0.931
0.259
0.939
0.159
INLA
0.924
0.283
0.962
0.194
GPBoost
0.912
0.305
0.889
0.207
Interaction-rich Fcov∗
SAMBoost
0.895
0.318
0.938
0.160
INLA
0.625
0.593
0.947
0.207
Table 3: Sensitivity analysis on covariate complexity. Dissociation metrics under a simple (nearly additive) and interaction-rich covariate function, with the same spatial design and river confounder. Reported as 30-seed means. Best value per column within each covariate setting in bold.
Figure 4: Fit time as a function of the number of training points N , with a fixed mesh and 100 trees. Points show the mean over three seeds and error bars the mean plus or minus the standard deviation.
Figure 5: Observed and predicted log prices (top row) and estimated spatial effects (bottom row) for the Lucas County data. Only the SAMBoost predictions are shown, as all methods produce visually identical predictions.
Method
1996
1997
1998
Mean
Fit time (s)
SAMBoost
0.246
0.239
0.247
0.244
319
GPBoost
0.248
0.243
0.247
0.246
850
INLA
0.357
0.339
0.281
0.326
1400
Table 4: Test-set RMSE (log price) and mean fit time on the Lucas County data under the expanding-window protocol. All methods use coordinates only through the spatial field. Best value per column in bold.
Figure 6: Sensitivity of SAMBoost to mesh resolution on the Lucas County data. Mean test-set RMSE (left axis) and mean fit time (right axis) for fixed numbers of SPDE mesh nodes. The grey band shows the GPBoost 95% bootstrap confidence interval around its mean RMSE (dashed line).
Method
Log loss ↓
Gini ↑
Fit time
SAMBoost
0.0297
0.695
119 s
INLA
0.0305
0.660
1996 s
GPBoost (500k sub.)
0.033
0.504
12,431 s
Table 5: Flood occurrence results on the temporal hold-out test set (2020-2022). The full portfolio contains approximately 4.9 million policy records, 0.22% positive rate. Log loss and Gini are exposure-weighted. Best value per column in bold.
Figure 7: Predicted flood probabilities (top) and estimated spatial effects (bottom) over the Occitanie region. GPBoost is trained on a random subsample of 500,000 records.
Bayesian Optimization (BO) is a popular method for efficiently optimizing expensive black-box objectives. However, BO utilizing standard Gaussian Processes is ill-suited for doubly stochastic Cox Processes that are often used in spatio-temporal problem spaces. We introduce INLA-SPDE Spatio-Temporal Bayesian Optimization (ISBO): the first scalable BO framework for spatio-temporal data, that models the log-intensity with a Log-Gaussian Cox Process(LGCP) and performs inference via Integrated Nested Laplace Approximation and Stochastic Partial Differential Equations (INLA-SPDE) approach. Using a Matern field on meshes yields a sparse Gaussian Markov Random Field, where INLA provides fast and accurate posterior inference throughout sequential optimization. ISBO stably locates high-intensity regions and the peak of the latent intensity with minimal evaluations. A time-varying Upper Confidence Bound acquisition with masking avoids revisits, while penalized-complexity priors regularize early rounds. Experiments on synthetic and real-world spatio-temporal datasets show accurate peak discovery, intensity recovery, and substantial speedups over an RKHS-based baseline, positioning ISBO as a practical choice for BO with point-process data.
Latent field models are essential for spatial statistics, but the posterior distribution of the latent field, even with a Gaussian process (GP) prior, is analytically intractable whenever the likelihood is non-Gaussian. Existing approximations are either too slow or too restrictive for large non-Gaussian datasets. We present a scalable variational inference framework that mirrors the sparse nearest-neighbor autoregressive structure of the prior in the variational family, uses normalizing-flow conditionals for expressiveness, and amortizes all variational parameters via a single shared neural conditioner. Doubly stochastic optimization with reduced ancestor sets achieves polylogarithmic complexity per iteration.
Paul F. V. Wiemann, Matthias Katzfuss
Department of Statistics, The Ohio State University, Columbus OH 43210, USA · Department of Statistics, University of Wisconsin–Madison, Madison WI 53706, USA
We introduce Gradient Boosted Mixed Models (GBMixed), a framework which extends boosting to clustered data by jointly modeling the mean and variance components in a linear mixed model via likelihood-based gradients. GBMixed estimates a nonparametric fixed effects function characterizing the overall mean of the response, while also allowing the random effects covariance matrix along with the residual variance to depend on covariates in a flexible manner. We demonstrate how GBMixed facilitates covariate-dependent random effect predictions, and subsequently point predictions and prediction intervals for individual treatment effects, that can adapt between population-level and cluster-level information. Simulations and applications to two real-world datasets demonstrate that GBMixed can accurately recover complex nonlinear fixed effect functions and covariate-dependent covariances in a linear mixed model, while also improving point and probabilistic predictive performance compared with several existing approaches such as parametric linear mixed models, Natural Gradient Boosting, and Gaussian Process Boosting.
Mitchell L. Prevett, Francis K. C. Hui, Zhi Yang Tho +2
Research School of Finance, Actuarial Studies and Statistics The Australian National University Canberra, ACT 2601, Australia