Conformal Risk Control for Safety-Critical Wildfire Evacuation Mapping: A Comparative Study of Tabular, Spatial, and Graph-Based Models

Baljinnyam Dayan

cs.LGMar 20, 2026

Conformal Risk Control for Safety-Critical Wildfire Evacuation Mapping: A Comparative Study of Tabular, Spatial, and Graph-Based Models

Authors: Baljinnyam Dayan

Organizations: Imperial College London, London, United Kingdom.

Abstract

Every wildfire prediction model deployed today shares a dangerous property: none of these methods provides formal guarantees on how much fire spread is missed. Despite extensive work on wildfire spread prediction using deep learning, no prior study has applied distribution-free safety guarantees to this domain, leaving evacuation planners reliant on probability thresholds with no formal assurance. We address this gap by presenting, to our knowledge, the first application of conformal risk control (CRC) to wildfire spread prediction, providing finite-sample guarantees on false negative rate (FNR <= 0.05). We expose a stark failure: across three model families of increasing complexity (tabular: LightGBM, AUROC 0.854; convolutional: Tiny U-Net, AUROC 0.969; and graph-based: Hybrid ResGNN-UNet, AUROC 0.964), standard thresholds capture only 7-72% of true fire spread. CRC eliminates this failure uniformly. Our central finding is that model architecture determines evacuation efficiency, while CRC determines safety: both spatial models with CRC achieve approximately 95% fire coverage while flagging only approximately 15% of total pixels, making them 4.2x more efficient than LightGBM, while the graph model's additional complexity over a simple U-Net yields no meaningful efficiency gain. We propose a shift-aware three-way CRC framework that assigns SAFE/MONITOR/EVACUATE zones for operational triage, and characterize a fundamental limitation of prevalence-weighted bounds under extreme class imbalance (approximately 5% fire prevalence). All models, calibration code, and evaluation pipelines are released for reproducibility.

Figures & tables

Figure 1: Why standard thresholds fail. A cross-section through a U-Net fire prediction shows probability peaking at ${\sim}0.8$ and decaying smoothly. The standard threshold ( $\hat{p}\geq 0.5$ , gray) misses the majority of the fire region; the CRC threshold ( $\hat{\lambda}=0.002$ , green) captures it entirely.

Figure 2: FNR as a function of decision threshold for all three models. The red dashed line marks

\alpha=0.05

. Both spatial models maintain safe FNR over a much wider threshold range than LightGBM, explaining their 4.2

\times

smaller evacuation zones under CRC. The near-overlapping U-Net and ResGNN-UNet curves illustrate diminishing returns from architectural complexity. Markers show each model’s CRC

\hat{\lambda}

Category	Feature	Variable	Unit
Topography	elevation	Terrain height	m
Weather	th	Wind direction	∘
	vs	Wind speed	m/s
	tmmn	Min temperature	K
	tmmx	Max temperature	K
	sph	Specific humidity	kg/kg

Table 1: NDWS input features (12 channels per

64{\times}64

pixel patch). All features are from remote-sensing and reanalysis sources ( Huot et al., 2022 ) .

Method	Params	$\hat{\lambda}$	Cov. $\uparrow$	FNR $\downarrow$	Set $\downarrow$	AUC $\uparrow$
LGBM ( $p{\geq}.5$ )	—	.500	.072 $\pm$ .02	.928 $\pm$ .02	.001	.854
LGBM+CRC	—	.003	.941 $\pm$ .01	.059 $\pm$ .01	.626 $\pm$ .01	.854
LGBM+3way	—	[0,.03]	1.00	.000	.040	.854
UNet ( $p{\geq}.5$ )	470K	.500	.390 $\pm$ .02	.610 $\pm$ .02	.007	.969
UNet+CRC	470K	.002	.947 $\pm$ .01	.053 $\pm$ .01	.149 $\pm$ .01	.969
UNet+3way	470K	[0,.02]	1.00	.000	.051	.969

Table 2: Test-set results across three models and three decision frameworks. CRC targets

\mathrm{FNR}\leq 0.05

. Three-way CRC uses cost-weighted

\alpha_{\mathrm{cw}}{=}0.50

c_{\mathrm{fn}}{=}5

c_{\mathrm{fp}}{=}1

\rho\in[0.9,1.1]

. All models trained from scratch on identical 70/15/15 splits. Values show

\pm

95% bootstrap CI (10K image resamples).

Figure 3: Same U-Net, different thresholds. Standard thresholding ( $\hat{p}\geq 0.5$ ) catches 56% of fires; CRC ( $\hat{\lambda}=0.002$ ) catches 100%. Bottom: probability cross-section showing how CRC captures low-probability fire pixels. Green shading marks pixels saved by CRC.

Figure 4: Three-model comparison. Left: AUROC. Center: without CRC, no model reaches 95% coverage. Right: with CRC, all three meet the safety target; both spatial models achieve

{\sim}4\times

smaller evacuation zones than LightGBM.

Figure 5: The safety gap across three test samples (0.5%, 2.7%, 7.8% fire). Col. 1: ground truth. Col. 2: standard threshold, with missed fire in red (100%, 73%, 34% missed). Col. 3: CRC threshold, with improved detection (green). Col. 4: three-way zones (SAFE/MONITOR/EVACUATE).

Figure 6: Per-pixel fire risk gradient. Col. 1: ground truth. Col. 2: continuous probability heatmap. Col. 3: probability bands with CRC contour (green dashed) and EVACUATE boundary (red). Col. 4: calibrated risk gradient for operational mapping.

Model	SAFE	MONITOR	EVACUATE	$B_{\mathrm{pw}}$
LightGBM	0.0%	95.96%	4.04%	0.988
U-Net	0.0%	94.90%	5.10%	0.987
ResGNN-UNet	0.0%	85.15%	14.85%	0.987

Table 3: Three-way zone fractions on the test set. The SAFE zone collapses because

B_{\mathrm{pw}}=\max(c_{\mathrm{fn}}\pi_{1},c_{\mathrm{fp}}\pi_{0})=0.95

is dominated by

c_{\mathrm{fp}}\pi_{0}

under

{\sim}5\%

fire prevalence.

Model	Params	Epochs	Optim	Val Loss	AUC
LightGBM	—	100 iter	GBM	—	.854
Tiny U-Net	470K	50	AdamW	0.048	.969
ResGNN-UNet	229K	23 $\dagger$	AdamW	0.131	.964
$\dagger$ Early-stopped from 30 (patience=7, best epoch 16).

Table 4: Model training summary. All models trained from scratch on identical NDWS splits.

Figure 7: Tiny U-Net training curves: BCE loss and cosine annealing LR over 50 epochs. Smooth convergence with minimal train–val gap.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Safety vs. efficiency for all nine configurations. Top: coverage (safety), where only CRC and three-way methods achieve

{\geq}95\%

. Bottom: set size (efficiency), where both spatial models with CRC achieve the best trade-off.

Figure 9: Bare model vs. CRC across four test samples. Fire catch rate improves from 8% $\to$ 98%, 0% $\to$ 83%, 56% $\to$ 100%, 48% $\to$ 100%.

Figure 10: Qualitative prediction maps: ground truth, probabilities, standard prediction, CRC prediction, three-way zones, and error analysis.

Explore similar work

May 4, 2026

Boundary-Aware Uncertainty Quantification for Wildfire Spread Prediction

Reliable wildfire spread prediction is vital for risk-aware emergency planning, yet most deep learning models lack principled uncertainty quantification (UQ). Further, for boundary-sensitive cases like wildfire spread, evaluating models with global metrics alone is often insufficient. To shift the focus of UQ evaluation toward a more operationally relevant approach, the Fire-Centered Evaluation Region (FCER) framework is introduced as a spatially conditioned protocol to characterize UQ within critical fire zones. Using FCER, an Ensemble is compared against an distilled single-pass student model on the WildfireSpreadTS dataset. The student model demonstrates comparable calibration and complementary uncertainty ranking in boundary-relevant regimes. Code is available at https://github.com/jonasvilhofunk/WildfireUQ-FCER

Jonas V. Funk

Independent Research 76199 Karlsruhe, Germany

Uncertainty Quantification Wildfire Forecasting Spatiotemporal Forecasting Uncertainty Calibration

Ask PDF

Aug 29, 2026

WildfireSpreadBench: The Metric Decides the Model in Wildfire Spread Prediction

Machine learning is being increasingly used to predict where active wildfires will burn the following day, helping inform evacuation boundaries and containment lines. Most models are evaluated using Average Precision (AP), which summarizes performance across all decision thresholds, although acting on a forecast requires choosing one. We benchmarked five discriminative architectures and one generative model on WildfireSpreadTS using a shared evaluation pipeline and two input configurations. We found that model rankings varied depending on whether performance was measured by AP or by threshold-dependent metrics like F1 and IoU. At the default 0.5 threshold, the highest-AP model flagged 4 to 5 times the observed active-fire area and ranked fifth of six on F1 and IoU, and the most recall-heavy model flagged 16 to 23 times. The three models whose flagged area stayed close to the observed area had AP scores 24 to 37% lower. Across models, we identified three distinct prediction profiles: over-predicting, balanced, and under-predicting, which AP alone could not distinguish. Expanding the input from 7 to 23 channels changed AP by 0.03 on average, against a 0.21 to 0.24 spread across models. Ultimately, these results show that AP alone can favor models whose predictions are poorly suited for operational wildfire forecasting.

Arin Gopakumar, Marco Pannozzo

UC Berkeley · Purdue University

Wildfire Forecasting Forecasting Benchmarks

Ask PDF

Oct 13, 2026

FireGraphNet: Sub-daily wildfire spread prediction with directed graph neural networks

Wildfire decisions are made in hours, yet most learned spread models predict daily on targets including burned area. We propose FireGraphNet which formulates 12-hour growth as node classification on directed H3 graphs, with candidates ahead of the front and wind alignment, terrain, fuel continuity and barriers on edges. On satellite perimeters it achieves average precision 1.7-1.9x the strongest distance baseline on test California fires and 1.62x out of state. We show this skill comes from message passing rather than cell features or the aggregator, is highest on young fires in state, is carried deep along the wind by its edges, and is hidden by pooling. A convolutional network matches this ranking once trained on new growth, at worse calibration, so candidate construction, target definition, and aggregation shape conclusions about model skill. Editing fuel and barrier attributes reduces predicted spread more under guided than random placement, extending FireGraphNet to operational intervention analysis.

Aiyshwariya Paulvannan Kanmani, Miguel Esparza, Ali Mostafavi

Urban Resilience.AI Lab, Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77840, United States of America

Wildfire Forecasting Spatiotemporal GNNs Graph Neural Networks

Ask PDF