Beyond Point Predictions: Uncertainty-Aware Satellite Poverty Mapping for Public Policy
Authors: Markus B. Pettersson, James Bailie, Mohammad Kakooei, Eagon Meng, Adel Daoud
Organizations: Division of Data Science and AI, Department of Computer Science and Engineering, Chalmers University of Technology and the University of Gothenburg, Gothenburg, Sweden · Institute for Analytical Sociology, Linköping University, Norrköping, Sweden · AI & Global Development Lab, Linköping University, Norrköping, Sweden · Geomatics, Department of Environmental and Life Sciences, Karlstad University, Karlstad, Sweden · Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
Despite their critical importance for policy and research, high-resolution poverty data remain limited across much of Africa. Machine learning (ML) with earth observation (EO) imagery has recently emerged as a way to supplement these data by predicting (i.e., estimating) poverty where it has not been directly measured. Yet to be used reliably, decision-makers and analysts need assurances that they will not be misled by the errors in these predictions. To meet this need, we develop an uncertainty-aware EO-ML method for poverty mapping based on simultaneous quantile regression and a novel form of conformal prediction. Using a spatiotemporal transformer trained on sequences of Landsat and nighttime-light images, we produce prediction intervals for neighborhood-level International Wealth Index estimates across Africa which are statistically guaranteed to achieve their desired coverage rates. While our method's point-prediction performance matches the state of the art, its prediction intervals are wider than might be expected given its high R2 of 0.75. However, other models of similar accuracy likely suffer from comparable uncertainty, pointing to an inherent limitation: even with its remarkably high explanatory power, EO-ML cannot naively be relied upon for policy-making, such as when designing poverty-targeting programs. To handle this challenge, we develop a procedure to efficiently allocate aid using both ground-truth surveys and model predictions while provably ensuring the risk of excluding eligible neighborhoods remains below a prespecified level. In simulations, this approach delivers substantially more aid per eligible recipient than other strategies, thereby demonstrating that EO-ML can indeed be a reliable supplement to traditional data sources.
Figures & tables
Figure 1: Pipeline for Conformalized Thresholding (CT) . (A) The simultaneous quantile-regression model maps Landsat and nighttime-light imagery to an initial prediction interval. These model-based intervals adapt to the input location but have no finite-sample coverage guarantee before calibration. (B) Using a held-out, labeled calibration set, CT measures one-sided errors relative to the policy threshold β separately for neighborhoods with Yi≤β and for neighborhoods with Yi>β . (C) From each set of errors, CT selects the conformal quantile associated with the target coverage rate α and uses it to adjust the corresponding interval bound, producing a calibrated thresholding interval for a new neighborhood. (D) If the calibrated interval lies entirely above β , the neighborhood is classified as “Above”; if it lies entirely below β , it is classified as “Below”; and if it overlaps β , CT abstains and returns “Indeterminate.” Under exchangeability, both the probability of misclassifying a neighborhood with Y≤β as “Above” and the probability of misclassifying a neighborhood with Y>β as “Below” are at most 1−α .
Figure 2: Strong aggregate predictive performance coexists with substantial local uncertainty. The main panel shows the distribution of 90% CQR prediction-interval widths across held-out DHS neighborhoods. Although the inset shows strong aggregate agreement between observed IWI and the median predictions ( R2=0.753 ), matching reported values in prior works, the median interval is still 28.7 IWI points wide (orange). When centered on the mean observed IWI, an interval of this width contains 39.7% of the observations in the full dataset, illustrating how limited the model’s local precision remains despite its strong aggregate performance.
Figure 3: Neighborhood-level maps of predicted asset wealth (left) and uncertainty (right), measured as the width of the 90% prediction interval given by conformal quantile regression. Predicted wealth is highest in North Africa, along the Nile valley, in South Africa, and around cities and transport corridors, while most of the sub-Saharan interior shows low predicted IWI. There are clear spatial patterns for uncertainty as well, but they are not clearly correlated with wealth. For example, most of the relatively wealthy North Africa have wide prediction intervals, while Egypt does not.
Error Rate “Below”
Error Rate “Above”
Decision Coverage
Accuracy
Error Rate
Point prediction
0.201
0.133
1.000
0.843
0.157
Naive intervals
0.027
0.004
0.465
0.453
0.012
CQR intervals
0.009
0.000
0.340
0.337
0.003
CT (ours)
0.050
0.050
0.693
0.644
0.050
Table 1: Results on using the different methods for threshold classification when the threshold β is 20 IWI and the target coverage α is 0.95. Our “Conformalized thresholding” (CT) method achieves the highest decision coverage of all methods respecting the α coverage. Red entries mark error rates that exceed the 5% targets; struck-out values are displayed for completeness but are not valid comparators, since the method attains them only by violating the error constraints. We use the total number of samples, not just the number of predictions, in the denominator when calculating Accuracy and Error Rate.
Figure 4: Aid-allocation frontiers under varying exclusion tolerance. Curves show the mean aid delivered per eligible recipient across 17 countries as the maximum permitted share of eligible neighborhoods left without aid varies from 0 to 0.30. Shaded regions span the lowest- and highest-performing countries. SAFE achieves the highest return among the implementable strategies throughout and approaches the unattainable “Oracle” benchmark as the permitted exclusion rate increases.
Figure 5: Spatial allocation decisions produced by SAFE. SAFE is applied to populated raster cells in countries with DHS surveys completed after 2020, using the most recent national survey as the calibration set. Bar height is proportional to cell population, while color indicates whether aid is given directly, withheld, or preceded by a follow-up survey. In general, aid tends to be allocated directly to poorer rural areas, withheld from dense urban centers, and directed through surveys in more ambiguous areas, including many smaller cities. These patterns vary locally: panel D , for example, shows sparsely populated rural cells in the relatively wealthier South-East Nigeria being assigned surveys rather than direct aid.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Figure 1 : Architecture of the EO-ML model. Landsat frames are encoded by a spatial transformer, combined with nighttime-light images and conditioning tokens in a temporal transformer, and mapped to a scalar IWI output.
Supplementary Figure 2 : QQ-plot of the model predicted against the observed quantiles before and after conformal prediction calibration. The naive prediction intervals capture much of the uncertainty but undercover: the nominal 90% interval contains the observed outcome for 79.5% of held-out neighborhoods. After conformal calibration, coverage matches the target (90.0%), in line with the theoretical guarantees.
Country
DBScan
First Kmeans
Second Kmeans
Shannon based
Angola
7.21
7.21
7.21
7.21
Burkina Faso
6.62
6.62
6.62
6.62
Benin
3.00
4.33
4.33
4.33
Burundi
0.01
1.39
2.57
3.51
Congo - Kinshasa
8.39
8.41
8.43
8.43
Central African Republic
5.03
5.03
5.03
5.03
Appendix
Supplementary Table 1 : Shannon entropy of clusters per country
Supplementary Figure 3 : Out of area clusters in Burundi. (a) Result from DBScan. (b) Applying the first round of KMeans. (c) Applying the Second round of KMeans. (d) Applying KMeans to countries with low Shannon entropy
Fold
Countries
A
Cameroon, Chad, Democratic Republic of the Congo, Egypt, Eswatini, Guinea, Mauritania
Burundi, Gambia, Ghana, Madagascar, Morocco, Sierra Leone, Tanzania, Zimbabwe
E
Benin, Burkina Faso, Central African Republic, Lesotho, Malawi, Mozambique, Senegal, South Africa
Appendix
Supplementary Table 2 : Countries assigned to each cross-validation fold during out-of-country training.
Paper
Model type
R2
Covariates
Pettersson et al. [26]
CNN + LSTM
0.76
Landsat + NL
Marty and Duhaut [25]
XGBoost
0.75
EO + other sources
Wang and Li [34]
RF
0.70
NL
Zheng et al. [36]
SwinV2-T
0.69
Landsat
Yeh et al. [35]
CNN
0.67
Landsat + NL
Sherman et al. [29]
MOSAIKS
0.67
MOSAIKS
Appendix
Supplementary Table 3 : Previous works Performance reported in earlier works trained to predict asset wealth from DHS data. Direct comparisons between different works remains difficult, but nevertheless, our proposed EO-SQR model appears competitive with reported figures.
Poverty statistics guide social policy, but in many low- and middle-income countries, censuses and household surveys that collect these data are costly, infrequent, quickly outdated, and sometimes error-prone. Satellite imagery offers global coverage and the possibility of predicting economic livelihoods at scale, yet existing approaches to predicting livelihoods with imagery or other non-traditional data often fail to reliably identify local-level variation and, as we show, degrade under temporal shift. Here we introduce Tempov, a satellite foundation model pretrained by self-supervision on three million bi-temporal Landsat pairs and adapted with parameter-efficient fine-tuning to sparse survey labels. The model enables large-scale, high-resolution wealth mapping and dynamic measurement, including zero-shot nowcasting up to a decade after observed labels, retrospective hindcasting, and decadal change tracking, while outperforming existing neural network and geospatial foundation-model baselines. In low-label regimes, Tempov achieves competitive accuracy with only 10% of survey samples, indicating substantially reduced dependence on expensive label collection. The model further generalizes across populous countries within and outside Africa, and scales to a unified Africa-wide model with strong continent-level performance (R2=0.63, r2=0.68), from which we generate high-resolution decadal maps of wealth and wealth changes for the African continent. Analysis of these maps shows large variation in recent economic performance both within and across countries. Our open-source approach provides a pathway to timely, scalable, low-cost monitoring of wealth and poverty from routinely collected satellite data.
Zhuo Zheng, Iván Higuera-Mendieta, Richard Lee +5
Department of Computer Science, Stanford University. · Department of Earth System Science, Stanford University. · Center on Food Security and the Environment, Stanford University. +2
Accurate poverty mapping using satellite imagery is often hindered by (i) noisy and sparse survey-derived supervision, (ii) image quality issues such as cloud cover and image corruption, and (iii) lack of explicit spatial structure in image-only models. Building on the KidSat framework, we develop an enhanced pipeline that improves predictive accuracy via refined data preprocessing, systematic image quality assessment, and mathematically defined geographic encoding. First, we refine the fine-tuning target matrix by resolving high-cardinality sparsity and reducing one-hot dimensionality from 103 to 51 via DHS re-aggregation. Second, we introduce a simple two-stage quality-screening procedure to filter heavily clouded or corrupted observations. Third, we fuse DINOv2 visual embeddings with Spherical Harmonics (SH) location features. Across extensive experiments, these changes reduce MAE from 0.2167 to 0.1759, corresponding to an 18.83% relative reduction on the cluster-level severe-deprivation proportion scale. When extended from 16 to 33 African countries, the best-performing configuration achieves an overall MAE of 0.1658. We find that SH features consistently improve performance over the image-only backbone, whereas higher-capacity coordinate Multi Layer Perception augmentation (SH+SIREN) can underperform without carefully designed objectives. Finally, gradient-boosted tree heads (XGBoost/LightGBM) most effectively exploit nonlinear interactions in the fused visual-geographic representation. These findings provide a scalable and principled recipe for improving satellite-based socioeconomic predictions using only publicly accessible data.
Hou Hin Ip, Ka Nam Lam, Joshua Man Yu Ng +4
School of Mathematics University of Bristol · Department of Computer Science University of Oxford
Recent years have seen a rapid expansion in the production of large-scale geospatial maps derived from Earth observation (EO) data, driven largely by advances in machine learning (ML) and large computing infrastructure. Although the barrier to generating such maps has dropped substantially, established best practices have yet to emerge, and design decisions made early in the pipeline can quietly propagate errors into the final product. Producing a technically sound and scientifically credible product remains challenging. Choices made at every stage are tightly coupled: preprocessing decisions shape the training signal, dataset design governs what the model can learn and how reliably its performance can be assessed, and global-scale inference introduces engineering challenges in compute and data access at scale, as well as artifact mitigation. Furthermore, uncertainty quantification and independent map validation each require dedicated methodological attention that is often underestimated. This paper presents a concise, end-to-end account of the recommended practices spanning the pipeline from satellite data to an operational map product. We organize the discussion around six interconnected themes: the EO data infrastructure landscape, data selection and preprocessing, ML dataset construction and model training, uncertainty quantification, map production and distribution, and validation. This paper is a condensed version of a longer guide that provides greater depth across all stages, accessible online at ghjuliasialelli.github.io/ML-EO-Maps/.
Ghjulia Sialelli, Robin Young, Yuchang Jiang +9
Photogrammetry and Remote Sensing, ETH Zurich, Switzerland · ETH AI Center, Zurich, Switzerland · Dept. of Computer Science and Technology, University of Cambridge, UK +6