Explainable Machine Learning for Broadband Adoption Disparities: Tract-Level Prediction and SHAP-Based Factor Profiling

Xiao Han

cs.LGAug 29, 2026

Explainable Machine Learning for Broadband Adoption Disparities: Tract-Level Prediction and SHAP-Based Factor Profiling

Authors: Xiao Han

Organizations: Independent Researcher San Jose, CA, USA

Abstract

The United States has allocated approximately $65 billion through the Infrastructure Investment and Jobs Act for broadband expansion, yet evidence-based methods for targeting these investments remain underdeveloped. This paper presents an explainable machine learning framework for profiling broadband adoption disparities at census-tract granularity across 83,359 tracts nationwide. Using 65 socioeconomic, demographic, and infrastructure features derived from the American Community Survey 2022, we train a LightGBM model under spatial five-fold cross-validation, achieving R^2 = 0.533 and Spearman rho = 0.763; state-held-out cross-validation (51 folds) confirms generalization (R^2 = 0.525). TreeSHAP analysis identifies income and education as the dominant factor group (with the engineered interaction term absorbing attribution from its constituent features), and SHAP-based clustering reveals three exploratory factor profiles: Well-Connected Moderate (~49K tracts), Affordability-Limited Severe (~21K tracts), and Rural-Elderly (~13K tracts). As a screening tool, ML-based tract selection captures 38.0% of the total adoption gap within the top 10% of tracts versus 35.2% for income-only heuristics (+2.8 pp, p < 0.002, county-block bootstrap); in regret-reduction terms, the model closes 19% of the remaining gap between income-only and oracle selection. The primary contribution is the per-tract factor decomposition: SHAP identifies which feature groups (income/education, rurality, age) are most strongly associated with each tract's predicted gap, and informs differentiated investigation. A temporal stability check, training on ACS 2017 and predicting ACS 2022 with zero survey-year overlap, confirms ranking stability (rho = 0.784, noting hyperparameters tuned on 2022 data).

Figures & tables

Model	RMSE	MAE	$R^{2}$	Spearman $\rho$
LightGBM	0.069	0.049	0.533	0.763
XGBoost	0.069	0.049	0.530	0.762
CatBoost	0.069	0.049	0.529	0.762
Random Forest	0.071	0.050	0.515	0.759
Ridge	0.079	0.053	0.380	0.733
Lasso	0.090	0.060	0.153	0.696

TABLE I: Model Performance Under Spatial 5-Fold Cross-Validation

Rank	Feature	Mean $\|\text{SHAP}\|$
1	income_x_education	0.0230
2	log_median_income	0.0182
3	pct_less_than_hs	0.0110
4	pct_hs_diploma	0.0068
5	gini_index	0.0060
6	elderly_x_renter	0.0056

TABLE II: Top 15 Features by Mean Absolute SHAP Value

Fig. 1: SHAP beeswarm plot. Each dot is one tract; color indicates feature value (red = high, blue = low). Income and education features collectively form the dominant factor group.

Fig. 2: SHAP dependence plots for top features. Non-linear thresholds are visible, particularly for income (sharp increase below $35K) and education (acceleration above 20% without high school diploma).

Feature 1	Feature 2	Mean $\|\text{Int.}\|$
income_x_education	log_median_income	0.0044
labor_force_part.	log_median_income	0.0017
income_x_education	labor_force_part.	0.0012
log_median_income	pct_less_than_hs	0.0008
income_x_education	pct_less_than_hs	0.0007
log_median_income	pct_hs_diploma	0.0006

TABLE III: Top 10 SHAP Feature Interaction Pairs

Metric	Moderate	Severe	Rural–Elderly
$n$ tracts	49,417	21,116	12,826
Mean adoption gap	$-0.034$	0.095	0.106
Poverty rate	8.5%	25.2%	13.8%
% $<$ High school	7.4%	20.0%	12.3%
% Bachelor’s+	43.3%	17.2%	19.4%
% Age 65+	17.4%	14.8%	21.1%

TABLE IV: Exploratory Factor Profile Characteristics

Fig. 3: Geographic distribution of exploratory factor profiles. Affordability-Limited Severe tracts concentrate in the Deep South; Rural–Elderly tracts dominate Appalachia and the rural Mountain West.

Fig. 4: Geographic distribution of predicted broadband adoption gaps (out-of-fold predictions). Strong spatial clustering is visible in the Deep South, Appalachia, and rural Mountain West.

MP $-$ Income: $+2.8$ pp [2.3, 3.7], $p<0.002$
Strategy	Gap Covered	% Total
Gap-Ranked (oracle)	2,057.8	49.8%
Model-Predicted	1,571.0	38.0%
Income-Only	1,455.2	35.2%
Random	836.5	20.2%

TABLE V: Policy Screening Comparison (top 8,335 tracts = 10%). County-block bootstrap (500 resamples, 3,114 counties).

Fig. 5: Policy screening comparison. ML-based selection captures 38.0% of the national adoption gap within the top 10% of tracts versus 35.2% for income-only heuristics.

Fig. 6: Actual versus predicted adoption gaps (out-of-fold predictions, $n=83{,}359$ ). Pooled $R^{2}=0.568$ , $\rho=0.774$ . Pooled values exceed the fold-mean ( $R^{2}=0.533$ ) due to between-fold mean differences.

Metric	Within-vintage	Temporal
Spearman $\rho$	0.763	0.784
$R^{2}$ (raw)	0.533	0.187
$R^{2}$ (recalibrated)	—	0.598
Screening gain (pp)	+2.8	+3.2

TABLE VI: Temporal Stability Check: ACS 2017

\to

2022

Interpretable Factor Decomposition for Decision Intelligence in Large-Scale Financial Markets: Evidence from China's A-Share Market
Jun 11, 2026Xiao Han, Yao Xiao, Zhen Zhang +1Quantitative Finance Interpretable ML
Goizueta Business School Emory University Atlanta, GA, USA · College of Computing Georgia Institute of Technology Atlanta, GA, USA · School of Data Science University of Pennsylvania Philadelphia, PA, USA +1
Evaluating Explanation Methods by the Predictors They Induce
Sep 17, 2026Jacob Selbæk, Hugo L. HammerFeature Attribution SHAP Feature Attribution
Department of Computer Science, Oslo Metropolitan University, Oslo, Norway · Department of Holistic Systems, SimulaMet, Oslo, Norway · Department of Plastic and Reconstructive Surgery, Oslo University Hospital, Oslo, Norway
RoSHAP: A Distributional Framework and Robust Metric for Stable Feature Attribution
May 14, 2026Lanxin Xiang, Liang Shi, Youhui Ye +3Feature Attribution Shapley Value Attribution
Department of Statistics Virginia Tech · Department of Statistics Virginia Tech Transportation Institute · Department of Computer Science Virginia Tech

Jun 11, 2026J/K move · Enter open · S save

Interpretable Factor Decomposition for Decision Intelligence in Large-Scale Financial Markets: Evidence from China's A-Share Market

Xiao Han, Yao Xiao, Zhen Zhang, Moxuan Zheng

Goizueta Business School Emory University Atlanta, GA, USA · College of Computing Georgia Institute of Technology Atlanta, GA, USA · School of Data Science University of Pennsylvania Philadelphia, PA, USA+1

Quantitative Finance Interpretable ML Finance Financial Decision-Making

Open paper AskPDF arXiv

We present an interpretable machine learning pipeline to decompose Cross-Sectional Equity Return Predictability into auditable factor contribution. We apply an XGBoost model with TreeSHAP attribution and conduct stress testing on 3632 Chinese A-share stocks from 2009 until 2019. Using 60-month, rolling windows over 55 months of out-of-sample data, XGBoost obtains a mean AUC of 0.547 and +2.38%/month (Newey-West t = 5.94; Annualized Sharpe 2.23) long-short spread for the top vs bottom quintiles. This alpha is persistent after adjusting for the Carhart four-factor model (+2.31%/month; t = 7.48). SHAP Decomposition indicates that behavioral signals (turnover and momentum) account for 58.2% of predictive attribution compared to 10.7% for valuation ratios, on average, across 55 industry groups. Ablation analysis serves to cross-validate this ranking and provides evidence that SHAP and ablation diverge in a manner that highlights feature substitutability structure that is largely invisible to either method used in isolation.

Abstract

Explore similar work

Interpretable Factor Decomposition for Decision Intelligence in Large-Scale Financial Markets: Evidence from China's A-Share Market

Evaluating Explanation Methods by the Predictors They Induce

RoSHAP: A Distributional Framework and Robust Metric for Stable Feature Attribution