We consider nonparametric regression when the association between a response and its covariates changes across an unknown partition of a spatial domain. The proposed estimator learns the partition and the cluster-specific regression functions jointly. A neural network depending only on location determines cluster membership, while separate neural networks describe the covariate--response relationship within the clusters. An annealed softmax relaxation permits gradient-based estimation of the otherwise discrete assignments. Graph-Laplacian and occupancy penalties are used to discourage fragmented regions and degenerate solutions. We establish identifiability up to label permutation, bound partition error under a margin condition, and decompose prediction risk into regression and assignment components. The resulting rate agrees with that of an oracle estimator when the partition is estimated sufficiently accurately. Simulations show that joint estimation is useful when regression surfaces change abruptly across spatial boundaries, including settings with nonlinear effects, unequal region sizes, preferential sampling, and spatially correlated errors. Finally, a real data analysis is provided to demonstrate the validity and effectiveness of the proposed method.
Figures & tables
Scenario
Method
n=300
n=600
n=1200
Balanced, strong
Global DNN
1.203 (0.012)
0.775 (0.006)
0.531 (0.005)
Balanced, strong
GWR
0.905 (0.006)
0.827 (0.005)
0.767 (0.002)
Balanced, strong
K -means + DNN
1.937 (0.083)
1.585 (0.065)
1.236 (0.052)
Balanced, strong
Proposed
0.886 (0.034)
0.425 (0.008)
0.276 (0.009)
Balanced, strong
Oracle DNN
0.490 (0.014)
0.182 (0.005)
0.084 (0.003)
Unbalanced, strong
Global DNN
2.021 (0.022)
1.217 (0.014)
0.776 (0.008)
Table 1: Experiment 1: integrated squared error by two-region scenario and sample size. Entries are Monte Carlo means over 100 replications, with Monte Carlo standard errors in parentheses.
Scenario
Method
Misclassification
ARI
Balanced, strong
K -means + DNN
0.285 (0.016)
0.290 (0.032)
Balanced, strong
Proposed
0.034 (0.001)
0.870 (0.005)
Unbalanced, strong
K -means + DNN
0.351 (0.005)
0.098 (0.006)
Unbalanced, strong
Proposed
0.036 (0.004)
0.866 (0.013)
Balanced, weak
K -means + DNN
0.219 (0.016)
0.415 (0.031)
Balanced, weak
Proposed
0.045 (0.001)
0.827 (0.004)
Table 2: Experiment 1: recovery of the two-region spatial partition at n=1200 . Entries are Monte Carlo means over 100 replications, with Monte Carlo standard errors in parentheses.
Figure 1: Experiment 1: true and estimated conditional-mean surfaces in prespecified representative data sets with n=1200 . The first row is the balanced strong-separation setting and the second is the unbalanced strong-separation setting. From left to right, the panels show the true mean, the proposed estimator, the global DNN, GWR, and spatial K -means followed by separate DNNs. The nonspatial covariates are fixed at x⋆=(0.5,0.5,0.5,0.5,0.5)⊤ , and all panels use the same color scale. Data sets were generated using fixed seeds before model fitting and were not selected from the Monte Carlo replications.
Figure 2: Experiment 1: distribution over 100 replications of label-permutation-adjusted test misclassification rates at n=1200 , shown separately for the balanced strong, unbalanced strong, and balanced weak settings.
Figure 3: Experiment 1: true and proposed-method partitions in a representative balanced strong-separation data set with n=800 . Cluster labels have been permuted to match the true labels.
Scenario
Method
Misclassification
ARI
L-IID
K -means + DNN
0.385 (0.003)
0.276 (0.007)
Proposed
0.027 (0.002)
0.922 (0.004)
NL-IID
K -means + DNN
0.385 (0.003)
0.277 (0.007)
Proposed
0.026 (0.001)
0.923 (0.004)
NL-SP
K -means + DNN
0.384 (0.005)
0.292 (0.009)
Proposed
0.049 (0.006)
0.870 (0.009)
Table 4: Experiment 2: recovery of the three-region partition at n=1200 . Entries are Monte Carlo means over 100 replications, with Monte Carlo standard errors in parentheses.
Figure 4: Experiment 2: true and estimated conditional-mean surfaces in prespecified representative data sets with n=1200 . The first row is NL-IID; the second is NL-SP, with preferential training locations and spatially correlated errors. From left to right, the panels show the true mean, the proposed estimator, the global DNN, GWR, and spatial K -means followed by separate DNNs. The nonspatial covariates are fixed at x⋆=(0.5,−0.5,0.5,−0.5,0.5)⊤ , and all panels use the same color scale. Fixed seeds were specified before fitting; the displayed data sets were not selected from the Monte Carlo replications.
Figure 5: Experiment 2: true and estimated partitions in the prespecified representative NL-SP data set, together with the preferential training locations. The panels identify the displayed method, cluster labels are permutation adjusted, and the data set was generated with a fixed seed rather than selected from the Monte Carlo replications.
Scenario
n
K=2
K=3
K=4
K=5
Selected-model ISE
NL-IID
600
2
71
21
6
0.680 (0.022)
1200
0
52
35
13
0.369 (0.011)
NL-SP
600
5
51
30
14
0.863 (0.029)
1200
0
29
44
27
0.546 (0.019)
Table 5: Experiment 2: selection of K by three-fold cross-validation with the one-standard-error rule. Selection columns give percentages over 100 replications in the NL-IID and NL-SP settings.
Figure 6: Experiment 2: empirical distribution of the number of regions selected by three-fold cross-validation with the one-standard-error rule in the NL-IID and NL-SP settings. The vertical reference indicates the true value K0=3 .
Method
Log RMSE
Log MAE
R2
Dollar MAE
MAPE
Moran’s I
Global DNN
0.189 (0.003)
0.139 (0.002)
0.864 (0.014)
78.1 (4.3)
0.142 (0.002)
0.233 (0.023)
GWR
0.196 (0.005)
0.144 (0.003)
0.854 (0.013)
81.9 (5.0)
0.148 (0.003)
0.270 (0.034)
K -means + DNN
0.237 (0.009)
0.180 (0.006)
0.786 (0.022)
99.9 (7.2)
0.184 (0.008)
0.384 (0.043)
Proposed
0.248 (0.014)
0.186 (0.011)
0.756 (0.045)
104.5 (6.3)
0.190 (0.010)
0.460 (0.053)
Table 6: Spatially blocked prediction of King County log sale price. Entries are means across five outer folds, with standard errors in parentheses. Dollar MAE is reported in thousands of dollars.
Figure 7: Observed log sale prices, the full-data three-region partition, out-of-fold predictions from the proposed method, and out-of-fold residuals. Cluster labels are arbitrary. Whole spatial blocks, rather than individual transactions, were held out during cross-validation.
Figure 8: Cluster-specific fitted sale price as a function of living area. All other property covariates are fixed at their within-region medians.
Figure 9: Average within-region derivatives of the fitted regional regressions with respect to standardized property covariates. Positive values indicate an increase in fitted log price.
Spatial prediction estimates unobserved values from spatially distributed observations. Although flexible predictive models are increasingly available, their practical use remains limited when interpretability is essential. In this study, we formulate a graph partitioning problem to construct a global surrogate model that approximates black-box predictions with simpler representations. Specifically, we select connected subgraphs to minimize within-cluster residuals in vertex-level predictions. Although this formulation can be viewed as a variant of spatial regionalization, it operates on model-based predictions at individual locations rather than predefined areal units. With many locations, this point-level formulation may produce artificial connections between regions, reducing interpretability. We therefore introduce a constraint that excludes such connections while retaining flexible partition boundaries. We theoretically analyze how this constraint affects the resulting partition. We also develop a computational procedure for adjusting partition boundaries. Experimental results show that the proposed method yields spatially coherent partitions with flexible boundaries.
Yuta Shikuri
The Graduate University for Advanced Studies, Tokyo, Japan · Tokio Marine Holdings, Inc., Tokyo, Japan
Spatial prediction refers to the estimation of unobserved values from spatially distributed observations. Although recent advances have improved the capacity to model diverse observation types, adoption in practice remains limited in industries that demand interpretability. To mitigate this gap, surrogate models that explain black-box predictors provide a promising path toward interpretable decision making. In this study, we propose a graph partitioning problem to construct spatial segments that minimize the sum of within-segment variances of individual predictions. The assignment of data points to segments can be formulated as a mixed-integer quadratic programming problem. While this formulation potentially enables the identification of exact segments, its computational complexity becomes prohibitive as the number of data points increases. Motivated by this challenge, we develop an approximation scheme that leverages the structural properties of graph partitioning. Experimental results demonstrate the computational efficiency of this approximation in identifying spatial segments.
Yuta Shikuri, Hironori Fujisawa
The Graduate University for Advanced Studies · Tokio Marine Holdings, Inc. · Institute of Statistical Mathematics +1
In this paper, we focus on fully connected deep neural networks utilizing the Rectified Linear Unit (ReLU) activation function for nonparametric estimation. We derive non-asymptotic bounds that lead to convergence rates, addressing both temporal and spatial dependence in the observed measurements. By accounting for dependencies across time and space, our models better reflect the complexities of real-world data, enhancing both predictive performance and theoretical robustness. We also tackle the curse of dimensionality by modeling the data on a manifold, exploring the intrinsic dimensionality of high-dimensional data. We broaden existing theoretical findings of temporal-spatial analysis by applying them to neural networks in more general contexts and demonstrate that our proof techniques are effective for models with short-range dependence. Our empirical simulations across various synthetic response functions underscore the superior performance of our method, outperforming established approaches in the existing literature. These findings provide valuable insights into the strong capabilities of dense neural networks (Dense NN) for temporal-spatial modeling across a broad range of function classes.
Carlos Misael Madrid Padilla, Zhi Zhang, Xiaokai Luo +2
Department of Statistics and Data Science, Washington University in St. Louis. · Department of Statistics and Data Science, University of California, Los Angeles · Department of Statistics, University of Notre Dame