Divide and Predict: An Architecture for Input Space Partitioning and Enhanced Accuracy

Fenix W. Huang; Henning S. Mortveit; Christian M. Reidys

cs.LGMar 9, 2026

Divide and Predict: An Architecture for Input Space Partitioning and Enhanced Accuracy

Authors: Fenix W. Huang, Henning S. Mortveit, Christian M. Reidys

Abstract

In this article the authors develop an intrinsic measure for quantifying heterogeneity in training data for supervised learning. This measure is the variance of a random variable which factors through the influences of pairs of training points. The variance is shown to capture data heterogeneity and can thus be used to assess if a sample is a mixture of distributions. The authors prove that the data itself contains key information that supports a partitioning into blocks. Several proof of concept studies are provided that quantify the connection between variance and heterogeneity for EMNIST image data and synthetic data. The authors establish that variance is maximal for equal mixes of distributions, and detail how variance-based data purification followed by conventional training over blocks can lead to significant increases in test accuracy.

Figures & tables

Figure 1. The two-stage approach involving purification followed by conventional machine learning. Prediction is done using a classifier routing input to the appropriate sub-model.

Figure 2. Outline: three classes of data each analyzed with respect to variance, heterogeneity, test accuracy and subjected to variance-based purification. SD-2 and SD-3 denote synthetic data containing two and three distinct distributions, respectively.

Figure 3. Variance

\mathbb{V}[X]

(blue curve) and prediction accuracy (orange curve) for EMNIST data as a function of the error rate

r

applied to training data. All 10 digits were used, with each digit

0

through

9

represented by

10^{3}

images to obtain

|Z|=10^{4}

. We train an MLP model which takes all

28\times 28

gray scale pixels as features, with learning rate

10^{-3}

and batch size

64

, and use the trained model to predict on a disjoint test set of size

10^{4}

sampled from EMNIST data. The horizontal axis gives the error rate, and the left (resp. right) vertical axis gives the variance

\mathbb{V}[X]

(resp. test accuracy). Filled circles/triangles show expectations, while bars show standard deviations across the 5 replications generated for each error rate

r

. Note that the error rates for test accuracy are small and that nearly all error bars are contained inside the triangle showing the expectation.

Figure 4. The evolution of training set variance

\mathbb{V}[X]

(blue curve) and test accuracy (orange curve) over 20 purification iterations. Here a training set

T

consists of 600 EMNIST samples digits ’4’ and ’8’ (evenly distributed) with an error rate

r=0.30

. An independent test set

S

contains 600 correctly labeled EMNIST samples with the same digit distributions as

T

. Training was conducted using MRL with batch size 64 and learning rate

10^{-3}

. For reference, we trained a model on a single distribution training set

T_{0}

of size

600

, evaluated on

S

, obtaining a test accuracy of

0.97

r=0

, shown as the dashed black line. The maximal accuracy reached during purification was

0.957

which occurred when

220

data points had been removed, shown at the red dashed line.

Figure 5. Variance

\mathbb{V}[X]

and test accuracy as functions of mixed synthetic data (SD-2) with two distributions (

A_{1}

A_{2}

) and

|Z|=600

using MLR. The horizontal axis shows

|A_{2}|

with

|A_{1}|=|Z|-|A_{2}|

. The left vertical axis shows test variance (blue curve) while the right axis shows the test accuracy (orange curve).

Z

is randomly split into training and test sets in an 8:2 ratio. The test accuracy is computed by training the model on the training set and evaluating it on the test set. Filled circles/triangles give the expectations while error bars show standard deviations; for each composition

k=5

replications were generated. For the MLR training batch size was

64

and learning rate was

10^{-3}

Figure 6. Evolution of training set variance

\mathbb{V}[X]

(blue curve) and test accuracy (orange curve) over 20 purification steps using synthetic data SD-2 and MRL. Here the SD-2 training set

Z

is composed of distributions

A_{1}

and

A_{2}

with

|A_{1}|=700

and

|A_{2}|=300

; the test set

S

consists of

1000

data points following the distribution

A_{1}

. Training was conducted using MRL with batch size 64 and learning rate

0.001

. For reference, we trained a model on a set

Z_{0}

consisting of

10^{3}

data points following distribution

A_{1}

and evaluated on

S

obtaining a test accuracy is 0.94 (black dashed line). During purification, the maximal test accuracy was measured to be

0.86

occurring after removal of 190 data points (red dashed line).

Panel of Figure 7 — Figure 7. Variance $\mathbb{V}[X]$ as a function of sample composition for SD-3 where $|Z|=|A_{1}|+|A_{2}|+|A_{3}|=600$ . Variance ( $z$ -axis) is shown as a function of $(|A_{2}|,|A_{3}|)$ where $|A_{2}|$ and $|A_{3}|$ are shown on the $x$ - and $y$ -axis, respectively. Here MLR was used with the synthetic SD-3 data. On the right, filled circles represent expectations and error bars encode standard deviations. Each data point was generated using $k=5$ replications. For the MLR training the batch size was $64$ and the learning rate was $0.001$ . The variance peaks at $|A_{1}|=|A_{2}|=|A_{3}|=200$ as indicated by the red dashed line on the right.

Figure 8. The evolution of training set variance

\mathbb{V}[X]

(blue curve) and test accuracy (orange curve) over 20 purification steps using synthetic SD-3 data. Here

Z

is composed of distributions

A_{1}

A_{2}

and

A_{3}

with

|A_{1}|=700

|A_{2}|=150

and

|A_{3}|=150

. The test set consists of

10^{3}

data points following the distribution

A_{1}

. Training was conducted using MRL with batch size

64

and learning rate

10^{-3}

. For reference, we trained a model

Z_{0}

consisting of

10^{3}

data points following the distribution

A_{1}

and evaluated it on

S

, obtaining accuracy

0.94

as shown by the black dashed line. The maximal test accuracy observed during purification was

0.86

which occurred after removal of 190 data points (red dashed line).

Training Data Curation

Training data curation improves the quality, diversity, organization, and reliability of datasets used to train machine-learning models.

See all 155 papers

Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods
Jul 29, 2026Yearn Tan Yin Tze, Charles GrelloisAutomated Machine Learning
School of Computer Science, University of Sheffield, U.K.
HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixtures
Jul 2, 2026Ziyun Qiao, Yue Min, Ruining Chen +1Language Model Pretraining Training Data Curation
Wizard Quant, Beijing, China · University of Science and Technology of China, Hefei, China
Beyond the Performance Illusion: Structure-Aware Stratified Partitioning and Curriculum Distributionally Robust Optimization for Spatially Correlated Domains
Jul 2, 2026Prathamesh Patil, Arpit Jain, Aswanth KrishnanData Leakage Distributionally Robust Optimization
1QpiAI India Pvt. Ltd.

Jul 29, 2026J/K move · Enter open · S save

Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods

Yearn Tan Yin Tze, Charles Grellois

School of Computer Science, University of Sheffield, U.K.

Automated Machine Learning

Open paper AskPDF arXiv

Accurate model evaluation in machine learning depends critically on how datasets are split into training and testing subsets. Standard random splitting assumes that both partitions share the same underlying distribution, an assumption often violated in datasets with class imbalance, natural clustering, or spatial autocorrelation. This paper investigates the role of statistical similarity in train-test splitting and its consequences for AutoML model evaluation. Five established strategies are compared across fifteen UCI benchmark datasets: random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY. Similarity is assessed using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Geometry-based methods consistently produce near-zero MMD scores, introducing instability into downstream performance estimates. The proposed Optimised-Distribution method treats similarity as an explicit optimisation objective and achieves the highest mean MMD similarity, 89.0%, across all strategies evaluated.

Abstract

Training Data Curation

Explore similar work

Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods

HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixtures

Beyond the Performance Illusion: Structure-Aware Stratified Partitioning and Curriculum Distributionally Robust Optimization for Spatially Correlated Domains