EasyResearch: Honest Regression, Forecasting and Signal Classification for Researchers Who Do Not Program

Ahmad B. A. Hassanat; Ghada A. Altarawneh

cs.LGOct 18, 2026

EasyResearch: Honest Regression, Forecasting and Signal Classification for Researchers Who Do Not Program

Authors: Ahmad B. A. Hassanat, Ghada A. Altarawneh

Organizations: Faculty of Information Technology, Mutah University, Karak, Jordan · Department of Accounting, Mutah University, Karak, Jordan

Abstract

EasyClassifier showed that researchers who do not program can obtain honest classification results when the correct evaluation is the default. Most applied studies, however, predict a number, forecast a series or classify recordings, and each task has its own way of producing scores that do not hold on new data. We present EasyResearch, open-source analysis cores for regression, time-series forecasting and signal classification, and EasyResearch Desktop, a Windows application that runs these three tasks and EasyClassifier's classification without code. All four follow the same rules: everything learned from data is learned on training data only, models keep their published defaults with fixed seeds, the selected model receives a separate final score, forecasts are never evaluated on shuffled data, and every run writes a report with a Methods paragraph and the references for every method used. We measured whether the reported scores are honest by keeping part of each dataset untouched. In regression (9 public datasets, 90 runs), the common practice overstated R^2 by +2.2 points and the best comparison score by +2.5, while EasyResearch's final score was -1.0; screening predictors on all rows before cross-validation, which EasyResearch does not offer, reported R^2 = 0.27 on pure noise against -0.44 on new data. In forecasting (15 series), shuffled cross-validation over windows of past values promised errors 19% smaller than its chosen models achieved in the untouched future, while EasyResearch's final-period score was unbiased to within 1.5%; the models EasyResearch chose had 14% lower future error, and on random walks shuffled cross-validation chose a machine-learning model in 5 of 5 series, each worse in the future than repeating the last value, which EasyResearch chose 4 times. In signal classification (4 UCR datasets) one method won every comparison and neither estimate was biased.

Figures & tables

Versions	EasyResearch 0.1.2 (analysis cores); EasyResearch Desktop 0.1.2; EasyClassifier 0.8.2
Code repository	https://github.com/ahmadhassant/easyresearch (desktop application in desktop )
Archived version (DOI)	doi:10.5281/zenodo.23305077 (all versions)
License	MIT
Language and requirements	Python $\geq$ 3.10 with pandas, NumPy, scikit-learn, statsmodels, PyTorch, matplotlib; desktop: C#/.NET (WPF), Windows x64, bundled Python runtime. The benchmarks in Section 6 were run with EasyResearch 0.1.0; versions 0.1.1 and 0.1.2 changed no benchmarked analysis
Documentation	README files of both folders; protocol of the desktop worker ( desktop/docs/PROTOCOL.md )
Support	GitHub issues; [email protected]

Table 1: Code metadata.

Panel of Figure 1 — Figure 1: Architecture. The desktop application starts a separate Python worker for every job and talks to it through a versioned JSON-lines protocol; the worker runs the same cores as the command line. Each task is a module with a manifest and an adapter, so a new task (for example multichannel signals) needs no change to the others.

Task and data	Models	Selected model (final evaluation)	Comparison	Final	Reference
Classification: Iris, 149 rows	3	Logistic regression (nested CV)	96.6	94.6	33.3
Regression: diabetes, 442 rows	10	Ridge regression (nested CV)	0.492	0.491	$-0.007$
Forecasting: CO 2 , 526 months	12 + 2 rules	TCN (last 108 months)	0.365	0.368	1.283
Signals: heartbeats, 300 recordings	6	ROCKET (untouched 20%)	98.3	96.7	33.3

Table 2: One run of each example. Comparison : score of the selected model in the comparison used to choose it; Final : the score EasyResearch reports, from data not used to choose the model; Reference : the most-frequent-class rule (classification and signals), the training mean (regression) and the seasonal naive rule in the final period (forecasting). Classification and signals: balanced accuracy (%); regression:

R^{2}

; forecasting: MASE (lower is better).

Panel of Figure 2 — Figure 2: One figure from each example report, as drawn by the software. (a) Iris: confusion matrix of the nested cross-validation predictions. (b) Diabetes: predicted against actual progression on held-out rows. (c) Synthetic heartbeats: confusion matrix on the 60 untouched recordings. (d) Mauna Loa CO 2 : 12-month forecasts of the selected model from each origin in the final period, which played no part in choosing the model; consecutive forecasts alternate between blue and orange, and thin grey lines mark the origins.

Task	Reference	Largest difference
Regression (5 datasets)	linear-regression $R^{2}$ , scikit-learn pipeline	0.0000
Forecasting (6 series)	seasonal-naive MAE, recomputed from the series	0.0000
Signals (4 UCR datasets)	ROCKET accuracy, aeon RocketClassifier	0.0171

Table 3: Agreement with reference implementations (largest absolute difference over the datasets).

Dataset	Rows	Common practice	Comparison best	EasyResearch final	Truth
Diabetes progression	221	49.4	49.2	45.9	46.6
Occupational prestige (Canada)	51	81.9	81.5	76.8	73.6
New York air quality (ozone)	58	63.9	64.9	60.4	59.5
Friedman #1 (synthetic, noise sd 1)	250	81.0	81.2	81.0	82.4
Professors’ salaries (US)	196	43.0	43.6	41.0	39.8
Car highway mileage (mpg)	109	87.0	86.4	84.0	83.5

Table 4: Regression:

R^{2}

(

\times 100

) estimated on the first half and measured on the untouched second half (mean of 10 random splits). The truth is that of EasyResearch ’s selected model; the common-practice estimate is compared with its own winner’s truth when its bias is computed.

Figure 3: Bias of each estimate against the untouched data, per dataset (regression and signals: mean over random splits; forecasting: one development–future split per series). Positive values are optimistic: the estimate promised more than was achieved. Forecasting is shown on the log scale of the ratio of achieved to estimated MASE, so

+1

means the achieved error was twice the estimate.

		Shuffled CV		EasyResearch		Rule
Series	$H$	estimate (model)	truth	final (model)	truth	truth
Mauna Loa CO2 (monthly)	12	0.361 (KNN)	0.444	0.447 (Ridge)	0.371	1.204
Airline passengers (monthly)	10	0.396 (Ridge)	0.527	0.462 (Ridge)	0.527	1.275
Nottingham temperature (monthly)	12	0.792 (Ridge)	0.591	0.757 (ETS)	0.609	0.877
UK gas consumption (quarterly)	4	0.927 (Ridge)	1.622	1.443 (Ridge)	1.622	1.740
Sunspot activity (yearly)	5	1.075 (RF)	1.069	1.022 (RF)	1.069	3.432

Table 5: Forecasting: MASE estimated on the development part (first 70%) and achieved in the untouched future (last 30%) by the model each procedure selected; lower is better.

H

is the horizon. The last column is the naive or seasonal naive rule in the future.

Dataset	Recordings	Comparison best	EasyResearch final	Truth
GunPoint (motion)	100	100.0	100.0	99.7
ItalyPowerDemand (electricity)	548	97.4	97.4	97.0
ArrowHead (shape outlines)	105	93.4	93.4	95.1
OSULeaf (leaf outlines, 6 classes)	221	94.0	94.0	93.0
Random labels (control)	100	46.9	42.7	52.9

Table 6: Signal classification: balanced accuracy (%) estimated on the first half and measured on the untouched second half (mean of 3 random stratified splits), default model set.

Automated Evaluation

Automated evaluation develops metrics, benchmarks, and AI judges to assess model outputs and system behavior at scale.

See all 398 papers

Oct 3, 2026

EasyClassifier: Honest, Reproducible Machine-Learning Classification for Researchers Who Do Not Program

Machine-learning classification is now used across medicine, the social sciences, economics, education and engineering, very often by researchers who do not program. Two errors recur in such work: preprocessing learned on all rows before cross-validation (data leakage), and reporting the cross-validation score of the classifier that won a comparison (selection bias). Both make published scores optimistic. We present EasyClassifier, an open-source Python package that guides a user from a CSV or Excel file to a finished analysis through a sequence of plain-language, multiple-choice questions, with no code. It makes the correct procedure the default rather than an option: every preprocessing step is learned inside each training fold; the selected classifier receives a separate final score from nested cross-validation (up to 2,000 rows), or an untouched 20% test set; classifiers run with fixed default parameters and fixed random seeds; and every run writes a report with a ready-to-adapt Methods paragraph, the references to cite, and figures prepared for publication. On ten public datasets from seven fields and a random-label control, split in half so that one half served as an untouched external test, the common practice overstated balanced accuracy by 2.9 percentage points on average (up to 9.9 on small data), whereas the score EasyClassifier reports had a mean signed bias of +0.7 points and a mean absolute error of 2.3 points (one-sided Wilcoxon p = 3.2 x 10^{-4}, 30 runs). On random labels it reported 48.8% balanced accuracy against a chance level of 50%. EasyClassifier is available under the MIT license from PyPI (pip install easyclassifier), GitHub, and Zenodo.

Ahmad B. A. Hassanat, Ghada A. Altarawneh

Faculty of Information Technology, Mutah University, Karak, Jordan · Department of Accounting, Mutah University, Karak, Jordan

ML Reproducibility Classification Data Leakage Tabular ML Cross-Validation Selection Bias

Ask PDF

Mar 16, 2026

Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains

We argue that the current practice of evaluating AI/ML time-series forecasting models, predominantly on benchmarks characterized by strong, persistent periodicities and seasonalities, obscures real progress by overlooking the performance of efficient classical methods. We demonstrate that these "standard" datasets often exhibit dominant autocorrelation patterns and seasonal cycles that can be effectively captured by simpler linear or statistical models, rendering complex deep learning architectures frequently no more performant than their classical counterparts for these specific data characteristics, and raising questions as to whether any marginal improvements justify the significant increase in computational overhead and model complexity. We call on the community to (I) retire or substantially augment current benchmarks with datasets exhibiting a wider spectrum of non-stationarities, such as structural breaks, time-varying volatility, and concept drift, and less predictable dynamics drawn from diverse real-world domains, and (II) require every deep learning submission to include robust classical and simple baselines, appropriately chosen for the specific characteristics of the downstream tasks' time series. By doing so, we will help ensure that reported gains reflect genuine scientific methodological advances rather than artifacts of benchmark selection favoring models adept at learning repetitive patterns.

Raeid Saqur, Christoph Bergmeir, Blanka Horvath +3

Dept. of Computer Science, University of Toronto · Dept. of Mathematics, University of Oxford · Vector Institute +5

Time Series Forecasting Time Series Modeling Forecasting Benchmarks Non-Stationary Time Series

Ask PDF

Mar 27, 2026

QuitoBench: A High-Quality Open Time Series Forecasting Benchmark

Time series forecasting is critical across finance, healthcare, and cloud computing, yet progress is constrained by a fundamental bottleneck: the scarcity of large-scale, high-quality benchmarks. To address this gap, we introduce QuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trend

\times

seasonality

\times

forecastability (TSF) regimes, designed to capture forecasting-relevant properties rather than application-defined domain labels. The benchmark is built upon Quito, a billion-scale time series corpus of application traffic from Alipay spanning nine business domains. Benchmarking 10 models from deep learning, foundation models, and statistical baselines across 232,200 evaluation instances, we report four key findings: (i) a context-length crossover where deep learning models lead at short context (

L=96

) but foundation models dominate at long context (

L \ge 576

); (ii) forecastability is the dominant difficulty driver, producing a

3.64 \times

MAE gap across regimes; (iii) deep learning models match or surpass foundation models at

59 \times

fewer parameters; and (iv) scaling the amount of training data provides substantially greater benefit than scaling model size for both model families. These findings are validated by strong cross-benchmark and cross-metric consistency. Our open-source release enables reproducible, regime-aware evaluation for time series forecasting research.

Siqiao Xue, Zhaoyang Zhu, Wei Zhang +7

Ant Group

Time Series Forecasting Time Series Modeling Forecasting Benchmarks Time-Series Foundation Models

Ask PDF

Abstract

Automated Evaluation

Explore similar work

EasyClassifier: Honest, Reproducible Machine-Learning Classification for Researchers Who Do Not Program

Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains

QuitoBench: A High-Quality Open Time Series Forecasting Benchmark