Sliding-window classifiers are often evaluated on thousands of overlapping test windows, even though neighboring predictions share observations and remain nested within recordings and subjects. Subject-disjoint evaluation prevents one form of leakage but does not make those test windows independent. We present a practical audit that maps three claims - performance on observed recordings, future recordings from observed subjects, and unseen subjects - to explicit aggregation rules and established dependence-robust inference. At 75% overlap, controlled simulations give 16.9% Type-I error for IID observed-record inference and 7.2% for session-centered Bartlett-HAC: a substantial improvement with residual miscalibration. Audits of frozen WISDM and HARTH predictions show that nearly fourfold growth in test rows yields only 1.75-1.94-fold variance-equivalent information growth. At that overlap, fixed-record paired Accuracy-difference intervals are 1.22-1.66 times the IID widths; this inflation is not universal at zero overlap. On HARTH, paired Accuracy-difference intervals include zero across three overlap settings, whereas Macro-F1 favors MiniROCKET. Independent recomputation, common-session checks, class-level results, and separately seeded calibration make the audit's scope and limitations inspectable. The resulting workflow distinguishes additional predictions from additional independent evidence.
Figures & tables
Fig. 1: Why nominal window count is not independent test count. Overlapping windows reuse the same multichannel samples; models are compared on matched windows, producing a temporally dependent paired contrast sequence.
Work
Demonstrated scope and relation to this audit
Binette–Reiter [ 7 ]
Defines targets, aggregation, uncertainty, and reporting; a sliding-window three-target implementation is not shown.
Nested-data metric intervals, F1 resampling, effective sample size and design degrees of freedom; sliding-window paired contrasts are not shown.
This audit
Maps temporal, session, and subject claims to weights and inference; checks paired windows, overlap eligibility, and relative information growth.
TABLE I: Capabilities demonstrated in the closest frameworks. “Not shown” describes the cited paper, not an impossibility.
Fig. 2: Estimand-aware audit workflow. The intended generalization target determines the highest independent unit and therefore the matched inference method. The report exposes confidence intervals, CI width ratios, relative information growth, and conclusion disagreement.
Field
Required interpretation
Target
Fixed records, future sessions, or future subjects
Metric
Accuracy, Macro-F1, and the aggregation rule
Hierarchy
Subject/session identifiers and the highest independent unit
Window geometry
Length, stride, overlap, purity, and boundary policy
Estimate
Paired point difference on identical eligible windows
Uncertainty
Method, bandwidth/block rule, confidence level, and unit count
TABLE II: Minimum fields in a dependence-aware audit report.
Dataset
Window
Subj.
Rate
Test overlap
Used sessions
WISDM
5 s
51
20 Hz
0/75%
933/933
WISDM
10 s
51
20 Hz
0/50/75%
933/933/933
HARTH
10 s
22
50 Hz
0/50/75%
393/403/409
TABLE III: Real-data configurations after the frozen boundary and eligibility checks.
Fig. 3: Monte Carlo null calibration with 95% Monte Carlo intervals. For the fixed-record target, the formal session-centered Bartlett-HAC procedure remains close to nominal at 0–50% overlap and substantially reduces, but does not eliminate, anti-conservatism at 75% overlap. For the new-subject target, window-level IID and within-session HAC inference remain severely target-misaligned, whereas equal-subject paired inference uses the matched sampling unit.
Fig. 4: Nominal test-window growth versus variance-equivalent information growth from 0% to 75% overlap. Information growth is the inverse session-centered Bartlett-HAC variance ratio relative to the 0% setting; HARTH here uses each setting’s eligible sessions.
Setting
Nom./Info.
IID/HAC width
HAC/IID
WISDM 5 s
3.97/1.94 ×
.510/.668
1.31 ×
WISDM 10 s
3.93/1.87 ×
.743/.906
1.22 ×
HARTH 10 s
3.95/1.75 ×
.520/.862
1.66 ×
TABLE IV: Fixed-record paired Accuracy-difference uncertainty at 75% overlap. Widths are percentage points.
Fig. 5: Absolute 95% confidence-interval widths for fixed-record paired Accuracy differences under IID-Normal and formal session-centered Bartlett-HAC inference. Table IV reports HAC/IID ratios.
Overlap
0%
50%
75%
Candidate windows
12,250
24,271
48,296
Retained windows
10,610
21,056
41,938
Retained (%)
86.61
86.75
86.84
Sessions / subjects
393/22
403/22
409/22
Mean purity
.990389
.990589
.990705
Fully pure (%)
91.31
91.63
91.69
TABLE V: HARTH overlap audit. Purity summaries weight retained windows equally. Difference point estimates are percentage points; complete CIs accompany the audit.
Dataset
Metric
Fixed records
New sessions
New subjects
WISDM
Accuracy
-9.65 [-10.10, -9.20]
-9.75 [-12.31, -7.18]
-9.38 [-11.81, -6.95]
WISDM
Macro-F1
-8.39 [-8.85, -7.93]
-8.53 [-11.05, -6.08]
-8.81 [-11.08, -6.53]
HARTH
Accuracy
+0.07 [-0.36, +0.50]
-2.25 [-5.87, +1.37]
+0.07 [-4.55, +4.69]
HARTH
Macro-F1
-13.21 [-16.03, -10.39]
-18.03 [-23.60, -10.21]
-13.49 [-18.65, -8.33]
TABLE VI: Cross-dataset RF − MiniROCKET differences in percentage points with 95% confidence intervals (10-s windows, 75% overlap). Negative values favor MiniROCKET. Each column uses target-matched inference.
Fig. 6: HARTH-only RF − MiniROCKET differences with 95% intervals (10-s windows, 75% overlap) from the same frozen predictions. Accuracy and Macro-F1 support different conclusions; negative values favor MiniROCKET. Table VI reports complete cross-dataset results.
Defense Innovation Institute, Academy of Military Science, Beijing, China · School of Software Engineering, South China University of Technology, Guangzhou, China