Prescribed-parameter convolutional transforms achieve strong time series classification performance using fixed kernels, simple pooling operations, and a linear classifier. Their success raises a mechanistic question: which properties of the representation make this construction effective? We study the proportion of positive values (PPV), a pooling operator shared by ROCKET, MiniROCKET, and MultiROCKET. We establish exact invariance to circular translations under periodic boundaries and quantify departures from invariance under finite boundaries. A complementary analysis bounds the sensitivity of a smooth PPV surrogate. We then define the population pairwise geometry induced by a distribution of convolutional parameters and show how a finite random kernel bank concentrates around that geometry. This separates the role of the parameter distribution from the number of sampled kernels. Experiments on the UCR classification collection examine effective dimensionality, distance correlation with the input, and classification accuracy using PPV-only, unit-energy ROCKET kernels. Increasing kernel count produces diminishing representational gains. Increasing kernel length generally raises effective dimensionality and distance correlation, while the best test performance occurs at intermediate lengths and training accuracy remains close to perfect. Together, these results support an interpretation in which nonlinear pooling creates a richer but structured representation that can favour linear separation, while further representational richness need not improve generalisation.
Figures & tables
Figure 1: Geometry versus kernel count, averaged over all 112 UCR datasets. Panels show relative and absolute ED and distance correlation for standardised training PPV features. Relative values are normalised by each dataset’s maximum within this experiment. Shading denotes pointwise 95% bootstrap confidence intervals across datasets.
Figure 2: Geometry versus kernel length with 10,000 kernels, averaged over the same 109 UCR datasets with N≥27 at every length. Panels show relative and absolute ED and distance correlation for standardised training PPV features. Relative values use each dataset’s maximum within this experiment. Shading denotes pointwise 95% bootstrap confidence intervals across datasets.
Figure 3: Training and test mean ranks across the same 109 UCR datasets. The 13 lengths are ranked separately by training and test accuracy within each dataset, with average ranks for ties. Lower ranks indicate better relative performance.
Kernel length
Train rank
Test rank
Train accuracy
Test accuracy
3
7.70
9.19
0.9958
0.8298
5
7.30
7.30
0.9966
0.8424
7
7.08
6.59
0.9977
0.8445
9
6.83
5.65
0.9982
0.8481
11
6.93
6.58
0.9982
0.8456
13
6.97
6.23
0.9971
0.8448
Table 1: Mean training and test ranks and mean absolute accuracies for the 13 kernel lengths across the same 109 UCR datasets with N≥27 . Lower ranks and higher accuracies indicate better performance. Tied accuracies receive average ranks within each dataset.
We introduce ROMAN (ROuting Multiscale representAtioN), a deterministic operator for time series that maps temporal scale and coarse temporal position into an explicit channel structure while reducing sequence length. ROMAN builds an anti-aliased multiscale pyramid, extracts fixed-length windows from each scale, and stacks them as pseudochannels, yielding a compact representation on which standard convolutional classifiers can operate. In this way, ROMAN provides a simple mechanism to control the inductive bias of downstream models: it can reduce temporal invariance, make temporal pooling implicitly coarse-position-aware, and expose multiscale interactions through channel mixing, while often improving computational efficiency by shortening the processed time axis. We formally analyze the ROMAN operator and then evaluate it in two complementary ways by measuring its impact as a preprocessing step for four representative convolutional classifiers: MiniRocket, MultiRocket, a standard CNN-based classifier, and a fully convolutional network (FCN) classifier. First, we design synthetic time series classification tasks that isolate coarse position awareness, long-range correlation, multiscale interaction, and full positional invariance, showing that ROMAN behaves consistently with its intended mechanism and is most useful when class information depends on temporal structure that standard pooled convolution tends to suppress. Second, we benchmark the same models with and without ROMAN on long-sequence subsets of the UCR and UEA archives, showing that ROMAN provides a practically useful alternative representation whose effect on accuracy is task-dependent, but whose effect on efficiency is often favorable. Code is available at https://github.com/gon-uri/ROMAN
Gonzalo Uribarri
Department of Computer and Systems Sciences, Stockholm University
Learnable downsampling and spectral-domain filter parameterizations have each made convolutional networks more flexible, but their interaction has not been systematically examined. We bridge this gap by examining convolutional kernels learned in the spatial and spectral domains, together with learned downsampling ratios under a cropped resizing implementation. We find that spectral and spatial kernels respond differently to learned resolution allocation, with larger gains for the spectral kernel. We observe this consistently on three datasets (EuroSAT, CIFAR-10 and STL-10) and at two model sizes (differences in accuracy gain of +0.013 to +0.028 on held-out test sets). Part of this difference reflects the greater gain of the spectral kernel from retaining more spatial area, but the difference remains positive when total retained area is matched. This pattern is observed across datasets and model sizes, although its magnitude and dependence on the complexity budget vary with model size and the resizing implementation.
Wei Wu, Eric C. Larson
Department of Computer Science, Southern Methodist University, Dallas, TX, USA
Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolution, standard unfolding describes a local patch map rather than the convolution operator. We introduce Muon-C, an operator-aligned optimizer that represents kernel momentum as frequency-wise channel-transfer matrices, polarizes these blocks independently, and uses a critical Fourier grid to return updates exactly to the original finite kernel support. We show that the new geometry arises from combining the block partition and Fourier coordinates. The exact-polar direction is a linear minimization oracle under the critically sampled convolution norm. Its worst-case guarantee relative to the continuous convolution-operator norm is never weaker than unfolding and is strictly stronger for 3×3 kernels. On CIFAR-10 flow matching with matched applied-update RMS, Muon-C reaches 9.87 FID at 40k iterations, compared with 22.26 for unfolded Muon and 51.31 for Adam. It reaches their final quality using 0.62× and 0.64× their model FLOPs, respectively. Under equal tuning budgets, Muon-C achieves 3.42 FID. Gains persist across data scales and transfer to classification across convolutional architectures.
Jiaxin Qing, Lexin Li
University of California, Berkeley Berkeley, CA 94720-1776, USA