Sea-Net: A Small and Inherently Interpretable Neural Network for Time Series Classification

Ubaid Ur Rehman; Tanmoy Mondal

Figure 3: The MIL view of a time series, read from left to right. The series

X_{i}

is the bag and each time point is one instance. The encoder gives one embedding

z_{i}^{j}

per instance; the pooling head weights those embeddings and combines them into the prediction

\hat{Y}_{i}

(right). The same weighted contributions, kept instead of discarded, are the interpretation

\hat{I}_{i}

drawn on the series below, where red pushes towards the predicted class and blue pushes away from it. The two grey boxes are interchangeable, which is what lets any encoder of Appendix B be paired with any head of Appendix C .

Figure 4: One multi-scale separable block. The three depth wise convolutions read the same input at the kernel sizes

5

,

11

and

23

, their outputs are summed, and the point-wise

1\times 1

convolution

P

mixes the channels; BatchNorm, ReLU and dropout close the unit

\mathcal{U}

of Equation 1 . The dashed box is what is applied twice, and the arrow over the top is the residual connection of Equation 2 .

Figure 5: The effect of the dilation, drawn with the same five taps of the

k=5

filter in both rows. Widening the region that the filter sees costs no extra weight, which is why the receptive field of the encoder can be grown without paying for it in parameters.

Stage	Layer	Output	Params
Input	z-normalised series	$b\times 1\times t$	–
Stem	Conv1d $1\to 64$ , kernel 7	$b\times 64\times t$	512
Block 1 (dilation 1), $2\times$ unit:	depthwise Conv1d $\times 3$ (kernels 5, 11, 23), summed	$b\times 64\times t$	2,688
	pointwise Conv1d $64\to 16$ (reduce)	$b\times 16\times t$	1,040
	pointwise Conv1d $16\to 64$ (expand)	$b\times 64\times t$	1,088
	BatchNorm $+$ ReLU $+$ dropout	$b\times 64\times t$	128

Appendix

Table 3: The SEA-Net bottleneck encoder (

d=64

, 4 blocks, kernels 5/11/23, dilations 1/2/4/8), layer by layer, with the output shape and the number of parameters, built from top/top_bottleneck_topk for WebTraffic (

c=10

). The point-wise convolution of every unit is factorised as

d\to d/4\to d

, which is where the parameter saving comes from. Every layer keeps the series length

t

, so the head sees one embedding per time point. Depthwise = one filter per channel (groups

=d

).

Stage	Layer	Output	Params
Input	z-normalised series	$b\times 1\times t$	–
Stem	Conv1d $1\to 64$ , kernel 7	$b\times 64\times t$	512
Block 1 (dilation 1), $2\times$ unit:	depthwise Conv1d $\times 3$ (kernels 5, 11, 23), summed	$b\times 64\times t$	2,688
	pointwise Conv1d $64\to 64$	$b\times 64\times t$	4,160
	BatchNorm $+$ ReLU $+$ dropout	$b\times 64\times t$	128
	after both units: residual add $+$ ReLU	$b\times 64\times t$	block total 13,952

Appendix

Table 4: The SEA-Net base encoder (

d=64

, 4 blocks, kernels 5/11/23, dilations 1/2/4/8), layer by layer, with the output shape and the number of parameters, built from seanet/seanet_slim_topk for WebTraffic (

c=10

). It has the same skeleton with the plain

d\to d

point-wise convolution, for comparison. Every layer keeps the series length

t

, so the head sees one embedding per time point. Depthwise = one filter per channel (groups

=d

).

Figure 6: Conjunctive pooling, the MILLET baseline of Equation 7 . Read from left to right: the time point embeddings feed an attention head and a classifier, their product is the evidence at each time point, which is also the interpretation, and aggregating that evidence over time gives the prediction for the series. The classifier is the same in every head; only the two blue boxes, the attention head and the aggregation, change from one head to the next, so Figure 7 to Figure 13 are best read against this figure.

Figure 7: Class-wise conjunctive pooling. Against Figure 6 , only the attention head changes: widening its last layer to

C

outputs costs

(C-1)(d_{a}+1)

weights and makes the evidence, and therefore the interpretation, class-specific.

Figure 8: Soft-Max conjunctive pooling. Both blue boxes change compared with Figure 6 : the attention is normalized over time, and the aggregation is a weighted sum rather than a mean. The single extra weight is the sharpness

\tau=\exp(\rho)

.

Figure 9: Top-

k

conjunctive pooling. The attention head is the one of Figure 7 , and only the aggregation moves: the mean runs over the

m

largest evidence values of the class instead of over all

t

. It adds no weight, because

\kappa

is a hyperparameter.

Figure 10: Adaptive class-wise pooling ( Equation 16 ). The attention head is the one of Figure 7 ; the aggregation slides between the mean and the maximum according to one learned number per class.

Figure 11: Attention-max pooling ( Equation 17 ), the hard counterpart of Figure 10 : one learned number per class blends the mean with the maximum directly.

Figure 12: Per-class gated attention ( Equation 18 and Equation 19 ). The attention head has two layers instead of one, and the second of them decides which attention features are admitted.

Figure 13: Dual-stream conjunctive pooling ( Equation 20 and Equation 21 ). A second stream, built around the most supporting time point of each class, is blended into the class-wise stream of Figure 7 by one learned scalar.

#	Head	Attention	Reduction over time	Reduces at	Targets
1	Conjunctive (baseline)	shared gate	mean	—	—
2	Class-wise conjunctive	per-class gate	mean	$a_{i}^{\,j,k}=a_{i}^{j}$	L1
3	Soft-Max conjunctive	per-class score	softmax, learnable $\tau$	$\tau\to\infty$	L2
4	Top- $k$ conjunctive	per-class gate	mean of the $m$ largest	$\kappa=1$	L3
5	Adaptive class-wise	per-class gate	soft-argmax, learnable $\beta_{k}$	$\beta_{k}\to 0$	L3
6	Attention-max	per-class gate	blend of mean and max	$\lambda_{k}\to 0$	L3

Appendix

Table 5: The eight heads as two separate choices. Reduces at gives the setting at which the head becomes conjunctive pooling exactly, and Targets gives the limitations of Section C.1 that it addresses.

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times 1$ (attention scores)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Pooling	Attention weighting	$b\times t\times c$	$b\times t\times c$ (interpretation)
	Mean over $t$	$b\times t\times c$	$b\times c$ (series logits)

Appendix

Table 6: Conjunctive pooling, the MILLET baseline head (their Table A.5;

d=128

). One scalar gate per time point, shared by every class. Registered as mil_conjunctive .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times c$ (one gate per class)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Evidence	Gate $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)
Pooling	Mean over $t$	$b\times t\times c$	$b\times c$ (series logits)

Appendix

Table 7: Class-wise conjunctive pooling: conjunctive pooling with one gate per class instead of one shared gate. Registered as sea_classwise_conjunctive .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times c$ (one gate per class)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Evidence	Gate $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)
Pooling	Top- $k$ over $t$ (per class)	$b\times t\times c$	$b\times k\times c$

Appendix

Table 8: Top-

k

conjunctive pooling, the head of every short-listed model: class-wise conjunctive pooling, but the series logit averages only the

k=\lceil\kappa t\rceil

largest evidence values of each class (

\kappa=0.1

). The interpretation is the full evidence map, so it is exactly zero wherever no evidence was used. Registered as sea_topk_conjunctive .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear (raw scores)	$b\times t\times a$	$b\times t\times c$
	$\div\,\tau$ , softmax over $t$	$b\times t\times c$	$b\times t\times c$ (weights, sum to 1)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Pooling	Weight $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)

Appendix

Table 9: Soft-Max conjunctive pooling: the attention is a distribution over time (a softmax with one learned temperature

\tau

), so the series logit is a true weighted average and long and short series are on one scale. Registered as sea_softmax_conjunctive .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times c$ (one gate per class)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Evidence	Gate $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)
Pooling	softmax over $t$ of $\beta_{k}\cdot$ evidence	$b\times t\times c$	$b\times t\times c$ (weights)

Appendix

Table 10: Adaptive class-wise pooling: class-wise conjunctive pooling with a learned mean-to-max aggregator, i.e. a soft-argmax with one sharpness

\beta_{k}=\mathrm{softplus}(\theta_{k})\geq 0

per class (

c

extra parameters;

\beta\to 0

gives the mean and

\beta\to\infty

the max). Registered as sea_adaptive_classwise .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times c$ (one gate per class)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Evidence	Gate $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)
Pooling	Mean over $t$	$b\times t\times c$	$b\times c$

Appendix

Table 11: Attention-max pooling: class-wise conjunctive pooling with a learned per-class blend

\lambda_{k}=\sigma(\theta_{k})

of the mean and the max over time (

c

extra parameters). Registered as sea_attention_max .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh ( $V$ )	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid ( $U$ )	$b\times t\times d$	$b\times t\times a$
	Element-wise product	$b\times t\times a$	$b\times t\times a$ (gated features)
	Linear (scores), softmax over $t$	$b\times t\times a$	$b\times t\times c$ (weights, sum to 1)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)

Appendix

Table 12: Per-class gated attention pooling ( Ilse et al. 2018 , made class-specific): a tanh branch multiplied by a sigmoid branch scores every time point, and a softmax over time turns the scores into weights. Registered as sea_gated_attention .

Process	Layer	Input	Output
Input	Positional encoding $+$ dropout	$b\times t\times d$	$b\times t\times d$ (time-point embs.)
Attention	Linear $+$ tanh	$b\times t\times d$	$b\times t\times a$
	Linear $+$ sigmoid	$b\times t\times a$	$b\times t\times c$ (one gate per class)
Classifier	Linear	$b\times t\times d$	$b\times t\times c$ (time-point logits)
Evidence	Gate $\times$ logit (element-wise)	$b\times t\times c$	$b\times t\times c$ (interpretation)
Stream A	Mean over $t$	$b\times t\times c$	$b\times c$

Appendix

Table 13: Dual-stream conjunctive pooling (DSMIL, Li et al. 2021 , wrapped around class-wise conjunctive pooling): stream A is the plain mean, and stream B re-weights the time points by their similarity to the most supporting one (the critical time point) of each class; a learned scalar

\lambda=\sigma(\theta)

blends the two and starts near stream A. Registered as sea_dualstream_conjunctive .

Figure 14: One example of each class of the WebTraffic dataset. Each series is a daily seasonal signal with a weekly trend and noise, and the signature that defines the class is injected inside a single window, shown shaded; the exception is class

0

(None), which has no signature. These windows are the per-time-point ground truth against which NDCG@

n

is scored, and they are never used during training.

#	Dataset	Train	Test	Length	Classes	Imbalance
	WebTraffic	500	500	1,008	10	1.00
1	Adiac	390	391	176	37	3.75
2	ArrowHead	36	175	251	3	1.00
3	Beef	30	30	470	5	1.00
4	BeetleFly	20	20	512	2	1.00
5	BirdChicken	20	20	512	2	1.00

Appendix

Table 14: The datasets: WebTraffic (the only one with per-time-point labels, hence the only one with NDCG@

n

) and the 85 UCR datasets that MILLET publishes, in the order of MILLET. Imbalance is the largest training-class share divided by the smallest (1.00 = balanced). The sizes are the original train / test splits; our training holds out 20 % of the training set for validation when it has at least 100 series.

Setting	Value
Encoder size (every SEA-Net model)	width $d=64$ , 4 blocks; 41,324–69,768 parameters
Optimizer	Adam ( Kingma & Ba, 2015 ) , lr $=1.25\times 10^{-3}$ , weight decay $=1.2\times 10^{-4}$
Epochs / early stopping	max 400 epochs; stop after 60 epochs with no improvement
Batch size	$\min(\lfloor n_{\text{train}}/10\rfloor,16)$ , floor of 2
Validation split	stratified 80/20 when $n_{\text{train}}\geq 100$ ; else train-loss early stopping
Label smoothing	0.13

Appendix

Table 15: Training configuration, identical for every model of this study. The complete hyperparameter list, including the head-specific initial values, is given in Table 16 .

Parameter	SEA-Net bottleneck $+$ Top- $k$ (the paper’s model)	SEA-Net gated (mean) $+$ Top- $k$	MILLET, our recipe
Training
max epochs	400	400	400
early-stopping patience	60	60	60
batch size (max)	16	16	16
learning rate	0.00125	0.00125	0.00125
weight decay	0.00012	0.00012	0.00012

Appendix

Table 16: Every hyperparameter, read from the configuration files under configs/models/ . The two SEA-Net columns and MILLET (our recipe) share one training block, so any difference between them is the architecture. – = not applicable (the InceptionTime backbone is the code of MILLET with its published defaults, so it has no settings here).

#	Encoder $\backslash$ head	class-wise conj.	Top- $k$ conj.	softmax conj.	adaptive cw.	attn-max	gated attn.	dual-stream	conj. M	additive M	attention M
1	bottleneck (shallow)	–	21,548	–	–	–	–	–	–	–	–
2	multi-scale pyramid (bottleneck base, over channels)	–	28,441	–	–	–	–	–	–	–	–
3	bottleneck	41,324	41,324 (+0.00%)	41,325 (+0.00%)	41,334 (+0.02%)	41,334 (+0.02%)	41,844 (+1.26%)	41,845 (+1.26%)	–	–	–
4	multi-scale pyramid (bottleneck base)	–	42,365	–	–	–	–	–	–	–	–
5	multi-scale channels (bottleneck base, lean)	–	43,576	–	–	–	–	–	–	–	–
6	multi-scale channels (bottleneck base)	–	47,176	–	–	–	–	–	–	–	–

Appendix

Table 17: Number of parameters of every encoder

\times

pooling head pair that was trained, measured on WebTraffic (10 classes; the size of a head grows with the number of classes). The percentages give the change relative to the same encoder with class-wise conjunctive pooling, in the manner of Table A.13 of MILLET: every attention head costs well under 1 % of the encoder, so the size of a model is set by its encoder. The bottom rows are the baseline encoders with the conjunctive head of MILLET.

Figure 15: Head-to-head comparison of the explanations of MILLET (published recipe) and of our SEA-Net bottleneck

+

Top-

k

model on four WebTraffic classes. The panels show, left to right, the series with its true signature window, the interpretation heatmap for the true class, and the perturbation curves under each model’s own deletion order (solid) against a random one (dotted). The vertical axis is the change as a fraction of the original logit, because the two models produce logits on very different scales ( Section E.6 ). The AOPCR of each panel is the area between the two curves; MILLET wins it on three of the four classes while our heatmaps are the more concentrated — the difference between the two metrics is analysed in Section 4.1 .

#	Encoder	Pooling head	Acc.	AUROC	NLL	AOPCR	NDCG@ $n$	Params	Epochs run / best	Train (s)
1	bottleneck	Top- $k$ conjunctive (paper model, re-run)	0.942	0.996	0.294	2.951	0.786	41,324	180 / 120	54
2	bottleneck	Top- $k$ conjunctive, thresholded	0.930	0.996	0.330	2.603	0.744	41,324	172 / 112	52
3	bottleneck	softmax conjunctive, per-class $\tau$	0.914	0.991	0.377	2.076	0.732	41,334	151 / 91	65
4	bottleneck	deviation-max (MLP form)	0.912	0.990	0.369	2.394	0.742	42,544	164 / 104	276
5	bottleneck	DSMIL (batched rewrite)	0.900	0.991	0.477	1.462	0.667	71,956	379 / 319	611
6	InceptionTime	conjunctive (MILLET, our recipe)	0.890	0.985	0.531	2.120	0.692	423,707	184 / 124	443

Appendix

Table 18: The 2026-09 pooling-head round on WebTraffic (results/top_results, one seed each): every new head is trained on the bottleneck encoder (one on the gated encoder), so row 1 (the model of the paper, trained again) is the number to beat. The MILLET row is the baseline retrained with our recipe. Epochs run / best gives the epoch at which early stopping fired and the epoch whose weights were kept; the voting heads that fail to fit are visible in their NLL.

#	Encoder $\backslash$ head	class-wise conj.	Top- $k$ conj.	softmax conj.	adaptive cw.	attn-max	gated attn.	dual-stream	conj. M	additive M	attention M
1	gated (mean)	0.902	0.955	0.912	0.860	0.832	–	–	–	–	–
2	base (wide, $d=128$ )	0.946	–	0.880	0.918	–	–	–	0.954	0.942	–
3	gated (max)	0.944	0.952	0.914	0.906	0.886	–	–	–	–	–
4	spike/trend	0.904	0.950	0.878	0.934	0.854	0.930	0.556	–	–	–
5	multi-scale channels (input-gated base)	–	0.948	–	–	–	–	–	–	–	–
6	multi-scale channels (gated base)	–	0.948	–	–	–	–	–	–	–	–

Appendix

Table 19: WebTraffic accuracy for every encoder

\times

pooling head pair that was trained (71-model sweep; the

\kappa

/

\lambda

ablation copies are left out). – = pair not trained; a superscript count means that several configurations share the cell and the most accurate one is shown. M = a MILLET head. The best value of each column is in bold.

#	Encoder $\backslash$ head	class-wise conj.	Top- $k$ conj.	softmax conj.	adaptive cw.	attn-max	gated attn.	dual-stream	conj. M	additive M	attention M
1	gated (mean)	2.49 / 0.714	2.23 / 0.750	2.09 / 0.765	2.24 / 0.731	1.67 / 0.704	–	–	–	–	–
2	base (wide, $d=128$ )	1.72 / 0.686	–	1.02 / 0.605	1.61 / 0.619	–	–	–	1.50 / 0.698	1.58 / 0.729	–
3	gated (max)	1.89 / 0.592	2.27 / 0.719	1.88 / 0.776	2.07 / 0.729	1.12 / 0.575	–	–	–	–	–
4	spike/trend	2.15 / 0.699	2.32 / 0.756	1.43 / 0.665	1.91 / 0.747	1.65 / 0.644	1.59 / 0.721	1.64 / 0.510	–	–	–
5	multi-scale channels (input-gated base)	–	2.32 / 0.757	–	–	–	–	–	–	–	–
6	multi-scale channels (gated base)	–	2.22 / 0.717	–	–	–	–	–	–	–	–

Appendix

Table 20: WebTraffic AOPCR / NDCG@

n

for every encoder

\times

pooling head pair that was trained (71-model sweep; the

\kappa

/

\lambda

ablation copies are left out). – = pair not trained; a superscript count means that several configurations share the cell and the most accurate one is shown. M = a MILLET head. AOPCR is unnormalised (the same recipe is used everywhere here).

#	Encoder	Pooling head	Acc.	NLL	AOPCR	NDCG@ $n$	Params	UCR-85 acc.
Accuracy $\geq 0.94$
1	gated (mean)	Top- $k$ conj. ∗	0.955	0.254	2.225	0.750	61,740	0.8083
2	base (wide, $d=128$ )	conj. M	0.954	0.262	1.502	0.698	269,083	0.8238
3	gated (max)	Top- $k$ conj.	0.952	0.308	2.268	0.719	61,740	0.8108
4	bottleneck	Top- $k$ conj. ( $\lambda_{\text{focus}}{=}0$ )	0.950	0.263	2.303	0.765	41,324	–
5	spike/trend	Top- $k$ conj.	0.950	0.288	2.316	0.756	67,020	0.8089

Appendix

Table 21: The complete WebTraffic leaderboard of the 71-model sweep (results/old_results), ranked by accuracy and grouped into accuracy bands. ∗ marks the mean over three seeds (seeds 0–2, the runs of Table 28 ); every other row is a single run with seed 0, so a comparison between a ∗ row and an unmarked row sets a three-run mean against one run. UCR-85 acc. is the mean accuracy over the 85 UCR datasets for the models that completed the archive, and – for the models that were only screened on WebTraffic. Encoder drops the SEA-Net prefix (MS = multi-scale); wide is the

d=128

encoder; M = a MILLET head. AUROC is given in the csv file next to this table.

Figure 16: Compute cost of one prediction, in millions of floating point operations, for our SEA-Net bottleneck

+

Top-

k

model and for the MILLET baseline ( InceptionTime

+

Conjunctive ). Left: as the length of the series grows, at

10

classes. Right: as the number of classes grows, at length

463

. The dashed lines mark the mean series length and the mean number of classes over the

85

UCR datasets.

#	Pooling head	Real multiplications	Scale w.r.t. $t$	Scale w.r.t. $c$	At WebTraffic (M)
1	conjunctive M	$dtc+dta+(a+2c)\,t$	$dc+da+a+2c$	$dt+2t$	2.351 ( $d=128$ )
2	class-wise conjunctive	$dtc+dta+(ac+2c)\,t$	$dc+da+ac+2c$	$dt+at+2t$	1.262 ( $d=64$ )
3	Top- $k$ conjunctive	$dtc+dta+(ac+c)\,t+ck$	$dc+da+ac+c+c\kappa$	$dt+at+t+k$	1.253 ( $d=64$ )
4	softmax conjunctive	$dtc+dta+(ac+3c)\,t$	$dc+da+ac+3c$	$dt+at+3t$	1.272 ( $d=64$ )
5	adaptive class-wise	$dtc+dta+(ac+4c)\,t$	$dc+da+ac+4c$	$dt+at+4t$	1.282 ( $d=64$ )
6	attention-max	$dtc+dta+(ac+2c)\,t+2c$	$dc+da+ac+2c$	$dt+at+2t+2$	1.262 ( $d=64$ )

Appendix

Table 23: Time complexity of the pooling heads as the number of real multiplications, counted from the code (matrix products only, as in Table A.15 of MILLET; additions, comparisons, the exponential of a softmax and the gather of the critical time point are not counted).

t

= series length,

c

= classes,

d

= embedding size,

a

= attention width,

k=\lceil\kappa t\rceil

. The last column evaluates the formula for WebTraffic (

t=1008

,

c=10

,

a=8

,

\kappa=0.1

, so

k=101

) with the own

d

of each head: the SEA-Net heads read

d=64

embeddings and the MILLET head

d=128

, so every SEA-Net head is cheaper than the baseline head in absolute terms, although its formula has more terms. M = MILLET.

#	Pooling head	Mean acc.	Mean AOPCR	Mean NDCG@ $n$	$n$	Best pairing (acc.)
SEA-Net heads
1	class-wise conjunctive	0.921	2.322	0.692	10	0.946 ( seanet_classwise )
2	gated attention	0.918	1.869	0.733	4	0.930 ( seanet_bottleneck_gatedattn )
3	adaptive class-wise	0.912	2.210	0.732	9	0.946 ( seanet_recon_adaptive )
4	softmax conjunctive	0.907	1.690	0.712	9	0.940 ( seanet_bottleneck_softmax )
5	Top- $k$ conjunctive	0.903	2.260	0.712	16	0.955 ( seanet_gated_mean_topk )

Appendix

Table 24: Effect of the pooling head: WebTraffic accuracy, AOPCR and NDCG@

n

averaged over every encoder that each head was paired with in the 71-model sweep (

n

= number of pairings; the

\kappa

/

\lambda

ablation copies are excluded, so that no pairing is counted twice). The last column is the best single pairing of that head. A head paired once (

n=1

) is not comparable with one averaged over ten.

#	Encoder	Mean acc.	Mean AOPCR	Mean NDCG@ $n$	$n$	Best pairing (acc.)
SEA-Net encoders ( $\dagger$ = a wrapper around another encoder)
1	SEA-Net multi-scale channels †	0.936	2.385	0.747	4	0.948 ( seanet_inputgate_mschan )
2	SEA-Net reconstruction-residual	0.924	2.315	0.719	5	0.946 ( seanet_recon_adaptive )
3	SEA-Net input-gated	0.903	2.345	0.718	5	0.946 ( seanet_inputgate_topk )
4	SEA-Net gated	0.902	2.056	0.710	19	0.955 ( seanet_gated_mean_topk )
5	SEA-Net base	0.898	1.933	0.693	12	0.954 ( seanet_conjunctive )

Appendix

Table 25: Effect of the encoder: WebTraffic accuracy, AOPCR and NDCG@

n

averaged over every pooling head that each encoder was paired with (

n

= number of pairings, with the same exclusions as in Table 24 ). The gated encoder has the largest

n

because it was paired with the most heads, which pulls its mean towards the weak heads; the last column shows what each encoder reaches with its best head.

#	$\kappa$	$k$ at $t=1008$	Accuracy	NLL	AOPCR	NDCG@ $n$
1	0.05	51	0.888	0.401	2.288	0.715
2	0.10 (default)	101	0.888	0.391	2.409	0.755
3	0.25	252	0.940	0.277	2.648	0.733
4	0.50	504	0.882	0.418	3.114	0.732
5	1.00 (= class-wise conjunctive)	1008	0.908	0.395	2.436	0.722
	0.10, mean $\pm$ std over 3 runs	101	$0.931\pm 0.032$	$0.312\pm 0.063$	$2.498\pm 0.149$	$0.765\pm 0.013$

Appendix

Table 26: Top-

k

fraction ablation on WebTraffic: the bottleneck

+

Top-

k

model trained five times with only

\kappa

changed (seed 0 everywhere, so the

\kappa=0.10

row is the seed-0 run of that model, and the last row is its mean over every run).

k=\lceil\kappa t\rceil

is the number of time points that the series logit averages;

\kappa=1

recovers class-wise conjunctive pooling exactly. The best single-seed value of each column is in bold.

$\lambda_{\text{focus}}$	Accuracy	NLL	AOPCR	NDCG@ $n$
0.01 (on, the default)	0.888	0.391	2.409	0.755
0.00 (off)	0.950	0.263	2.303	0.765

Appendix

Table 27: The attention-entropy term of the loss, switched on and off, on WebTraffic (seed 0): the bottleneck

+

Top-

k

model with only

\lambda_{\text{focus}}

changed. This is one matched pair on one seed, so it indicates a direction rather than an effect size.

Figure 17: SEA-Net bottleneck

+

Top-

k

(ours). Wrong predictions on the hardest WebTraffic class, i.e. the class with the lowest accuracy. Top: the series with its true signature window in yellow. Middle: the interpretation for the class that the model predicted. Bottom: the interpretation for the true class, with its NDCG@

n

.

Figure 18: MILLET (published recipe). Wrong predictions on the hardest WebTraffic class, i.e. the class with the lowest accuracy. Top: the series with its true signature window in yellow. Middle: the interpretation for the class that the model predicted. Bottom: the interpretation for the true class, with its NDCG@

n

.

#	Model	Metric	Run 1	Run 2	Run 3	Run 4	Mean $\pm$ std
1	SEA-Net gated (mean) + Top- $k$ conjunctive	accuracy	0.954	0.958	0.952	–	$0.955\pm 0.002$
		AOPCR	2.293	2.074	2.308	–	$2.225\pm 0.107$
		NDCG@ $n$	0.772	0.749	0.728	–	$0.750\pm 0.018$
		run	archive, seed 0	archive, seed 1	archive, seed 2
2	SEA-Net bottleneck + Top- $k$ conjunctive	accuracy	0.888	0.938	0.966	–	$0.931\pm 0.032$
		AOPCR	2.409	2.708	2.378	–	$2.498\pm 0.149$

Appendix

Table 28: Every WebTraffic run of the models that were trained more than once, run by run ( archive = the 71-model sweep in results/old_results, 2026-09 = results/top_results; a seed run in both folders is two different trainings). These are the numbers behind every

\pm

in the main tables. The spread on AOPCR is far larger than on accuracy for every model, which is why AOPCR differences of a few tenths are not read as wins.

Figure 19: Model variance over the

85

UCR datasets, in the style of Early et al. (2024) . For each dataset and each seed, the balanced accuracy minus the median balanced accuracy of that model on that dataset, pooled over the datasets; the models are ordered from left to right by their total variance, which is printed under each of them. Our SEA-Net model is the least stable of the four and has a long lower tail, i.e. on a few datasets one seed falls far below the other two, which is the same effect that Table 28 shows on WebTraffic.

Model	Params	Acc.	Mean rank	W/T/L
MILLET, published ( Early et al., 2024 )	423,707	0.8445	—	—
SEA-Net wide $+$ class-wise conjunctive	269,164	0.8292	11.53	26/19/39
SEA-Net wide $+$ MILLET additive	269,083	0.8298	11.98	23/19/42
SEA-Net wide $+$ MILLET conjunctive	269,083	0.8238	12.43	22/20/42
MILLET, matched budget	423,707	0.8274 ‡	12.60	13/25/46
ResNet $+$ conjunctive	506,331	0.8146	13.54	25/11/49

Appendix

Table 29: UCR-85 accuracy and mean rank. Mean rank is over the 84 datasets finished by all 28 models we ran on the archive; lower is better. Win/tie/loss is counted per dataset against MILLET’s published accuracies. ‡ 84 of 85 datasets finished.

Figure 20: Direct comparison of our SEA-Net bottleneck

+

Top-

k

model against the published MILLET accuracies on the

85

UCR datasets, one point per dataset. The points above the diagonal are the datasets where our model is the more accurate one. The title gives the number of wins / ties / losses (15 / 17 / 52, where a tie is a difference below

0.005

), and the datasets with the largest positive and negative differences are named.

Figure 21: The effect of the class imbalance, on the UCR datasets whose training set is imbalanced (normalized class entropy below one). Top: the balanced accuracy of our model and of MILLET measured three ways, i.e. published, re-trained with the published recipe, and re-trained with the configuration of Table 15 . Bottom: the difference, ours minus MILLET, where green means that our model is the better one. Each title gives the wins / ties / losses and the mean gain. The imbalance does not hurt the smaller model more than the baseline: the differences are small on both sides, and they do not grow with the imbalance.

Figure 22: Average accuracy rank (

1=

best, lower is better) of our model and of the three MILLET measurements, binned by two properties of the dataset: the length of the series ( left ) and the number of training series ( right ). The bin edges are chosen to give bins of roughly equal size, and

n

is the number of datasets in each bin. The ranking is stable across the bins, i.e. our SEA-Net model does not fall further behind on the long series or on the small training sets.

#	Model	$n$	Acc.	Bal. acc.	AUROC	NLL	AOPCR	Params
1	SEA-Net base (wide, $d=128$ ) + additive (MILLET)	85	0.8298	0.8077	0.9295	0.5088	0.551	269,083
2	SEA-Net base (wide, $d=128$ ) + class-wise conjunctive	85	0.8292	0.8039	0.9255	0.5113	0.564	269,164
3	MILLET (InceptionTime + conjunctive), our recipe	84	0.8274	0.7997	0.9275	0.5297	0.812	423,707
4	SEA-Net base (wide, $d=128$ ) + conjunctive (MILLET)	85	0.8238	0.7993	0.9281	0.5180	0.626	269,083
5	SEA-Net gated (max) + class-wise conjunctive	85	0.8237	0.7979	0.9288	0.5214	0.619	61,740
6	SEA-Net input-gated + adaptive class-wise	85	0.8153	0.7905	0.9208	0.5545	0.964	58,102

Appendix

Table 30: The 85 UCR datasets: every model that completed the archive, sorted by mean accuracy (

n

= number of datasets finished, out of the 85 that MILLET publishes). The means are over the datasets that each model finished; the win / tie / loss counts of the same rows are given in Table 31 . AOPCR is unnormalised: the rows trained with our recipe can be compared with each other, whereas the † row (the published numbers of MILLET) is on another scale. The best value of each column is in bold (the † row is excluded for AOPCR).

#	Model	W/T/L acc.	W/T/L NLL	W/T/L AOPCR
1	SEA-Net base (wide, $d=128$ ) + additive (MILLET)	24/19/42	60/2/23	3/0/82
2	SEA-Net base (wide, $d=128$ ) + class-wise conjunctive	27/19/39	60/1/24	3/0/82
3	MILLET (InceptionTime + conjunctive), our recipe	13/25/46	56/2/26	3/0/81
4	SEA-Net base (wide, $d=128$ ) + conjunctive (MILLET)	23/20/42	57/5/23	4/0/81
5	SEA-Net gated (max) + class-wise conjunctive	20/17/48	61/0/24	4/0/81
6	SEA-Net input-gated + adaptive class-wise	20/15/50	60/0/25	4/0/81

Appendix

Table 31: Wins / ties / losses (W/T/L) of the models of Table 30 , with the same rows and the same numbering, against the published per-dataset numbers of MILLET, with tie bands of 0.005 (accuracy), 0.01 (NLL) and 0.1 (AOPCR). The published AOPCR values come from MILLET’s recipe and are on another scale, which is why every model “loses” the AOPCR W/T/L against them.

#	Dataset	MILLET pub.	bottleneck	gated (mean)	input-gated	spike/trend	MILLET (ours)	ResNet	FCN
1	Adiac	0.832	0.733	0.788	0.749	0.772	0.812	0.785	0.788
2	ArrowHead	0.838	0.851	0.829	0.840	0.869	0.827	0.886	0.840
3	Beef	0.847	0.822	0.867	0.833	0.767	0.800	0.800	0.733
4	BeetleFly	0.920	0.867	0.750	0.850	0.900	0.883	0.850	0.700
5	BirdChicken	0.940	0.850	0.800	0.950	0.850	0.983	1.000	0.900
6	Car	0.920	0.867	0.883	0.917	0.883	0.911	0.933	0.883

Appendix

Table 32: Per-dataset accuracy on the 85 UCR datasets that MILLET publishes. MILLET pub. is the published mean of 5 seeds of MILLET; the next four columns are SEA-Net encoders with the Top-

k

conjunctive head (the model of the paper is bottleneck ); MILLET (ours) , ResNet and FCN are the baselines re-trained under our recipe, each with conjunctive pooling. The best value of each row is in bold (the published column included).

#	Dataset	MILLET pub. †	bottleneck	gated (mean)	input-gated	spike/trend	MILLET (ours)	ResNet	FCN
1	Adiac	15.89	-0.83	-0.02	-0.27	-0.64	1.60	-2.05	1.73
2	ArrowHead	3.20	0.24	0.45	1.37	0.67	0.60	0.84	1.31
3	Beef	5.49	1.33	1.95	0.80	-0.30	1.34	2.07	1.63
4	BeetleFly	1.24	0.58	2.35	1.18	1.59	0.24	2.29	1.35
5	BirdChicken	3.37	1.96	2.77	1.30	1.58	0.96	1.15	1.36
6	Car	6.62	0.40	1.81	-0.03	2.80	1.02	2.55	3.66

Appendix

Table 33: Per-dataset AOPCR on the 85 UCR datasets that MILLET publishes. MILLET pub. is the published mean of 5 seeds of MILLET; the next four columns are SEA-Net encoders with the Top-

k

conjunctive head (the model of the paper is bottleneck ); MILLET (ours) , ResNet and FCN are the baselines re-trained under our recipe, each with conjunctive pooling. The best value of each row is in bold among the columns trained with our recipe; the published column ( † ) was trained with MILLET’s recipe and its AOPCR is on another scale, so it is shown but not compared.

			WebTraffic			UCR-85 mean
#	How the number was obtained	Runs	Acc.	AOPCR	NDCG@ $n$	Acc.	AOPCR
1	Published, mean of 5 seeds	5	0.924	12.768 †	0.674	0.8445	4.553 †
2	Published, 5-model ensemble	1	0.940	11.100 †	0.694	–	–
3	Re-trained here, our recipe (400 epochs, label smoothing, early stopping): the baseline of every comparison	4	$0.888$ $\pm 0.007$	$2.457$ $\pm 0.655$	$0.681$ $\pm 0.013$	0.8274 (84)	0.812

Appendix

Table 34: The MILLET baseline (InceptionTime

+

conjunctive pooling): the published numbers of Early et al. (2024) and our retrained baseline. The WebTraffic columns are means (

\pm

std) over the runs in both results folders, and the UCR columns are means over the 85 published datasets (the number in brackets if fewer were finished). Only the last row is used in the main comparisons. † : the published AOPCR comes from MILLET’s own recipe and is on another scale, so AOPCR must only be compared within one recipe.

Sea-Net: A Small and Inherently Interpretable Neural Network for Time Series Classification

Abstract

Figures & tables

Appendix

Interpretable Time-Series Classification

AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE

SPaRSe-TIME: Saliency-Projected Low-Rank Temporal Modeling for Efficient and Interpretable Time Series Prediction

Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-Fields

Abstract

Interpretable Time-Series Classification

Explore similar work

AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE

SPaRSe-TIME: Saliency-Projected Low-Rank Temporal Modeling for Efficient and Interpretable Time Series Prediction

Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-Fields