HieraNTP: Structured Risk Representations for Time-Series Anomaly Detection via Hierarchical Next-Token Prediction

Yujie Li; Zezhi Shao; Boyu Diao; Zhulin An; Yongjun Xu; Xueqi Cheng; Fei Wang

cs.LGOct 31, 2026

HieraNTP: Structured Risk Representations for Time-Series Anomaly Detection via Hierarchical Next-Token Prediction

Authors: Yujie Li, Zezhi Shao, Boyu Diao, Zhulin An, Yongjun Xu, Xueqi Cheng, Fei Wang

Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Abstract

Time-series anomaly detection commonly converts deviations from learned temporal regularities into anomaly scores. This practice can conflate violations of temporal regularity with task-specific anomaly judgments: the same deviation may be anomalous under one task definition and benign under another. Aggregating heterogeneous deviations into a single score before task-specific adaptation can obscure the evidence needed to distinguish these cases. We introduce HieraNTP, a Hierarchical Next-Token Prediction framework that separates self-supervised predictive modeling from task-specific risk interpretation. HieraNTP pretrains a hierarchical tokenizer and a causal next-token predictor to capture temporal regularities at multiple scales. It then organizes prediction discrepancies into a structured risk representation, preserving complementary evidence across temporal scales in token, local-statistic, and observation spaces. A task-adaptive detector uses anomaly labels to select and combine this evidence for the target task. Across anomaly detection benchmarks, HieraNTP improves VUS-PR by 12.0% on TSB-AD-U and 15.5% on TSB-AD-M relative to the strongest baselines, and achieves the highest composite score on both TAB tracks, with controlled comparisons supporting the value of retaining distinct evidence. Our code is available at https://github.com/blisky-li/HieraNTP.

Figures & tables

Figure 1: Schematic motivation using a Yahoo sequence from TSB-AD. The task treats missing peaks (A) as normal and disrupted cycles (B) as anomalous. (a) Early scalar reduction can obscure this distinction. (b) HieraNTP retains a structured risk representation for task-adaptive detection.

Figure 2: Overview of HieraNTP. (a) Multiscale reconstruction pretrains the causal tokenizer. (b) Prediction and realized-target ( $\mathrm{GT}$ ) branches yield token-, local-statistic-, and observation-space discrepancies. (c) Labels train the task-adaptive detector with predictive modules frozen.

Panel of Figure 3 — Figure 3: TSB-AD Eval VUS-PR. HieraNTP is red, target-adapted methods are blue, and zero-shot or training-free methods are green.

		Univariate datasets											Multivariate datasets					Rank counts
Metric	Model	IOPS	MGAB	NAB	NEK	Power	SED	Stock	TODS	UCR	WSD	YAHOO	MSL	PSM	SMAP	SMD	SWaT	1st	2nd
Affiliation-F	HieraNTP	93.68	67.67	96.53	95.25	68.39	96.20	97.86	87.57	86.13	97.96	95.19	83.37	77.87	89.91	84.47	73.41	07	02
	TimeRCD $\diamond$	86.98	74.54	93.81	83.95	85.02	88.99	41.31	84.86	84.34	91.00	95.94	84.66	82.31	86.01	91.75	69.43	03	06
	MOMENT $\diamond$	87.54	66.76	90.45	92.26	75.97	59.13	45.26	59.76	75.77	95.39	79.99	74.55	65.79	77.42	74.00	70.17	00	01
	TimesFM $\diamond$	81.88	66.95	79.73	90.49	69.88	67.14	97.53	89.08	70.03	78.97	91.28	20.35	71.24	45.44	62.85	44.37	00	02
	Chronos $\diamond$	90.12	67.89	86.66	93.63	69.72	67.89	96.85	91.96	74.35	90.98	96.34	75.52	70.88	72.22	75.31	70.43	02	02

Table 1: Four-metric results on the 16 datasets selected by TimeRCD (%).

\diamond

: zero-shot. Red bold and blue underlined italics mark the best and second-best results per dataset–metric column.

Univariate Detection with 14 datasets
Metric	HieraNTP	KMeans	DWT- MLEAD	CALF	iTrans former	EIF	ContraAD	DLinear	TranAD	DC detector	DualTF	Modern TCN
F1	0.322	0.293	0.242	0.182	0.194	0.166	0.041	0.165	0.158	0.024	0.078	0.163
Aff-F1	0.849	0.792	0.716	0.820	0.810	0.782	0.684	0.808	0.774	0.688	0.728	0.807
AP	0.419	0.356	0.291	0.231	0.243	0.213	0.033	0.213	0.201	0.034	0.083	0.210
AUC-ROC	0.817	0.790	0.739	0.687	0.687	0.633	0.499	0.659	0.545	0.473	0.539	0.663
VUS-PR	0.439	0.362	0.312	0.251	0.244	0.202	0.059	0.202	0.179	0.057	0.108	0.210

Table 2: TAB full-shot results. Red bold: best; blue underlined italics: second best.

Evidence construction			Temporal structure			Scoring and adaptation
Configuration	U	M	Configuration	U	M	Configuration	U	M
HieraNTP	65.56	50.28	HieraNTP	65.56	50.28	HieraNTP	65.56	50.28
Prediction-to-observation only	59.78	44.60	w/o patch 256	65.53	49.98	Mean + detector	38.20	29.12
w/o token-space	65.35	49.69	w/o patch 128	65.52	49.29	Max + detector	32.11	27.47
w/o patch mean	64.31	50.33	w/o patch 64	65.83	49.92	Mean–max + detector	44.51	33.83
w/o patch variance	62.47	47.25	w/o patch 32	66.36	50.08	Time-domain error	33.85	30.81

Table 3: Evidence and adaptation analysis on TSB-AD-U/M (VUS-PR, %).

Figure 4: Dataset-wise VUS-PR differences: HieraNTP minus component removal (pp). Numbers mark magnitudes

\geq 5

; parentheses give sequence counts. Rows are clustered within each track.

Figure 5: Evidence usage in dataset-specific extensions. Split-gain shares (%) are normalized by column group; numbers mark row-wise maxima per group. Full denotes full resolution; parentheses give evaluation sequence counts. Rows are clustered by source profiles within each track.

Figure 6: Top-five features in Exathlon (U) and TAO (M). Colors are scaled per feature; percentages show dataset-specific split gain. VUS-PR is computed per window.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition
$t,\tau,c,h$	Time index, target-token position, variable, and hierarchy level.
$j,a,d,f,k$	Base discrepancy channel, temporal alignment, processing direction, dataset, and appended-tree index.
$X=(x_{t,c})\in\mathbb{R}^{T\times C}$	Input time series with $T$ time steps and $C$ variables; $X_{\mathrm{hist}}$ and $X_{\mathrm{GT}}$ denote the observed history and realized target segment.
$y_{t},s_{t}$	Target-task anomaly label available on training sequences and the resulting task-specific anomaly score.
$X_{\mathrm{ref}}$	Each sequence’s designated training prefix, used to estimate discrepancy location and scale.
$H,\delta_{h}$	Number of hierarchy levels and temporal support of level $h$ ; the evaluated configuration uses $H=6$ and $(\delta_{1},\ldots,\delta_{H})=(8,16,32,64,128,256)$ .

Appendix

Table 4: Notation used in Figure 2 , the method, and the appendix.

Source	Comparison at each token-grid offset	Channels
Token-space	Predicted and realized tokens at six levels	$2\times 6=12$
Patch mean	Predicted and realized patch means at six levels	$2\times 6=12$
Patch variance	Predicted and realized patch variances at six levels	$2\times 6=12$
Decoded observation	Decoded prediction and observed target at six native scales and full resolution	$2\times(6+1)=14$
Decoded target	Decoded prediction and realized-token reconstruction at six native scales and full resolution	$2\times(6+1)=14$
Total	Two token-grid offsets: 0 and 4 observations	64

Appendix

Table 5: Composition of the 64 base discrepancy channels.

Variant	Ordered operations before temporal summaries	U	M
Direct	Robust standardization, target-interval projection, positive-excess log transform, variable aggregation, sequence ranking	832	1664
Pool-then-calibrate	Hold and log-transform responses, aggregate variables, calibrate the aggregate	832	1664
Calibrate-then-pool	Hold and log-transform responses, calibrate eligible variable responses, aggregate valid variables	832	1664
Complete representation	Concatenation of the three paths after temporal summaries	2496	4992

Appendix

Table 6: Processing paths and resulting representation dimensions.

Name	Generating form or sampling rule	Share
Sinusoidal	$P_{t}=\sin\theta_{t}$	—
Multiharmonic	$P_{t}=\sum_{k=1}^{K}a_{k}\sin(k\theta_{t}+\phi_{k})$	—
Square	$P_{t}=\operatorname{sign}(\sin\theta_{t})$	—
Triangular	$P_{t}=2\|2r_{t}\|-1$	—
Sawtooth	$P_{t}=2r_{t}$	—
Repeated motif	$P_{t}=m(\operatorname{frac}(v_{t}))$ ; smooth piecewise patterns or pulse trains	—

Appendix

Table 7: Synthetic patterns and sampling strata for next-token pretraining. Component formulas are representative forms before amplitude scaling and normal shape variations; sampling proportions apply to the four strata.

Parameter	TSB-AD-U	TSB-AD-M	TAB univariate	TAB multivariate
Feature dimension	2496	4992	2496	4992
Number of trees	384	384	384	384
Maximum depth	3	6	6	6
Minimum child weight	64	4	4	4
$L_{2}$ regularization	10	100	100	100
Column subsampling	0.35	0.75	0.75	0.75

Appendix

Table 8: Initial-stage XGBoost parameters. TAB uses the same parameters for both tracks and appends 32 trees for each adapted dataset.

Method	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	Std.-F1	PA-F1	Event-F1	R-F1	Aff.-F
HieraNTP	0.643	0.925	0.656	0.939	–	–	–	–	–
Time-RCD+MAFT (FT)	0.590	0.901	0.586	0.920	0.555	0.701	–	–	0.915
TSPulse (FT)	–	–	0.552	–	–	–	–	–	–
TSPulse (ZS)	–	–	0.480	–	–	–	–	–	–
CHARM	–	–	0.501	–	–	–	–	–	–
Time-RCD	0.495	0.843	0.522	0.871	0.527	0.672	–	–	0.886

Appendix

Table 9: Results on TSB-AD-U (350 evaluation sequences).

Method	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	Std.-F1	PA-F1	Event-F1	R-F1	Aff.-F
HieraNTP	0.514	0.865	0.503	0.870	–	–	–	–	–
TSPulse (FT)	–	–	0.387	–	–	–	–	–	–
TSPulse (ZS)	–	–	0.361	–	–	–	–	–	–
xLSTMAD	0.344	0.739	0.392	0.778	0.425	0.621	0.532	0.390	0.833
MMPAD	0.309	0.744	0.354	0.759	0.359	0.480	0.391	0.300	0.814
CHARM	–	–	0.391	–	–	–	–	–	–

Appendix

Table 10: Results on TSB-AD-M (180 evaluation sequences).

Dataset	Sequences	AP	AUC-ROC	VUS-PR	VUS-ROC
CATSv2	1	0.503	0.725	0.375	0.741
Daphnet	1	0.176	0.871	0.188	0.876
Exathlon	30	0.950	0.996	0.950	0.996
IOPS	15	0.477	0.958	0.515	0.977
LTDB	8	0.329	0.619	0.408	0.711
MGAB	8	0.004	0.612	0.010	0.686

Appendix

Table 11: Per-dataset results on the 350 TSB-AD-U evaluation sequences.

Dataset	Sequences	AP	AUC-ROC	VUS-PR	VUS-ROC
CATSv2	5	0.256	0.796	0.221	0.799
CreditCard	1	0.041	0.758	0.062	0.753
Daphnet	1	0.387	0.938	0.400	0.941
Exathlon	25	0.999	1.000	0.998	1.000
GECCO	1	0.346	0.789	0.099	0.675
Genesis	1	0.005	0.724	0.015	0.837

Appendix

Table 12: Per-dataset results on the 180 TSB-AD-M evaluation sequences.

Method	F1	Aff-F1	AP	AUC-ROC	VUS-PR	VUS-ROC
HieraNTP	0.322	0.849	0.419	0.817	0.439	0.837
KMeans	0.293	0.792	0.356	0.790	0.362	0.784
DWT-MLEAD	0.242	0.716	0.291	0.739	0.312	0.699
CALF	0.182	0.820	0.231	0.687	0.251	0.758
iTransformer	0.194	0.810	0.243	0.687	0.244	0.753
HBOS	0.107	0.696	0.104	0.599	0.242	0.625

Appendix

Table 13: Extended TAB univariate comparison on 14 datasets.

Method	F1	Aff-F1	AP	AUC-ROC	VUS-PR	VUS-ROC
HieraNTP	0.331	0.793	0.376	0.782	0.388	0.794
OCSVM	0.161	0.677	0.176	0.645	0.309	0.656
KMeans	0.296	0.698	0.340	0.744	0.302	0.685
KNN	0.179	0.699	0.336	0.714	0.301	0.707
DAGMM	0.083	0.561	0.171	0.648	0.294	0.660
TimesNet	0.199	0.789	0.259	0.688	0.271	0.722

Appendix

Table 14: Extended TAB multivariate comparison on 28 datasets.

Dataset	F1	Aff-F1	AP	AUC-ROC	VUS-PR	VUS-ROC
Daphnet	0.051	0.712	0.063	0.642	0.112	0.715
ECG	0.616	0.903	0.728	0.969	0.842	0.984
GAIA	0.262	0.877	0.444	0.906	0.382	0.909
Genesis	0.523	0.974	0.630	0.983	0.740	0.994
GHL	0.000	0.714	0.003	0.188	0.005	0.305
IOPS	0.336	0.898	0.462	0.923	0.478	0.934

Appendix

Table 15: HieraNTP results on all 15 TAB univariate datasets.

Dataset	F1	Aff-F1	AP	AUC-ROC	VUS-PR	VUS-ROC
ASD	0.384	0.832	0.377	0.837	0.389	0.862
CalIt2	0.149	0.780	0.076	0.811	0.102	0.807
CATSv2	0.134	0.689	0.117	0.663	0.147	0.709
CICIDS	0.040	0.854	0.125	0.958	0.153	0.931
Creditcard	0.078	0.754	0.128	0.955	0.098	0.914
DAPHNET	0.277	0.828	0.262	0.787	0.372	0.843

Appendix

Table 16: HieraNTP results on all 29 TAB multivariate datasets.

Configuration	VUS-PR	AP	VUS-ROC	AUC-ROC
HieraNTP	65.56	64.26	93.85	92.51
Shared base detector only	61.48	59.93	93.36	91.87
Time-domain error	33.85	34.42	80.36	76.18
Fixed evidence averaging	43.65	39.96	87.42	84.61
Time-domain error + detector	29.89	29.71	78.77	73.77
Mean + detector	38.20	32.71	86.72	82.56

Appendix

Table 17: Complete TSB-AD-U ablations: sequence-mean metrics (%, higher is better).

Configuration	VUS-PR	AP	VUS-ROC	AUC-ROC
HieraNTP	50.28	51.39	86.99	86.48
Shared base detector only	49.91	51.04	86.79	86.33
Time-domain error	30.81	30.91	74.86	72.06
Fixed evidence averaging	35.27	34.09	77.53	75.74
Time-domain error + detector	28.34	28.44	74.09	70.57
Mean + detector	29.12	24.77	78.22	74.25

Appendix

Table 18: Complete TSB-AD-M ablations: sequence-mean metrics (%, higher is better).

Track	Labels	Datasets	Sequences	VUS-PR	AP	VUS-ROC	AUC-ROC
U	Available	19	346	65.94	64.57	94.07	92.70
U	Unavailable	4	4	32.45	37.55	75.58	75.77
M	Available	11	173	51.38	52.31	87.40	86.79
M	Unavailable	6	7	23.03	28.78	76.75	78.75

Appendix

Table 19: Full-model results by target-dataset anomaly-label availability (%). Datasets and sequences refer to the Eval subsets.

	TSB-AD-U		TSB-AD-M
Removed component	$\Delta$	95% interval	$\Delta$	95% interval
Patch variance	$+3.09$	$[+1.49,+4.72]$	$+3.03$	$[+1.49,+4.70]$
Observation-space discrepancy	$+3.02$	$[+1.24,+4.95]$	$+2.51$	$[+0.90,+4.36]$
Temporal summaries	$+6.31$	$[+4.54,+8.11]$	$+4.64$	$[+2.87,+6.56]$
Reference calibration	$+5.28$	$[+3.32,+7.28]$	$-1.43$	$[-4.33,+1.37]$

Appendix

Table 20: Paired VUS-PR differences on TSB-AD-U/M (percentage points), with 95% sequence-resampling intervals. Positive values favor HieraNTP.

Track	Target-span labels	Token pairs	$\cos\theta<0$ (%)	$\cos\theta<0.5$ (%)	Median
U	All	111,516	5.09	33.58	0.656
	Normal	95,723	4.75	33.11	0.659
	Anomalous	4,837	5.50	30.39	0.710
	Mixed	10,956	7.85	39.10	0.612
M	All	89,528	5.96	31.05	0.687
	Normal	76,965	5.83	30.62	0.699

Appendix

Table 21: Cosine similarity between predicted and realized tokens in the diagnostic sample.

Figure 7: Evidence responses in (a) YAHOO, (b) SED, (c) NAB, and (d) NEK. YAHOO emphasizes short-patch variance, whereas SED combines patch lengths 8 and 64 with sharp maxima and sustained temporal means. NAB’s large-patch mean evidence forms broad responses around later changes. In NEK, short-patch mean evidence concentrates around early deviations, while token-space and variance responses extend over different intervals. Throughout the gallery, bins show mean responses and any anomaly; observations are min–max normalized. Multivariate panels show the smoothest variable with appreciable variation, selected without labels. Axes give time indices, and dashed lines mark reference-prefix boundaries.

Figure 8: Evidence responses in (a) SMAP, (b) WSD, (c) Stock, and (d) IOPS. SMAP’s highest-gain variance feature follows the downward excursion alongside broader decoded-target responses. WSD combines decoded-observation and variance evidence across recurring cycles. Stock emphasizes current full-resolution responses. IOPS’s decoded summaries extend beyond the labeled event.

Figure 9: Multivariate evidence in (a) SMD entity 1, (b) TAO entity 7, (c) SMD entity 18, and (d) SVDB entity 14. The same SMD features respond to recurring changes in (a) and the later level drop in (c). TAO combines slowly varying token-space responses with faster decoded-observation and temporally averaged decoded-target evidence. SVDB combines decoded, variance, and mean evidence with distinct temporal response patterns.

Time-Series Anomaly Detection

Time-Series Anomaly Detection identifies unusual patterns or events in sequential data, from sensor readings and network traffic to health and industrial telemetry.

See all 215 papers

LEAP: a Latent Embedding Anomaly Patch-Transformer for time-series anomaly detection
Oct 26, 2026Cristhian Moya-Mota, Ignacio Aguilera-Martos, Diego García-Gil +1Time-Series Anomaly Detection Unsupervised Anomaly Detection
Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain. · Andalusian Institute of Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain. · Department of Software Engineering, University of Granada, Granada, Spain
Generalist Representation, Specialist Detection: TS-Router for Time-Series Anomaly Detection
Oct 1, 2026Tian Lan, Yifei Gao, Yimeng Lu +6Unsupervised Anomaly Detection Time-Series Anomaly Detection
Department of Industrial Engineering Tsinghua University · Huawei · Datadog AI Research Paris, France
LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection
Feb 9, 2026Dezheng Wang, Tong Chen, Guansong Pang +3Multi-View Learning Unsupervised Anomaly Detection
School of Automation, Southeast University Nanjing, China · School of Electrical Engineering and Computer Science, The University of Queensland Brisbane, Australia · School of Computing and Information Systems, Singapore Management University Singapore

Oct 26, 2026J/K move · Enter open · S save

LEAP: a Latent Embedding Anomaly Patch-Transformer for time-series anomaly detection

Cristhian Moya-Mota, Ignacio Aguilera-Martos, Diego García-Gil, Julián Luengo

Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain. · Andalusian Institute of Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain. · Department of Software Engineering, University of Granada, Granada, Spain

Time-Series Anomaly Detection Unsupervised Anomaly Detection Time Series Transformers Sequence Modeling

Open paper AskPDF arXiv

Time-series anomaly detection is dominated by reconstruction-based neural detectors, whose objective is structurally flawed. Since the target is also the input, an expressive model can learn to reconstruct anomalies too, leaving uninformative residuals. Recent large-scale benchmarks show that, on curated data, elaborate deep detectors rarely beat simple statistical baselines, while current leaders rely on representation learning. We propose LEAP (Latent Embedding Anomaly Patch-Transformer), which combines masked prediction with latent representation-based anomaly evidence, without relying on reconstruction or attention-based scoring. The model removes patches from a window and predicts them from the remaining temporal and cross-channel context, so prediction errors measure how well learned dynamics explain hidden segments. Deterministic complementary group masking ensures every time step is scored while hidden, with identical visible fractions during training and inference. LEAP adds three label-free evidence sources: latent-embedding novelty calibrated by local density, local temporal contrast, and per-channel novelty pooled over channels selected by six label-free criteria. Under the TSB-AD protocol, the full model achieves a VUS-PR of

0.5179 \pm 0.0022

180

held-out multivariate series. On TSB-AD-U, a univariate variant combining multi-scale masked-prediction scoring with latent novelty, without contrast or channel-pool novelty, achieves

0.6194 \pm 0.0016

350

held-out series. Both results average three random seeds. A five-stage ablation on TSB-AD-M, supported by paired Wilcoxon signed-rank tests, shows statistically significant gains from each successive component.

Abstract

Time-Series Anomaly Detection

Explore similar work

LEAP: a Latent Embedding Anomaly Patch-Transformer for time-series anomaly detection

Generalist Representation, Specialist Detection: TS-Router for Time-Series Anomaly Detection

LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection