HieraNTP: Structured Risk Representations for Time-Series Anomaly Detection via Hierarchical Next-Token Prediction
Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Abstract
Time-series anomaly detection commonly converts deviations from learned temporal regularities into anomaly scores. This practice can conflate violations of temporal regularity with task-specific anomaly judgments: the same deviation may be anomalous under one task definition and benign under another. Aggregating heterogeneous deviations into a single score before task-specific adaptation can obscure the evidence needed to distinguish these cases. We introduce HieraNTP, a Hierarchical Next-Token Prediction framework that separates self-supervised predictive modeling from task-specific risk interpretation. HieraNTP pretrains a hierarchical tokenizer and a causal next-token predictor to capture temporal regularities at multiple scales. It then organizes prediction discrepancies into a structured risk representation, preserving complementary evidence across temporal scales in token, local-statistic, and observation spaces. A task-adaptive detector uses anomaly labels to select and combine this evidence for the target task. Across anomaly detection benchmarks, HieraNTP improves VUS-PR by 12.0% on TSB-AD-U and 15.5% on TSB-AD-M relative to the strongest baselines, and achieves the highest composite score on both TAB tracks, with controlled comparisons supporting the value of retaining distinct evidence. Our code is available at https://github.com/blisky-li/HieraNTP.
Figures & tables
| Univariate datasets | Multivariate datasets | Rank counts | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metric | Model | IOPS | MGAB | NAB | NEK | Power | SED | Stock | TODS | UCR | WSD | YAHOO | MSL | PSM | SMAP | SMD | SWaT | 1st | 2nd |
| Affiliation-F | HieraNTP | 93.68 | 67.67 | 96.53 | 95.25 | 68.39 | 96.20 | 97.86 | 87.57 | 86.13 | 97.96 | 95.19 | 83.37 | 77.87 | 89.91 | 84.47 | 73.41 | 07 | 02 |
| TimeRCD | 86.98 | 74.54 | 93.81 | 83.95 | 85.02 | 88.99 | 41.31 | 84.86 | 84.34 | 91.00 | 95.94 | 84.66 | 82.31 | 86.01 | 91.75 | 69.43 | 03 | 06 | |
| MOMENT | 87.54 | 66.76 | 90.45 | 92.26 | 75.97 | 59.13 | 45.26 | 59.76 | 75.77 | 95.39 | 79.99 | 74.55 | 65.79 | 77.42 | 74.00 | 70.17 | 00 | 01 | |
| TimesFM | 81.88 | 66.95 | 79.73 | 90.49 | 69.88 | 67.14 | 97.53 | 89.08 | 70.03 | 78.97 | 91.28 | 20.35 | 71.24 | 45.44 | 62.85 | 44.37 | 00 | 02 | |
| Chronos | 90.12 | 67.89 | 86.66 | 93.63 | 69.72 | 67.89 | 96.85 | 91.96 | 74.35 | 90.98 | 96.34 | 75.52 | 70.88 | 72.22 | 75.31 | 70.43 | 02 | 02 | |
| Univariate Detection with 14 datasets | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Metric | HieraNTP | KMeans | DWT- MLEAD | CALF | iTrans former | EIF | ContraAD | DLinear | TranAD | DC detector | DualTF | Modern TCN |
| F1 | 0.322 | 0.293 | 0.242 | 0.182 | 0.194 | 0.166 | 0.041 | 0.165 | 0.158 | 0.024 | 0.078 | 0.163 |
| Aff-F1 | 0.849 | 0.792 | 0.716 | 0.820 | 0.810 | 0.782 | 0.684 | 0.808 | 0.774 | 0.688 | 0.728 | 0.807 |
| AP | 0.419 | 0.356 | 0.291 | 0.231 | 0.243 | 0.213 | 0.033 | 0.213 | 0.201 | 0.034 | 0.083 | 0.210 |
| AUC-ROC | 0.817 | 0.790 | 0.739 | 0.687 | 0.687 | 0.633 | 0.499 | 0.659 | 0.545 | 0.473 | 0.539 | 0.663 |
| VUS-PR | 0.439 | 0.362 | 0.312 | 0.251 | 0.244 | 0.202 | 0.059 | 0.202 | 0.179 | 0.057 | 0.108 | 0.210 |
| Evidence construction | Temporal structure | Scoring and adaptation | ||||||
|---|---|---|---|---|---|---|---|---|
| Configuration | U | M | Configuration | U | M | Configuration | U | M |
| HieraNTP | 65.56 | 50.28 | HieraNTP | 65.56 | 50.28 | HieraNTP | 65.56 | 50.28 |
| Prediction-to-observation only | 59.78 | 44.60 | w/o patch 256 | 65.53 | 49.98 | Mean + detector | 38.20 | 29.12 |
| w/o token-space | 65.35 | 49.69 | w/o patch 128 | 65.52 | 49.29 | Max + detector | 32.11 | 27.47 |
| w/o patch mean | 64.31 | 50.33 | w/o patch 64 | 65.83 | 49.92 | Mean–max + detector | 44.51 | 33.83 |
| w/o patch variance | 62.47 | 47.25 | w/o patch 32 | 66.36 | 50.08 | Time-domain error | 33.85 | 30.81 |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Definition |
|---|---|
| Time index, target-token position, variable, and hierarchy level. | |
| Base discrepancy channel, temporal alignment, processing direction, dataset, and appended-tree index. | |
| Input time series with time steps and variables; and denote the observed history and realized target segment. | |
| Target-task anomaly label available on training sequences and the resulting task-specific anomaly score. | |
| Each sequence’s designated training prefix, used to estimate discrepancy location and scale. | |
| Number of hierarchy levels and temporal support of level ; the evaluated configuration uses and . |
| Source | Comparison at each token-grid offset | Channels |
|---|---|---|
| Token-space | Predicted and realized tokens at six levels | |
| Patch mean | Predicted and realized patch means at six levels | |
| Patch variance | Predicted and realized patch variances at six levels | |
| Decoded observation | Decoded prediction and observed target at six native scales and full resolution | |
| Decoded target | Decoded prediction and realized-token reconstruction at six native scales and full resolution | |
| Total | Two token-grid offsets: 0 and 4 observations | 64 |
| Variant | Ordered operations before temporal summaries | U | M |
|---|---|---|---|
| Direct | Robust standardization, target-interval projection, positive-excess log transform, variable aggregation, sequence ranking | 832 | 1664 |
| Pool-then-calibrate | Hold and log-transform responses, aggregate variables, calibrate the aggregate | 832 | 1664 |
| Calibrate-then-pool | Hold and log-transform responses, calibrate eligible variable responses, aggregate valid variables | 832 | 1664 |
| Complete representation | Concatenation of the three paths after temporal summaries | 2496 | 4992 |
| Name | Generating form or sampling rule | Share |
| Sinusoidal | — | |
| Multiharmonic | — | |
| Square | — | |
| Triangular | — | |
| Sawtooth | — | |
| Repeated motif | ; smooth piecewise patterns or pulse trains | — |
| Parameter | TSB-AD-U | TSB-AD-M | TAB univariate | TAB multivariate |
|---|---|---|---|---|
| Feature dimension | 2496 | 4992 | 2496 | 4992 |
| Number of trees | 384 | 384 | 384 | 384 |
| Maximum depth | 3 | 6 | 6 | 6 |
| Minimum child weight | 64 | 4 | 4 | 4 |
| regularization | 10 | 100 | 100 | 100 |
| Column subsampling | 0.35 | 0.75 | 0.75 | 0.75 |
| Method | AUC-PR | AUC-ROC | VUS-PR | VUS-ROC | Std.-F1 | PA-F1 | Event-F1 | R-F1 | Aff.-F |
|---|---|---|---|---|---|---|---|---|---|
| HieraNTP | 0.643 | 0.925 | 0.656 | 0.939 | – | – | – | – | – |
| Time-RCD+MAFT (FT) | 0.590 | 0.901 | 0.586 | 0.920 | 0.555 | 0.701 | – | – | 0.915 |
| TSPulse (FT) | – | – | 0.552 | – | – | – | – | – | – |
| TSPulse (ZS) | – | – | 0.480 | – | – | – | – | – | – |
| CHARM | – | – | 0.501 | – | – | – | – | – | – |
| Time-RCD | 0.495 | 0.843 | 0.522 | 0.871 | 0.527 | 0.672 | – | – | 0.886 |
| Method | AUC-PR | AUC-ROC | VUS-PR | VUS-ROC | Std.-F1 | PA-F1 | Event-F1 | R-F1 | Aff.-F |
|---|---|---|---|---|---|---|---|---|---|
| HieraNTP | 0.514 | 0.865 | 0.503 | 0.870 | – | – | – | – | – |
| TSPulse (FT) | – | – | 0.387 | – | – | – | – | – | – |
| TSPulse (ZS) | – | – | 0.361 | – | – | – | – | – | – |
| xLSTMAD | 0.344 | 0.739 | 0.392 | 0.778 | 0.425 | 0.621 | 0.532 | 0.390 | 0.833 |
| MMPAD | 0.309 | 0.744 | 0.354 | 0.759 | 0.359 | 0.480 | 0.391 | 0.300 | 0.814 |
| CHARM | – | – | 0.391 | – | – | – | – | – | – |
| Dataset | Sequences | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|
| CATSv2 | 1 | 0.503 | 0.725 | 0.375 | 0.741 |
| Daphnet | 1 | 0.176 | 0.871 | 0.188 | 0.876 |
| Exathlon | 30 | 0.950 | 0.996 | 0.950 | 0.996 |
| IOPS | 15 | 0.477 | 0.958 | 0.515 | 0.977 |
| LTDB | 8 | 0.329 | 0.619 | 0.408 | 0.711 |
| MGAB | 8 | 0.004 | 0.612 | 0.010 | 0.686 |
| Dataset | Sequences | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|
| CATSv2 | 5 | 0.256 | 0.796 | 0.221 | 0.799 |
| CreditCard | 1 | 0.041 | 0.758 | 0.062 | 0.753 |
| Daphnet | 1 | 0.387 | 0.938 | 0.400 | 0.941 |
| Exathlon | 25 | 0.999 | 1.000 | 0.998 | 1.000 |
| GECCO | 1 | 0.346 | 0.789 | 0.099 | 0.675 |
| Genesis | 1 | 0.005 | 0.724 | 0.015 | 0.837 |
| Method | F1 | Aff-F1 | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|---|
| HieraNTP | 0.322 | 0.849 | 0.419 | 0.817 | 0.439 | 0.837 |
| KMeans | 0.293 | 0.792 | 0.356 | 0.790 | 0.362 | 0.784 |
| DWT-MLEAD | 0.242 | 0.716 | 0.291 | 0.739 | 0.312 | 0.699 |
| CALF | 0.182 | 0.820 | 0.231 | 0.687 | 0.251 | 0.758 |
| iTransformer | 0.194 | 0.810 | 0.243 | 0.687 | 0.244 | 0.753 |
| HBOS | 0.107 | 0.696 | 0.104 | 0.599 | 0.242 | 0.625 |
| Method | F1 | Aff-F1 | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|---|
| HieraNTP | 0.331 | 0.793 | 0.376 | 0.782 | 0.388 | 0.794 |
| OCSVM | 0.161 | 0.677 | 0.176 | 0.645 | 0.309 | 0.656 |
| KMeans | 0.296 | 0.698 | 0.340 | 0.744 | 0.302 | 0.685 |
| KNN | 0.179 | 0.699 | 0.336 | 0.714 | 0.301 | 0.707 |
| DAGMM | 0.083 | 0.561 | 0.171 | 0.648 | 0.294 | 0.660 |
| TimesNet | 0.199 | 0.789 | 0.259 | 0.688 | 0.271 | 0.722 |
| Dataset | F1 | Aff-F1 | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|---|
| Daphnet | 0.051 | 0.712 | 0.063 | 0.642 | 0.112 | 0.715 |
| ECG | 0.616 | 0.903 | 0.728 | 0.969 | 0.842 | 0.984 |
| GAIA | 0.262 | 0.877 | 0.444 | 0.906 | 0.382 | 0.909 |
| Genesis | 0.523 | 0.974 | 0.630 | 0.983 | 0.740 | 0.994 |
| GHL | 0.000 | 0.714 | 0.003 | 0.188 | 0.005 | 0.305 |
| IOPS | 0.336 | 0.898 | 0.462 | 0.923 | 0.478 | 0.934 |
| Dataset | F1 | Aff-F1 | AP | AUC-ROC | VUS-PR | VUS-ROC |
|---|---|---|---|---|---|---|
| ASD | 0.384 | 0.832 | 0.377 | 0.837 | 0.389 | 0.862 |
| CalIt2 | 0.149 | 0.780 | 0.076 | 0.811 | 0.102 | 0.807 |
| CATSv2 | 0.134 | 0.689 | 0.117 | 0.663 | 0.147 | 0.709 |
| CICIDS | 0.040 | 0.854 | 0.125 | 0.958 | 0.153 | 0.931 |
| Creditcard | 0.078 | 0.754 | 0.128 | 0.955 | 0.098 | 0.914 |
| DAPHNET | 0.277 | 0.828 | 0.262 | 0.787 | 0.372 | 0.843 |
| Configuration | VUS-PR | AP | VUS-ROC | AUC-ROC |
|---|---|---|---|---|
| HieraNTP | 65.56 | 64.26 | 93.85 | 92.51 |
| Shared base detector only | 61.48 | 59.93 | 93.36 | 91.87 |
| Time-domain error | 33.85 | 34.42 | 80.36 | 76.18 |
| Fixed evidence averaging | 43.65 | 39.96 | 87.42 | 84.61 |
| Time-domain error + detector | 29.89 | 29.71 | 78.77 | 73.77 |
| Mean + detector | 38.20 | 32.71 | 86.72 | 82.56 |
| Configuration | VUS-PR | AP | VUS-ROC | AUC-ROC |
|---|---|---|---|---|
| HieraNTP | 50.28 | 51.39 | 86.99 | 86.48 |
| Shared base detector only | 49.91 | 51.04 | 86.79 | 86.33 |
| Time-domain error | 30.81 | 30.91 | 74.86 | 72.06 |
| Fixed evidence averaging | 35.27 | 34.09 | 77.53 | 75.74 |
| Time-domain error + detector | 28.34 | 28.44 | 74.09 | 70.57 |
| Mean + detector | 29.12 | 24.77 | 78.22 | 74.25 |
| Track | Labels | Datasets | Sequences | VUS-PR | AP | VUS-ROC | AUC-ROC |
|---|---|---|---|---|---|---|---|
| U | Available | 19 | 346 | 65.94 | 64.57 | 94.07 | 92.70 |
| U | Unavailable | 4 | 4 | 32.45 | 37.55 | 75.58 | 75.77 |
| M | Available | 11 | 173 | 51.38 | 52.31 | 87.40 | 86.79 |
| M | Unavailable | 6 | 7 | 23.03 | 28.78 | 76.75 | 78.75 |
| TSB-AD-U | TSB-AD-M | |||
|---|---|---|---|---|
| Removed component | 95% interval | 95% interval | ||
| Patch variance | ||||
| Observation-space discrepancy | ||||
| Temporal summaries | ||||
| Reference calibration | ||||
| Track | Target-span labels | Token pairs | (%) | (%) | Median |
|---|---|---|---|---|---|
| U | All | 111,516 | 5.09 | 33.58 | 0.656 |
| Normal | 95,723 | 4.75 | 33.11 | 0.659 | |
| Anomalous | 4,837 | 5.50 | 30.39 | 0.710 | |
| Mixed | 10,956 | 7.85 | 39.10 | 0.612 | |
| M | All | 89,528 | 5.96 | 31.05 | 0.687 |
| Normal | 76,965 | 5.83 | 30.62 | 0.699 |
Time-Series Anomaly Detection
Time-Series Anomaly Detection identifies unusual patterns or events in sequential data, from sensor readings and network traffic to health and industrial telemetry.
See all 214 papers