ORBIT: Autoregressive Time Series Generation via Next Frequency-Band Prediction

Chunyi Hou; Xiangfei Qiu; Hanyin Cheng; Chenchen Sun; Yisheng An; Yang Shu; Jilin Hu; Bin Yang

cs.LGOct 19, 2026

ORBIT: Autoregressive Time Series Generation via Next Frequency-Band Prediction

Authors: Chunyi Hou, Xiangfei Qiu, Hanyin Cheng, Chenchen Sun, Yisheng An, Yang Shu, Jilin Hu, Bin Yang

Organizations: School of Data Science and Engineering, East China Normal University, Shanghai, China · Chang’an University, Xi’an, China

Abstract

Deep generative models provide an effective modern solution to alleviating the problem of data scarcity by synthesizing realistic time series. However, despite the considerable progress achieved by existing deep time series generators, two key challenges remain: (1) learning hierarchical representations for heterogeneous temporal patterns; and (2) coordinating the generation process of heterogeneous temporal dynamics. To address these key bottlenecks, in this work, we propose ORBIT, a unified deep generative framework for time series. Specifically, we develop two key components: the Frequency-Hierarchical VQ-VAE (FH-VQ-VAE) and the Frequency-Band Autoregressive Transformer (FBAR). The former learns hierarchical representations by decomposing latent temporal representations into different frequency bands and applying residual quantization with increasing token budgets. The latter establishes a novel low-to-high frequency autoregressive direction, yielding a time series generation process that first constructs global temporal structures and then incrementally incorporates fine-grained temporal details. These compact designs not only disentangle heterogeneous temporal patterns for more effective representation learning but also mitigate potential interference among different temporal dynamics during generation. Extensive experiments on multiple challenging real-world benchmarks demonstrate that ORBIT achieves state-of-the-art overall performance while exhibiting superior inference efficiency compared with advanced baselines.

Figures & tables

Figure 1. Different frequency bands correspond to distinct temporal dynamics, with low-frequency components capturing smooth and slow variations, while high-frequency components characterize rapid temporal fluctuations.

Figure 2. Time Series Generation with a Standard Autoregressive Backbone. At certain inference steps (highlighted by the red boxes ), the model is needed to jointly generate coarse-grained global trends and fine-grained local fluctuations.

Figure 3. The overview of our proposed ORBIT.

Dataset	Length	ORBIT ( Ours )		TimeMAR ( Xu et al., 2026 )		TimeDP ( Huang et al., 2025 )		TimeVQVAE ( Lee et al., 2023 )		TimeVQVAE-C ( Lee et al., 2023 )		TimeVAE ( Desai et al., 2021 )		GT-GAN ( Jeon et al., 2022 )		TimeGAN ( Yoon et al., 2019 )
Dataset	Length	MMD	K-L	MMD	K-L	MMD	K-L	MMD	K-L	MMD	K-L	MMD	K-L	MMD	K-L	MMD	K-L
Electricity	24	${\color[rgb]{1,0,0}\mathbf{0.001}\!\pm\!0.000}$	${\color[rgb]{1,0,0}\mathbf{0.007}\!\pm\!0.002}$	${\color[rgb]{0,0,1}\underline{0.001\!\pm\!0.001}}$	${\color[rgb]{0,0,1}\underline{0.010\!\pm\!0.000}}$	${\color[rgb]{0,0,1}\underline{0.001\!\pm\!0.001}}$	$0.016\!\pm\!0.015$	$0.142\!\pm\!0.017$	$0.302\!\pm\!0.022$	$0.002\!\pm\!0.001$	$0.023\!\pm\!0.007$	$0.980\!\pm\!0.016$	$1.245\!\pm\!0.018$	$0.189\!\pm\!0.085$	$0.423\!\pm\!0.178$	$0.270\!\pm\!0.095$	$0.333\!\pm\!0.134$
	96	${\color[rgb]{1,0,0}\mathbf{0.001}\!\pm\!0.000}$	$0.029\!\pm\!0.005$	${\color[rgb]{0,0,1}\underline{0.001\!\pm\!0.001}}$	${\color[rgb]{0,0,1}\underline{0.026\!\pm\!0.004}}$	$0.002\!\pm\!0.001$	${\color[rgb]{1,0,0}\mathbf{0.015}\!\pm\!0.017}$	$0.153\!\pm\!0.023$	$0.245\!\pm\!0.039$	$0.002\!\pm\!0.000$	$0.031\!\pm\!0.005$	$0.648\!\pm\!0.012$	$0.790\!\pm\!0.018$	$0.317\!\pm\!0.311$	$0.447\!\pm\!0.147$	$0.401\!\pm\!0.219$	$0.552\!\pm\!0.241$
	168	${\color[rgb]{1,0,0}\mathbf{0.001}\!\pm\!0.000}$	${\color[rgb]{1,0,0}\mathbf{0.012}\!\pm\!0.002}$	${\color[rgb]{0,0,1}\underline{0.001\!\pm\!0.001}}$	${\color[rgb]{0,0,1}\underline{0.012\!\pm\!0.003}}$	${\color[rgb]{0,0,1}\underline{0.001\!\pm\!0.001}}$	$0.012\!\pm\!0.016$	$0.152\!\pm\!0.024$	$0.280\!\pm\!0.051$	$0.002\!\pm\!0.001$	$0.027\!\pm\!0.015$	$0.577\!\pm\!0.006$	$0.734\!\pm\!0.023$	$0.254\!\pm\!0.166$	$0.407\!\pm\!0.079$	$0.367\!\pm\!0.255$	$0.488\!\pm\!0.175$
	Average	0.001	0.016	0.001	0.016	0.001	0.014	0.149	0.276	0.002	0.027	0.735	0.923	0.253	0.426	0.346	0.458
Solar	24	${\color[rgb]{1,0,0}\mathbf{0.011}\!\pm\!0.001}$	${\color[rgb]{1,0,0}\mathbf{0.012}\!\pm\!0.001}$	${\color[rgb]{0,0,1}\underline{0.015\!\pm\!0.000}}$	${\color[rgb]{0,0,1}\underline{0.013\!\pm\!0.000}}$	$0.023\!\pm\!0.016$	$0.017\!\pm\!0.009$	$0.411\!\pm\!0.004$	$0.878\!\pm\!0.031$	$0.036\!\pm\!0.011$	$0.120\!\pm\!0.036$	$0.321\!\pm\!0.005$	$0.458\!\pm\!0.016$	$0.517\!\pm\!0.027$	$0.102\!\pm\!0.039$	$0.513\!\pm\!0.031$	$0.410\!\pm\!0.042$

Table 1. Maximum mean discrepancy (MMD) and Kullback–Leibler divergence (K-L) for time series generation at sequence lengths 24, 96, and 168. Lower values indicate better performance. Results at individual lengths are reported as mean

\pm

standard deviation. Best means are highlighted in red and bold, and second-best results are highlighted in blue and underlined.

Figure 4. MMD and K-L divergence in cross-domain settings of Web datasets for sequence length 168. U, F, and P denote unconditional generation, fine-tuning, and sample prompting, respectively. ORBIT uses CFG to achieve unconditional generation.

Metrics	Length	$\mathrm{ORBIT}_{\mathrm{Full}}$	Variant 1	Variant 2	Variant 3
MMD	96	0.059	0.074	0.079	0.107
MMD	168	0.068	0.108	0.095	0.124
Improvement		–	30.22%	27.01%	45.02%
K-L	96	0.271	0.327	0.336	0.354
K-L	168	0.202	0.228	0.243	0.302
Improvement		–	14.77%	18.31%	27.90%

Table 2. Ablation study results. We report MMD and K-L averaged across datasets for long-sequence time series generation task. Improvement denotes the relative reduction achieved by the full model over each variant, computed from the average scores across lengths 96 and 168.

Figure 5. Comparison of generation time, peak allocated memory, and peak reserved memory between ORBIT and TimeMAR ( Xu et al., 2026 ) for generating 5,000 samples on the Electricity dataset with a fixed batch size of 1,000.

Figure 6. t-SNE visualization of real and synthetic time series samples on the Electricity and Wind benchmarks with a sequence length of 168. Additional qualitative visualizations are provided in Appendix D .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Baseline	Code Repository
TimeMAR	https://github.com/decisionintelligence/TimeMAR
TimeDP	https://github.com/microsoft/TimeCraft/tree/main/TimeDP
TimeGAN	https://github.com/jsyoon0823/TimeGAN
GT-GAN	https://github.com/Jinsung-Jeon/GT-GAN
TimeVAE	https://github.com/abudesai/timeVAE
TimeVQVAE	https://github.com/ML4ITS/TimeVQVAE

Appendix

Table 3. Code repositories of the baseline methods. TimeVQVAE-C shares the implementation of TimeVQVAE.

Hyperparameter		$M$	$d$	$B$	$n_{b}$	$K$
Length	24	6	16	6	[1, 2, 3, 4, 5, 6]	2048
	96	24	16	6	[1, 2, 3, 6, 12, 24]	2048
	168	42	16	6	[1, 2, 6, 12, 24, 42]	2048

Appendix

Table 4. Detailed hyperparameter settings of FH-VQ-VAE.

M

d

B

n_{b}

, and

K

denote the number of continuous latents produced by the encoder, the hidden dimensionality, the number of frequency bands, the number of quantization tokens allocated to the

b

-th frequency band, and the total number of codewords in the codebook, respectively.

Hyperparameter		$D$	$Layer$	$Head$	$FNN$
Length	24	512	4	8	4* $D$
	96	512	4	8	4* $D$
	168	512	4	8	4* $D$

Appendix

Table 5. Detailed hyperparameter settings of FBAR.

D

Layer

Head

, and

FNN

denote the backbone width, number of Transformer layers, number of attention heads, and hidden dimensionality of the feed-forward network, respectively.

Panel of Figure 7 — Figure 7. t-SNE visualization of real and synthetic time series samples on the Traffic, Taxi, Pedestrian, and Fred-MD benchmarks with a sequence length of 24.

Panel of Figure 8 — Figure 8. t-SNE visualization of real and synthetic time series samples on the Traffic, Taxi, Pedestrian, and Fred-MD benchmarks with a sequence length of 96.

Panel of Figure 9 — Figure 9. t-SNE visualization of real and synthetic time series samples on the Traffic, Taxi, Pedestrian, and Fred-MD benchmarks with a sequence length of 168.

Figure 10. Sensitivity analysis of MMD to sampling temperature across eight benchmark datasets.

Figure 11. Sensitivity analysis of K-L to sampling temperature across eight benchmark datasets.

Figure 12. Autoregressive generation process guided by next frequency-band prediction on the Electricity and Taxi benchmarks with a sequence length of 24.

Time Series Generation

Time series generation creates synthetic sequential data that preserve temporal patterns, cross-variable structure, and controllable conditions.

See all 87 papers

HALO: Enhancing Time Series Generation via Hyperspherical Latents and Masked AutoregRessive Modeling
Sep 28, 2026Chunyi Hou, Xiangfei Qiu, Hanyin Cheng +2Variational Autoencoders Time Series Generation
School of Data Science and Engineering, East China Normal University, Shanghai, China
UPLOTS: A Unified Pretrained Language Model for Constrained Time-series Generation
Jun 9, 2026Du Yin, Hao Xue, Jinliang Deng +4Constrained Generative Modeling Time Series Generation
University of New South Wales Sydney, NSW, Australia · HKUST(GZ) China · BUAA Beijing, China
FracTS: Hierarchical and Autoregressive Time Series Generation
Oct 18, 2026Jiayu Li, Umair Afzal, Zilong Zhao +3Time Series Generation Autoregressive Generation
University of Illinois Urbana-Champaign · Betterdata AI · National University of Singapore

Sep 28, 2026J/K move · Enter open · S save

HALO: Enhancing Time Series Generation via Hyperspherical Latents and Masked AutoregRessive Modeling

Chunyi Hou, Xiangfei Qiu, Hanyin Cheng, Yutong Li+1

School of Data Science and Engineering, East China Normal University, Shanghai, China

Variational Autoencoders Time Series Generation Autoregressive Generation Autoregressive Modeling

Open paper AskPDF arXiv

Most existing time series generators rely on a two-stage modeling paradigm: the first stage learns discrete latent representations of time series; the second stage performs autoregressive modeling on these discrete latents through next token prediction. However, this paradigm suffers from two stage-specific limitations: the first stage can lead to information loss when discretizing continuous time series, while the second stage is prone to error accumulation during autoregressive generation. To address these limitations, our core idea is to perform generative modeling in a continuous latent space with a more efficient autoregressive framework. We propose HALO, which enhances time series generation via Hyperspherical Latents and Masked Autoregressive modeling to achieve this goal by tackling two key bottlenecks: (1) variance and scale heterogeneity of continuous latent representations; (2) the difficulty of balancing generation efficiency with temporal correlation modeling. HALO first introduces a hyperspherical VAE that constrains continuous latents to a fixed-radius hyperspherical shell, effectively stabilizing the numerical fluctuations of continuous latent representations. Secondly, we develop a masked autoregressive model that balances parallel decoding and temporal correlation learning, substantially reducing the number of inference steps required for generation and improving generation stability. Our extensive experiments demonstrate that HALO achieves state-of-the-art generation performance while offering significantly improved inference efficiency over existing advanced baselines.

Abstract

Time Series Generation

Explore similar work

HALO: Enhancing Time Series Generation via Hyperspherical Latents and Masked AutoregRessive Modeling

UPLOTS: A Unified Pretrained Language Model for Constrained Time-series Generation

FracTS: Hierarchical and Autoregressive Time Series Generation