How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

Adir Saly-Kaufmann; Kieran Wood; Jan-Peter Calliess; Stefan Zohren

q-fin.TRMar 2, 2026

How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

Authors: Adir Saly-Kaufmann, Kieran Wood, Jan-Peter Calliess, Stefan Zohren

Organizations: Machine Learning Research Group, Department of Engineering Science, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford

Abstract

Forecasting financial asset returns is challenging due to low signal-to-noise ratios, non-stationarity and heavy-tailed distributions. We benchmark 16 deep learning architectures for time series forecasting, including Transformers, State Space Models (SSMs), xLSTMs and recurrent hybrids, on a 15-year daily futures dataset using an end-to-end volatility-targeted Sharpe ratio optimisation task. Our results challenge prevailing assumptions in the literature. Several widely used 'state-of-the-art' architectures, including generic Transformers and SSMs, underperformed a simple AR(1) benchmark in a realistic financial forecasting task. The strongest results came from comparatively simple hybrid approaches using recurrent structures: a Variable Selection Network combined with an LSTM achieved the highest Sharpe ratio, while xLSTM showed the greatest robustness to transaction costs. These findings suggest that success on standard time-series benchmarks does not reliably transfer to finance, and that architectures tailored to the unique properties of asset returns may remain more effective than generic attention-based approaches for financial forecasting.

Figures & tables

Figure 1: End-to-end portfolio optimization pipeline: Statistical and technical indicators are extracted from historical close prices, serving as the predictive model’s inputs. The model outputs are transformed into portfolio weights via a linear projection followed by a hyperbolic tangent activation. Training is performed by minimizing the negative Sharpe Ratio.

Strategy	2010-2024	2015-2024	2010-2014	2015-2019	2020-2024
AR1x	0.77	0.71	0.89	0.06	1.35
AR $n$ x	0.63	0.55	0.80	-0.01	1.11
DLinear	0.64	0.64	0.65	0.00	1.28
NLinear	0.52	0.48	0.61	0.20	0.77
Mamba	0.36	0.28	0.53	-0.01	0.56
Mamba2	0.80	0.87	0.68	0.19	1.54

Table 1: Out-of-sample Sharpe Ratio by subperiod. Annual Sharpe Ratio in Table 12

Figure 2: Performance comparison across models 10% volatility-rescaled gross PnL.

Figure 3: Pairwise Significance Test based on p-value

Figure 4: Pairwise Significance Test based on t-statistics

Model	CAGR	Ann. Ret.	$t$ (HAC)	Hit	Turnover	xGMV	Info. Ratio	$t$ (HAC) v Passive	Corr. v Passive
Passive	0.0435	0.048	1.65	0.531	–	–	–	–	–
AR1x	0.0813	0.077	3.12	0.537	353.64	90.421	-0.0086	-0.0305	0.3533
AR $n$ x	0.0647	0.063	2.52	0.538	280.66	69.525	-0.0830	-0.3011	0.4325
DLinear	0.0750	0.064	2.87	0.539	278.41	75.282	0.0141	0.0501	0.2612
NLinear	0.0541	0.052	2.10	0.543	178.87	46.786	-0.0812	-0.2896	0.4806
Mamba	0.0266	0.036	1.15	0.523	128.99	34.673	-0.2439	-0.8925	0.2289

Table 2: 2010–2025 Gross return performance, statistical significance, and passive-relative diagnostics (volatility-targeted at 10%).

Model	Max DD	Calmar	Worst 3m Sharpe	Min Ann. Sharpe	CVaR 5%
Passive	-0.308	0.14	-6.11	-1.53	0.0144
AR1x	-0.167	0.49	-3.92	-0.59	0.0147
AR $n$ x	-0.206	0.31	-4.52	-0.90	0.0148
DLinear	-0.180	0.42	-4.93	-0.94	0.0149
NLinear	-0.227	0.24	-4.54	-1.19	0.0151
Mamba	-0.247	0.11	-4.76	-0.54	0.0152

Table 3: Downside risk and tail behavior (gross returns).

Panel of Figure 5 — Figure 5: PnL for different transaction costs. Sharpe ratio in brackets.

Return Performance						Risk and Downside Metrics
Model	CAGR	Ann. Ret.	Sharpe	$t$ (HAC)	Hit	Model	Max DD	Calmar	Worst 3m	Min Ann.	CVaR 5%
AR1x	0.084	0.085	0.854	3.208	0.537	AR1x	-0.165	0.506	-4.068	-0.524	0.0147
VLSTM	0.264	0.240	2.397	8.857	0.589	VLSTM	-0.159	1.664	-3.730	-0.250	0.0139
TFT	0.251	0.229	2.290	8.575	0.586	TFT	-0.210	1.196	-3.499	-0.175	0.0141
VxLSTM	0.203	0.190	1.898	7.236	0.578	VxLSTM	-0.108	1.878	-3.718	-1.064	0.0138

Table 4: Reduced-seed benchmark (25 runs, top 5 seeds selected).

Model	Number of Parameters	Runtime Complexity	Memory Complexity
TFT	$O(CH^{2}+\ell H^{2})$	$O(L^{2}H+LCH^{2})$	$O(L^{2}+LH)$
VLSTM	$O(CH^{2}+H^{2})$	$O(LCH^{2}+LH^{2})$	$O(LH)$
Mamba	$O(\ell H^{2})$	$O(LH^{2})$	$O(LH)$
Mamba2	$O(\ell H^{2})$	$O(LH^{2})$	$O(LH)$
VSN + Mamba2	$O(CH^{2}+\ell H^{2})$	$O(LCH^{2}+LH^{2})$	$O(LH)$
AR1x	$\mathbf{O(1)}$	$\mathbf{O(L)}$	$\mathbf{O(1)}$

Table 5: Comparison of parameter count, runtime complexity, and memory complexity for different time-series forecasting models. The emphasis on order of results is as follows: best , second_best ,

{\color[rgb]{0.5,0.5,0.5}third\_best}

. The forecasting horizon is fixed to one.

L

=sequence length,

C

=number of input features,

H

=hidden dimension,

N

L-S

=number of patches (for shifted versions adjusted for each time step),

P

=patch length,

S

=stride,

K

=convolution kernel size,

\ell

=number of layers,

M

=number of attention heads.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Distribution of daily returns. To make the central mass visible, the figure focuses on the bulk of the distribution; tail behavior is examined separately.

Figure 7: Distribution of daily realized volatility (log scale). Volatility exhibits strong right skewness and a long upper tail.

Figure 8: Left: Quantile–quantile plot against the normal distribution. Right: Tail behavior of daily returns. The figures indicate substantial deviations from Gaussianity and heavy-tailed return dynamics.

Figure 9: Distribution of Momentum Features.

Figure 10: Induced Exposure by Volatility Targeting.

Model	Params	SR	CAGR	Max DD	t (HAC)	Calmar	Turnover	Info. Ratio	Hit
VLSTM	1,142,963	2.40	0.2632	-0.229	8.81	1.15	966.86	0.854	0.588
PsLSTM	1,963,841	1.78	0.1923	-0.127	7.17	1.51	884.61	0.389	0.560
PatchTST	1,139,739	0.75	0.0855	-0.184	3.31	0.47	613.85	-0.199	0.538
LPatchTST	1,714,433	0.56	0.0393	-0.325	1.53	0.12	105.88	-0.176	0.532
Mamba2	1,967,714	0.46	0.0376	-0.277	1.55	0.14	138.83	-0.161	0.532
VMamba2	1,096,304	1.02	0.1617	-0.116	5.85	1.39	549.06	0.505	0.564

Appendix

Table 8: Performance under a controlled capacity budget (1–2 M trainable parameters).

Ticker	Group	Description	Ticker	Group	Description	Ticker	Group	Description
AN	FX	Australian dollar comp	BC	Energy	Brent crude oil	BG	Energy	Brent gasoil
BN	FX	British pound comp	CA	Index	CAC 40 index	CADJPY	FX	CAD JPY cross
CB	Bond	Canada 10Y bond	CC	Comdty	Cocoa	CN	FX	Canadian dollar comp
CR	Comdty	CRB index	CT	Comdty	Cotton No.2	DA	Comdty	Milk III
DT	Bond	Euro Bund	DX	FX	US dollar index	EN	Index	Nasdaq mini
ER	Index	Russell 2000 mini	ES	Index	S&P 500 mini	FB	Bond	US 5Y note comp

Appendix

Table 9: Asset universe and classification.

Figure 11: VLSTM PnL per asset - bond Futures

Figure 12: VLSTM PnL per asset - Commodities Futures

Figure 13: VLSTM PnL per asset - Index Futures

Figure 14: VLSTM PnL per asset - Energy Futures

Figure 15: VLSTM PnL per asset - FX Futures

Figure 16: VLSTM box and whisker annual Sharpe Ratio per asset - Bond Futures

Figure 17: VLSTM box and whisker annual Sharpe Ratio per asset - Commodities Futures

Figure 18: VLSTM box and whisker annual Sharpe Ratio per asset - Index Futures

Figure 19: VLSTM box and whisker annual Sharpe Ratio per asset - Energy Futures

Figure 20: VLSTM box and whisker annual Sharpe Ratio per asset - FX Futures

Ticker	Gross (ann.)	Turnover (ann.)	$c^{*}$ (bps)	Ticker	Gross (ann.)	Turnover (ann.)	$c^{*}$ (bps)
LB	0.2485	80.58	30.84	ZO	0.2027	84.01	24.13
DA	0.2529	114.78	22.04	ZN	0.0618	45.04	13.73
JO	0.0986	74.19	13.29	EN	0.1003	86.69	11.57
KC	0.0738	63.92	11.54	ZR	0.1339	137.44	9.75
BC	0.0512	61.26	8.36	ZF	0.1279	157.12	8.14
NR	0.1086	137.78	7.88	NK	0.0706	89.77	7.87

Appendix

Table 10: VLSTM: Annualised Volatility-Rescaled Performance and Breakeven Transaction Costs (bps)

Figure 21: xLSTM PnL per asset - bond Futures

Figure 22: xLSTM PnL per asset - Commodities Futures

Figure 23: xLSTM PnL per asset - Index Futures

Figure 24: xLSTM PnL per asset - Energy Futures

Figure 25: xLSTM PnL per asset - FX Futures

Figure 26: xLSTM box and whisker annual Sharpe Ratio per asset - Bond Futures

Figure 27: xLSTM box and whisker annual Sharpe Ratio per asset - Commodities Futures

Figure 28: xLSTM box and whisker annual Sharpe Ratio per asset - Index Futures

Figure 29: xLSTM box and whisker annual Sharpe Ratio per asset - Energy Futures

Figure 30: xLSTM box and whisker annual Sharpe Ratio per asset - FX Futures

Ticker	Gross (ann.)	Turnover (ann.)	$c^{*}$ (bps)	Ticker	Gross (ann.)	Turnover (ann.)	$c^{*}$ (bps)
LB	0.2338	68.93	33.92	DA	0.2200	83.29	26.41
ZO	0.2157	98.88	21.81	ES	0.0501	26.01	19.25
ZN	0.0544	29.02	18.76	CC	0.0419	24.23	17.31
EN	0.0572	38.19	14.97	JO	0.1375	96.18	14.30
ZR	0.1818	132.86	13.68	NR	0.1489	135.22	11.01
LX	0.0430	40.14	10.72	BG	0.0443	51.13	8.66

Appendix

Table 11: xLSTM: Annualised Volatility-Rescaled Performance and Breakeven Transaction Costs (bps)

Strategy	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021	2022	2023	2024
AR1x	1.45	-0.07	-0.11	0.23	2.97	-0.02	0.05	0.67	-0.59	0.18	3.02	0.63	1.37	1.45	0.30
AR $n$ x	1.60	0.06	-0.25	0.06	2.51	0.22	-0.17	0.30	-0.90	0.48	2.07	0.90	1.64	1.11	-0.16
DLinear	1.21	-0.11	-0.43	-0.29	2.89	0.33	-0.19	0.42	-0.28	-0.27	3.22	0.35	1.45	2.30	-0.94
NLinear	1.25	0.25	-0.85	-0.18	2.58	0.10	-0.08	1.13	-0.94	0.77	2.74	1.29	0.12	0.87	-1.19
LSTM	3.58	0.29	2.47	0.74	1.81	2.09	2.19	-1.19	1.72	3.20	-1.51	1.12	3.44	1.22	1.06
VLSTM	3.96	1.48	3.92	1.13	3.03	1.90	4.03	1.06	1.33	4.74	0.10	2.81	3.82	2.76	-0.10

Appendix

Table 12: Annual Sharpe Ratios by Strategy (2010–2024)

Financial Time Series

Financial time series studies methods for modeling market and economic data to forecast returns, volatility, risk, and changing regimes.

See all 95 papers

A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments
Feb 27, 2026Nabeel Ahmad SaiddTime Series Forecasting Forecasting Benchmarks
Dr. A.P.J. Abdul Kalam Technical University (AKTU)
Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series
Mar 3, 2026Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa +2Neural Network Optimization Financial Time Series
Massachusetts Institute of Technology, Cambridge, MA, USA · Princeton University, Princeton, NJ, USA.
FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance
Jun 2, 2026Jiaze Sun, Kelvin J. L. Koa, Ruiyang Ni +3Financial Forecasting Time Series Forecasting
National University of Singapore Singapore · National University of Singapore Asian Institute of Digital Finance Singapore · Nanyang Technological University Singapore

Feb 27, 2026J/K move · Enter open · S save

A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments

Nabeel Ahmad Saidd

Dr. A.P.J. Abdul Kalam Technical University (AKTU)

Time Series Forecasting Forecasting Benchmarks

Open paper AskPDF arXiv

Multi-horizon price forecasting is central to portfolio allocation, risk management, and algorithmic trading, yet deep learning architectures have proliferated faster than rigorous financial benchmarks can evaluate them. This study provides a controlled comparison of nine architectures (Autoformer, DLinear, iTransformer, LSTM, ModernTCN, N-HiTS, PatchTST, TimesNet, and TimeXer) spanning Transformer, MLP, CNN, and RNN families across cryptocurrency, forex, and equity index markets at 4-hour and 24-hour horizons. A total of 918 experiments were conducted under a strict five-stage protocol including fixed-seed Bayesian hyperparameter optimization, configuration freezing per asset class, multi-seed retraining, uncertainty aggregation, and statistical validation. ModernTCN achieves the best mean rank (1.333) with a 75 percent first-place rate, followed by PatchTST (2.000). Results reveal a clear three-tier ranking structure and show that architecture explains nearly all performance variance, while seed randomness is negligible. Rankings remain stable across horizons despite 2 to 2.5 times error amplification. Directional accuracy remains near 50 percent across all configurations, indicating that MSE-trained models lack directional skill at hourly resolution. The findings highlight the importance of architectural inductive bias over raw parameter count and provide reproducible guidance for multi-step financial forecasting.

Abstract

Financial Time Series

Explore similar work

A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments

Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series

FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance