Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series

Federico Vittorio Cortesi; Giuseppe Iannone; Giulia Crippa; Tomaso Poggio; Pierfrancesco Beneventano

cs.LGMar 3, 2026

Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series

Authors: Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa, Tomaso Poggio, Pierfrancesco Beneventano

Organizations: Massachusetts Institute of Technology, Cambridge, MA, USA · Princeton University, Princeton, NJ, USA.

Abstract

Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale volatility forecasting for S $\&$ P 500 stocks, we show that different model-training-pipeline pairs with identical test loss learn qualitatively different functions. Across architectures, predictive accuracy remains unchanged, yet optimizer choice reshapes non-linear response profiles and temporal dependence differently. These divergences have material consequences for decisions: volatility-ranked portfolios trace a near-vertical Sharpe-turnover frontier, with nearly $3\times$ turnover dispersion at comparable Sharpe ratios. We conclude that in underspecified settings, optimization acts as a consequential source of inductive bias, thus model evaluation should extend beyond scalar loss to encompass functional and decision-level implications.

Figures & tables

Figure 1: LSTM Response Surface. Visualization of the impulse response $\mathcal{R}(k,\delta)$ . The LSTM (trained with Muon) exhibits a curved decision boundary, indicating distinct non-linear sensitivity to volatility shocks at specific lags.

Panel A: Linear Baselines
Model	NMSE
OLS	$0.5751$
LASSO	$0.5771$
Panel B: Neural Models
Architecture	Adam	Muon	SGD
MLP	$0.5752\pm 0.0012$	$0.5770\pm 0.0002$	$0.5781\pm 0.0040$

Table 1: Out-of-Sample Performance (NMSE) . Comparison of generalization error across all Architecture-Optimizer pairs. We report the mean

\pm

standard deviation across 13 independent training runs. Lower is better.

Figure 2: Functional Divergence: Adaptive vs. SGD. Impulse response analysis of the CNN architecture at Lag $t-1$ . The optimizer dictates the complexity of the learned function: Adam and Muon (Blue/Red) identify a complex non-linear dampening mechanism, whereas SGD (Green) reverts to a distinctively different function. All models achieve comparable predictive error, yet represent fundamentally different functional interpretations of the same data.

Panel of Figure 3 — Figure 3: Functional Divergence Across Architectures. The difference surface $D=\hat{y}_{\text{Muon}}-\hat{y}_{\text{Adam}}$ plotted for LSTM (left), CNN (middle), and Transformer (right). All three architectures produce complex, non-flat difference landscapes, confirming that Adam and Muon settle into fundamentally different local minima regardless of the specific architecture used.

Panel of Figure 4 — Figure 4: Mechanism of Divergence (SHAP Values). Feature attribution analysis comparing Adam and Muon optimizers across lags $t-1$ (Feature 99) to $t-100$ (Feature 0).

Figure 5: Edge of Stability Trace under SGD. Evolution of the maximum Hessian eigenvalue $\lambda_{max}$ during SGD training for the MLP architecture relative to the stability threshold $2/\eta$ . Sharpness rises until it equilibrates at the edge of instability. Financial neural networks exhibit EoS behavior where sharpness tracks the stability limit.

Figure 6: Optimizer Intervention. Response profiles when swapping optimizers after convergence. (Blue) Starting from a complex Adam model, SGD forces a collapse to a simpler solution. (Orange) Starting from a piecewise linear SGD model, Adam recovers the non-linear complexity.

Figure 7: Early Intervention (Warm Start). The difference surface between the Early-Swap model (Adam $\to$ SGD at step 500) and the Baseline SGD model. The surface is effectively flat ( $D\approx 0$ ), confirming that even with an early "Adam push," the model gravitates entirely to the simpler SGD attractor.

Figure 8: Sharpe–Turnover Frontier across Volatility Quintiles. Each point corresponds to a volatility model embedded in a volatility-managed portfolio rule. Panel (a) reports the low-volatility portfolio (Q1) and panel (b) the high-volatility portfolio (Q5). Across both quintiles, Sharpe ratios are broadly similar across models, while induced turnover varies substantially with architecture and optimizer choice, implying that metrically equivalent predictors can yield markedly different trading behavior.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Variable	Description
PERMNO	CRSP permanent security identifier
date	Trading day (NYSE calendar)
ret	Daily return (CRSP; split/dividend adjusted)
open, high, low, close	Daily O/H/L/C prices
vol	Daily share volume
cfacpr	Price adjustment factor for corporate actions

Appendix

Table A1 : Core variables in the analysis dataset

Figure A1 : Distribution of observation counts by GICS sector.

Figure A2 : Yearly $\log_{10}$ distribution of average market capitalization per stock.

Figure A3 : Upper-triangular correlation matrices of daily returns by year.

Figure A4 : Distribution of Test Error. Box plots showing the spread of NMSE across 13 independent runs for each Architecture-Optimizer pair. SGD exhibits heavier-tail variability with occasional outliers.

Panel of Figure 15 — Figure A5 : Optimizer-Driven Feature Attribution (SHAP). Comparison of lag importance across architectures.

Figure A6 : Linear Baselines. Coefficients of OLS and LASSO regressions for 100 lagged squared volatility terms ( $\sigma^{2}_{t-99}$ through $\sigma^{2}_{t}$ ). Note the sparsity of LASSO versus the dense allocation of OLS.

Panel of Figure 17 — Figure A7 : Functional Divergence (Impulse Responses). Comparison of model sensitivity at $t-1$ across optimizers.

Panel of Figure 18 — Figure A8 : Response Surfaces for CNN Architecture. Comparing the learned decision boundaries across optimizers.

Panel of Figure 19 — Figure A9 : Response Surfaces for LSTM Architecture. Comparing the learned decision boundaries across optimizers.

Panel of Figure 20 — Figure A10 : Response Surfaces for MLP Architecture. Comparing the learned decision boundaries across optimizers.

Panel of Figure 21 — Figure A11 : Response Surfaces for Transformer Architecture. Comparing the learned decision boundaries across optimizers.

Panel of Figure 22 — Figure A12 : Edge of Stability Scaling Analysis. Evolution of the Hessian spectral norm ( $\lambda_{max}$ ) relative to the optimizer stability threshold ( $2/\eta$ ). We observe a clear scaling law where increasing the dataset size $N$ delays the onset of the unstable, chaotic regime.

Q1: Low Volatility
Model	Ann. Ret.	Ann. Vol.	Sharpe	Max DD	Turnover
OLS	0.120	0.130	0.921	-0.397	0.203
LASSO	0.119	0.130	0.917	-0.402	0.217
CNN (SGD)	0.117	0.129	0.906	-0.393	0.069
CNN (Muon)	0.124	0.129	0.959	-0.389	0.180
CNN (Adam)	0.117	0.130	0.898	-0.398	0.193

Appendix

Table A2: Volatility-sorted portfolios. Performance and turnover of equal-weight portfolios formed on predicted volatility. Q1 denotes the lowest-volatility quintile; Q5 the highest.

Figure A13 : Rolling turnover of volatility rankings (Q1), averaged by optimizer. Turnover is computed for equal-weight daily volatility-quintile portfolios and shown as a rolling one-year average. Shaded regions denote major stress episodes.

Figure A14 : Rolling turnover by volatility quintile (Adam-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.

Figure A15 : Rolling turnover by volatility quintile (SGD-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.

Figure A16 : Rolling turnover by volatility quintile (Muon-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 331 papers

How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance
Mar 2, 2026Adir Saly-Kaufmann, Kieran Wood, Jan-Peter Calliess +1Financial Time Series Time Series Forecasting
Machine Learning Research Group, Department of Engineering Science, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford
Financially Guided Deep Portfolio Optimization
May 16, 2026Rahul Fernandes, Travis DesellQuantitative Finance Portfolio Optimization
Department of Software Engineering Rochester Institute of Technology Rochester, New York, USA
Forecast collapse of transformer-based models under squared loss in financial time series
Mar 31, 2026Pierre AndreolettiTime Series Forecasting Time Series Transformers
IDP · Institut Denis Poisson, UMR C.N.R.S. 7013, France.

Mar 2, 2026J/K move · Enter open · S save

How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

Adir Saly-Kaufmann, Kieran Wood, Jan-Peter Calliess, Stefan Zohren

Machine Learning Research Group, Department of Engineering Science, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford

Financial Time Series Time Series Forecasting Forecasting Benchmarks Algorithmic Trading Time Series Representation Learning

Open paper AskPDF arXiv

Forecasting financial asset returns is challenging due to low signal-to-noise ratios, non-stationarity and heavy-tailed distributions. We benchmark 16 deep learning architectures for time series forecasting, including Transformers, State Space Models (SSMs), xLSTMs and recurrent hybrids, on a 15-year daily futures dataset using an end-to-end volatility-targeted Sharpe ratio optimisation task. Our results challenge prevailing assumptions in the literature. Several widely used 'state-of-the-art' architectures, including generic Transformers and SSMs, underperformed a simple AR(1) benchmark in a realistic financial forecasting task. The strongest results came from comparatively simple hybrid approaches using recurrent structures: a Variable Selection Network combined with an LSTM achieved the highest Sharpe ratio, while xLSTM showed the greatest robustness to transaction costs. These findings suggest that success on standard time-series benchmarks does not reliably transfer to finance, and that architectures tailored to the unique properties of asset returns may remain more effective than generic attention-based approaches for financial forecasting.

Abstract

Neural Network Optimization

Explore similar work

How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

Financially Guided Deep Portfolio Optimization

Forecast collapse of transformer-based models under squared loss in financial time series