cs.LGMar 3, 2026

Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series

Authors: Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa, Tomaso Poggio, Pierfrancesco Beneventano

Organizations: Massachusetts Institute of Technology, Cambridge, MA, USA · Princeton University, Princeton, NJ, USA.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale volatility forecasting for S&\&P 500 stocks, we show that different model-training-pipeline pairs with identical test loss learn qualitatively different functions. Across architectures, predictive accuracy remains unchanged, yet optimizer choice reshapes non-linear response profiles and temporal dependence differently. These divergences have material consequences for decisions: volatility-ranked portfolios trace a near-vertical Sharpe-turnover frontier, with nearly 3×3\times turnover dispersion at comparable Sharpe ratios. We conclude that in underspecified settings, optimization acts as a consequential source of inductive bias, thus model evaluation should extend beyond scalar loss to encompass functional and decision-level implications.

Figures & tables

Figure 1: LSTM Response Surface. Visualization of the impulse response R(k,δ)\mathcal{R}(k,\delta) . The LSTM (trained with Muon) exhibits a curved decision boundary, indicating distinct non-linear sensitivity to volatility shocks at specific lags.
Panel A: Linear Baselines
ModelNMSE  
OLS0.57510.5751  
LASSO0.57710.5771  
Panel B: Neural Models
ArchitectureAdamMuonSGD
MLP0.5752±0.00120.5752\pm 0.00120.5770±0.00020.5770\pm 0.00020.5781±0.00400.5781\pm 0.0040
Table 1: Out-of-Sample Performance (NMSE) . Comparison of generalization error across all Architecture-Optimizer pairs. We report the mean ±\pm standard deviation across 13 independent training runs. Lower is better.
Figure 2: Functional Divergence: Adaptive vs. SGD. Impulse response analysis of the CNN architecture at Lag t−1t-1 . The optimizer dictates the complexity of the learned function: Adam and Muon (Blue/Red) identify a complex non-linear dampening mechanism, whereas SGD (Green) reverts to a distinctively different function. All models achieve comparable predictive error, yet represent fundamentally different functional interpretations of the same data.
Figure 3: Functional Divergence Across Architectures. The difference surface D=y^Muon−y^AdamD=\hat{y}_{\text{Muon}}-\hat{y}_{\text{Adam}} plotted for LSTM (left), CNN (middle), and Transformer (right). All three architectures produce complex, non-flat difference landscapes, confirming that Adam and Muon settle into fundamentally different local minima regardless of the specific architecture used.
Figure 4: Mechanism of Divergence (SHAP Values). Feature attribution analysis comparing Adam and Muon optimizers across lags t−1t-1 (Feature 99) to t−100t-100 (Feature 0).
Figure 5: Edge of Stability Trace under SGD. Evolution of the maximum Hessian eigenvalue λmax\lambda_{max} during SGD training for the MLP architecture relative to the stability threshold 2/η2/\eta . Sharpness rises until it equilibrates at the edge of instability. Financial neural networks exhibit EoS behavior where sharpness tracks the stability limit.
Figure 6: Optimizer Intervention. Response profiles when swapping optimizers after convergence. (Blue) Starting from a complex Adam model, SGD forces a collapse to a simpler solution. (Orange) Starting from a piecewise linear SGD model, Adam recovers the non-linear complexity.
Figure 7: Early Intervention (Warm Start). The difference surface between the Early-Swap model (Adam →\to SGD at step 500) and the Baseline SGD model. The surface is effectively flat ( D≈0D\approx 0 ), confirming that even with an early "Adam push," the model gravitates entirely to the simpler SGD attractor.
Figure 8: Sharpe–Turnover Frontier across Volatility Quintiles. Each point corresponds to a volatility model embedded in a volatility-managed portfolio rule. Panel (a) reports the low-volatility portfolio (Q1) and panel (b) the high-volatility portfolio (Q5). Across both quintiles, Sharpe ratios are broadly similar across models, while induced turnover varies substantially with architecture and optimizer choice, implying that metrically equivalent predictors can yield markedly different trading behavior.
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

VariableDescription
PERMNOCRSP permanent security identifier
dateTrading day (NYSE calendar)
retDaily return (CRSP; split/dividend adjusted)
open, high, low, closeDaily O/H/L/C prices
volDaily share volume
cfacprPrice adjustment factor for corporate actions
Appendix
Table A1 : Core variables in the analysis dataset
Figure A1 : Distribution of observation counts by GICS sector.
Figure A2 : Yearly log⁡10\log_{10} distribution of average market capitalization per stock.
Figure A3 : Upper-triangular correlation matrices of daily returns by year.
Figure A4 : Distribution of Test Error. Box plots showing the spread of NMSE across 13 independent runs for each Architecture-Optimizer pair. SGD exhibits heavier-tail variability with occasional outliers.
Figure A5 : Optimizer-Driven Feature Attribution (SHAP). Comparison of lag importance across architectures.
Figure A6 : Linear Baselines. Coefficients of OLS and LASSO regressions for 100 lagged squared volatility terms ( σt−992\sigma^{2}_{t-99} through σt2\sigma^{2}_{t} ). Note the sparsity of LASSO versus the dense allocation of OLS.
Figure A7 : Functional Divergence (Impulse Responses). Comparison of model sensitivity at t−1t-1 across optimizers.
Figure A8 : Response Surfaces for CNN Architecture. Comparing the learned decision boundaries across optimizers.
Figure A9 : Response Surfaces for LSTM Architecture. Comparing the learned decision boundaries across optimizers.
Figure A10 : Response Surfaces for MLP Architecture. Comparing the learned decision boundaries across optimizers.
Figure A11 : Response Surfaces for Transformer Architecture. Comparing the learned decision boundaries across optimizers.
Figure A12 : Edge of Stability Scaling Analysis. Evolution of the Hessian spectral norm ( λmax\lambda_{max} ) relative to the optimizer stability threshold ( 2/η2/\eta ). We observe a clear scaling law where increasing the dataset size NN delays the onset of the unstable, chaotic regime.
ModelAnn. Ret.Ann. Vol.SharpeMax DDTurnover
Q1: Low Volatility
OLS0.1200.1300.921-0.3970.203
LASSO0.1190.1300.917-0.4020.217
CNN (SGD)0.1170.1290.906-0.3930.069
CNN (Muon)0.1240.1290.959-0.3890.180
CNN (Adam)0.1170.1300.898-0.3980.193
Appendix
Table A2: Volatility-sorted portfolios. Performance and turnover of equal-weight portfolios formed on predicted volatility. Q1 denotes the lowest-volatility quintile; Q5 the highest.
Figure A13 : Rolling turnover of volatility rankings (Q1), averaged by optimizer. Turnover is computed for equal-weight daily volatility-quintile portfolios and shown as a rolling one-year average. Shaded regions denote major stress episodes.
Figure A14 : Rolling turnover by volatility quintile (Adam-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.
Figure A15 : Rolling turnover by volatility quintile (SGD-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.
Figure A16 : Rolling turnover by volatility quintile (Muon-trained models). Rolling six-month turnover for Q1/Q3/Q5 equal-weight portfolios formed by predicted volatility rankings.
More in this topic

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 327 papers

Explore similar work

CardsList
Mar 2, 2026

How do standard time series forecasting benchmarks translate to finance? A broad study of risk adjusted performance

Forecasting financial asset returns is challenging due to low signal-to-noise ratios, non-stationarity and heavy-tailed distributions. We benchmark 16 deep learning architectures for time series forecasting, including Transformers, State Space Models (SSMs), xLSTMs and recurrent hybrids, on a 15-year daily futures dataset using an end-to-end volatility-targeted Sharpe ratio optimisation task. Our results challenge prevailing assumptions in the literature. Several widely used 'state-of-the-art' architectures, including generic Transformers and SSMs, underperformed a simple AR(1) benchmark in a realistic financial forecasting task. The strongest results came from comparatively simple hybrid approaches using recurrent structures: a Variable Selection Network combined with an LSTM achieved the highest Sharpe ratio, while xLSTM showed the greatest robustness to transaction costs. These findings suggest that success on standard time-series benchmarks does not reliably transfer to finance, and that architectures tailored to the unique properties of asset returns may remain more effective than generic attention-based approaches for financial forecasting.

Adir Saly-Kaufmann, Kieran Wood, Jan-Peter Calliess +1

Machine Learning Research Group, Department of Engineering Science, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford

Financial Time SeriesTime Series ForecastingForecasting BenchmarksAlgorithmic TradingTime Series Representation Learning
AskPDF
May 16, 2026

Financially Guided Deep Portfolio Optimization

Portfolio optimization in real-world financial markets is notoriously difficult due to non-stationarity, noisy data, and high transaction costs. Standard predict-then-optimize methods first forecast returns and then solve for weights, compounding prediction errors and often failing under regime shifts. We propose an end-to-end framework that directly optimizes differentiable surrogates of key financial metrics (Sharpe ratio, Omega ratio, Conditional Value-at-Risk, and risk parity), allowing neural networks to learn portfolio weights via backpropagation. Our expanding-window walk-forward procedure, applied to 50 S&P 500 stocks from 2007 to 2023, incorporates realistic bid-ask spread costs and rebalances quarterly. On the challenging out-of-sample test period (2022-2023), the best model, an AttentionLSTM with the Omega-CVaR-RiskParity loss, achieves an annualized Sharpe of 0.29 and a total compounded return of +7.86%, while the S&P 500 delivers -4.52% total compounded return and an annualized Sharpe of -0.02. This outperforms the S&P 500 by 12.38 percentage points, while keeping tail risk (CVaR) nearly unchanged. The framework outperforms the equal-weight portfolio, S&P 500, and traditional methods (MVP, HRP, NCO, ERC), demonstrating that embedding financial objectives directly into model training yields robust, economically meaningful outperformance even in adverse market conditions.

Rahul Fernandes, Travis Desell

Department of Software Engineering Rochester Institute of Technology Rochester, New York, USA

Quantitative FinancePortfolio OptimizationDifferentiable OptimizationFinancial Time Series
AskPDF
Mar 31, 2026

Forecast collapse of transformer-based models under squared loss in financial time series

We study trajectory forecasting under squared loss for time series with weak conditional structure, using highly expressive prediction models. Building on the classical characterization of squared-loss risk minimization, we emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate, leading to trivial Bayes-optimal predictors (flat for prices and zero for returns in standard financial settings). In this regime, increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor. These fluctuations arise from the reuse of noise and result in increased prediction variance without any reduction in bias. This provides a process-level explanation for the degradation of Transformerbased forecasts on financial time series. We complement these theoretical results with numerical experiments on high-frequency EUR/USD exchange rate data, analyzing the distribution of trajectory-level forecasting errors. The results show that Transformer-based models yield larger errors than a simple linear benchmark on a large majority of forecasting windows, consistent with the variance-driven mechanism identified by the theory.

Pierre Andreoletti

IDP · Institut Denis Poisson, UMR C.N.R.S. 7013, France.

Time Series ForecastingTime Series TransformersFinancial Time Series
AskPDF