Stock markets exhibit regime-dependent behavior where prediction models optimized for stable conditions often fail during volatile periods. Existing approaches typically treat all market states uniformly or require manual regime labeling, which is expensive and quickly becomes stale as market dynamics evolve. This paper introduces an adaptive prediction framework that adaptively identifies deviations from normal market conditions and routes data through specialized prediction pathways. The architecture consists of three components: (1) an autoencoder trained on normal market conditions that identifies anomalous regimes through reconstruction error, (2) dual node transformer networks specialized for stable and event-driven market conditions respectively, and (3) a Soft Actor-Critic reinforcement learning controller that adaptively tunes the regime detection threshold and pathway blending weights based on prediction performance feedback. The reinforcement learning component enables the system to learn adaptive regime boundaries, defining anomalies as market states where standard prediction approaches fail. Experiments on 20 S&P 500 stocks spanning 1982 to 2025 demonstrate that the proposed framework achieves 0.68% mean absolute percentage error (MAPE) for one-day predictions without the reinforcement controller and 0.59% MAPE with the full adaptive system, compared to 0.80% for the baseline integrated node transformer. Directional accuracy reaches 72% with the complete framework. The system maintains robust performance during high-volatility periods, with MAPE below 0.85% when baseline models exceed 1.5%. Ablation studies confirm that each component contributes meaningfully: autoencoder routing accounts for 36% relative MAPE degradation upon removal, followed by the SAC controller at 15% and the dual-path architecture at 7%.
Figures & tables
Fig. 1: System architecture overview. Market features xt enter the autoencoder, which produces reconstruction error et (shown on arrow). The router directs data to normal or event node transformer pathways based on whether et exceeds the learned threshold τ . Each pathway produces a prediction ( yt+hN , yt+hE ), and adaptive blending combines them into the final forecast y^t+h . The SAC controller observes evaluation metrics and adjusts both τ and α (dashed blue arrows) to optimize forecasting accuracy.
Fig. 2: Feature engineering pipeline. Raw OHLCV data is processed through technical indicator computations (SMA, EMA, RSI, MACD, volatility). All features undergo expanding-window z-score normalization to prevent look-ahead bias, producing prediction features and router-specific features.
Fig. 3: Autoencoder architecture for regime detection. The encoder compresses the input feature vector through two hidden layers (64, 32 units) to a latent representation zt of dimension dz=32 . The decoder reconstructs the input through symmetric layers. Reconstruction error et serves as the anomaly score for regime classification.
Fig. 4: Dual node transformer architecture. The router directs data based on reconstruction error. The normal pathway (left, orange) processes typical market conditions with base features. The event pathway (right, blue) augments inputs with event context features ct . Both pathways follow the same architectural design (layer count, attention heads, model dimension) but maintain independently trained weights and differ in input dimensionality, as the event pathway accepts additional context features. Outputs are blended with adaptive weight α .
Fig. 5: Graph representation of stock relationships (representative subset of 11 stocks shown for clarity; full graph contains all 20 stocks). Nodes represent individual stocks, colored by sector. Solid edges indicate same-sector connections with higher learned weights (annotated values show correlation-based initialization from training data). Dashed edges represent weaker cross-sector correlations that are learned during training.
Fig. 6: Single transformer layer architecture. Input passes through multi-head self-attention, residual connection with layer normalization, feed-forward network, and another residual connection with normalization. The architecture stacks 6 such layers.
Fig. 7: Soft Actor-Critic network architecture. The actor network (left, orange) maps states to a Gaussian policy over actions [Δτ,Δα] . Twin critic networks (right, green/cyan) estimate Q-values; the minimum is used to prevent overestimation. All networks use two hidden layers with 256 units and ReLU activation.
Fig. 8: Multi-stage training pipeline. Stage 1 trains the autoencoder on stable market data. Stage 2 trains both node transformers on their respective data subsets. Stage 3 trains the SAC controller with frozen prediction components. Stage 4 performs end-to-end fine-tuning with all weights unfrozen.
Fig. 9: Threshold τ produced by the frozen SAC policy over the test period. Dips correspond to volatile periods (COVID crash around index 35, 2022 market decline around index 75) where elevated reconstruction errors cause the fixed policy to output lower threshold values, routing more data through the event pathway. The dashed red line shows the static threshold ( τ=e95 ) used in the No SAC ablation variant. All policy weights are frozen after training; the trajectory reflects state-dependent outputs from a fixed function, not online learning.
Comparison
t -statistic
p -value
Cohen’s d
AE-NF+SAC vs Integrated NF-BERT
−5.82
<0.0001
0.46
AE-NF+SAC vs AE-NF (no SAC)
−3.45
0.0006
0.27
AE-NF+SAC vs LSTM
−7.21
<0.0001
0.57
TABLE IX: Statistical Significance ( n=1,580 Test Days)
Figure 20Figure 21Figure 22
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Model
Hyperparameter
Search Range
Selected
ARIMA
AR order ( p )
{0,1,2,3,4,5}
Per stock (AIC)
Differencing ( d )
{0,1,2}
Per stock (AIC)
MA order ( q )
{0,1,2,3,4,5}
Per stock (AIC)
Selection criterion
AIC, BIC
AIC
VAR
Lag order
{1,2,3,…,10}
3
Trend
{none, constant, both}
constant
Appendix
TABLE X: Hyperparameter Search Ranges and Selected Values for Baseline Models
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Adaptive Financial Transformer (AFT) is proposed for stock return prediction under non-stationary financial markets. The model incorporates a Market Regime Encoder, an Adaptive Gate Network, and an Adaptive Financial Context module to dynamically bias self-attention based on semantic relationships between financial indicators. Unlike conventional Transformer architectures that treat all input features uniformly, the proposed approach groups 95 engineered financial features into 11 semantic categories and adapts attention according to latent market regimes. The study also identifies and corrects sequence alignment and backtesting issues that can inflate reported trading performance, and introduces a financially-aware composite objective that jointly optimizes prediction error, directional accuracy, and non-overlapping Sharpe ratio. Extensive experiments compare the proposed architecture against classical machine learning models, recurrent neural networks, and Transformer baselines using chronological evaluation, five random seeds, ablation studies, hyperparameter optimization, explainability analysis, and multi-stock validation. Results demonstrate competitive predictive performance while reducing model complexity by 15.2% and improving parameter efficiency through feature selection, providing an interpretable Transformer architecture for financial time-series forecasting.
Dishan Sarkar
Department of Artificial Intelligence and Machine Learning Birla Institute of Technology, Mesra Ranchi, India
This research aims to leverage machine learning to improve stock price prediction and support informed investment decisions related to buying, selling, and holding assets. Specifically, this work investigates transformer-based models for stock prediction and examines the impact of pre-training strategies on forecasting performance. A transformer model was first pre-trained on the Toronto Stock Exchange Index (TSX) to predict intra-day return direction and subsequently fine-tuned on individual TSX stocks. The model was further adapted for return-value regression tasks. Performance was benchmarked against Long Short-Term Memory (LSTM) and XGBoost models. Pre-training on the market index improved the binary cross-entropy loss for individual stock prediction from 0.69 to 0.64. The fine-tuned transformer regression model achieved lower mean squared error than the benchmark models, although the ensemble and XGBoost models achieved higher average daily returns. In addition, a practical application was developed to deliver real-time stock predictions for trading support. Future work will focus on increasing transformer model capacity, incorporating broader global technical indicators, and filtering out stocks with low predictability.
Marie Soehl Coolsaet, Roberto Gallardo, Zhen Gao
Faculty of Engineering, McMaster University · Department* of Economics, Universidad Veracruzana, Mexico
Stock market prediction presents considerable challenges for investors, financial institutions, and policymakers operating in complex market environments characterized by noise, non-stationarity, and behavioral dynamics. Traditional forecasting methods, including fundamental analysis and technical indicators, often fail to capture the intricate patterns and cross-sectional dependencies inherent in financial markets. This paper presents an integrated framework combining a node transformer architecture with BERT-based sentiment analysis for stock price forecasting. The proposed model represents the stock market as a graph structure where individual stocks form nodes and edges capture relationships including sectoral affiliations, correlated price movements, and supply chain connections. A fine-tuned BERT model extracts sentiment information from social media posts and combines it with quantitative market features through attention-based fusion mechanisms. The node transformer processes historical market data while capturing both temporal evolution and cross-sectional dependencies among stocks. Experiments conducted on 20 S&P 500 stocks spanning January 1982 to March 2025 demonstrate that the integrated model achieves a mean absolute percentage error (MAPE) of 0.80% for one-day-ahead predictions, compared to 1.20% for ARIMA and 1.00% for LSTM. The inclusion of sentiment analysis reduces prediction error by 10% overall and 25% during earnings announcements, while the graph-based architecture contributes an additional 15% improvement by capturing inter-stock dependencies. Directional accuracy reaches 65% for one-day forecasts. Statistical validation through paired t-tests confirms the significance of these improvements (p < 0.05 for all comparisons). The model maintains lower error during high-volatility periods, achieving MAPE of 1.50% while baseline models range from 1.60% to 2.10%.
Mohammad Al Ridhawi, Mahtab Haj Ali, Hussein Al Osman
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada