Stock market prediction presents considerable challenges for investors, financial institutions, and policymakers operating in complex market environments characterized by noise, non-stationarity, and behavioral dynamics. Traditional forecasting methods, including fundamental analysis and technical indicators, often fail to capture the intricate patterns and cross-sectional dependencies inherent in financial markets. This paper presents an integrated framework combining a node transformer architecture with BERT-based sentiment analysis for stock price forecasting. The proposed model represents the stock market as a graph structure where individual stocks form nodes and edges capture relationships including sectoral affiliations, correlated price movements, and supply chain connections. A fine-tuned BERT model extracts sentiment information from social media posts and combines it with quantitative market features through attention-based fusion mechanisms. The node transformer processes historical market data while capturing both temporal evolution and cross-sectional dependencies among stocks. Experiments conducted on 20 S&P 500 stocks spanning January 1982 to March 2025 demonstrate that the integrated model achieves a mean absolute percentage error (MAPE) of 0.80% for one-day-ahead predictions, compared to 1.20% for ARIMA and 1.00% for LSTM. The inclusion of sentiment analysis reduces prediction error by 10% overall and 25% during earnings announcements, while the graph-based architecture contributes an additional 15% improvement by capturing inter-stock dependencies. Directional accuracy reaches 65% for one-day forecasts. Statistical validation through paired t-tests confirms the significance of these improvements (p < 0.05 for all comparisons). The model maintains lower error during high-volatility periods, achieving MAPE of 1.50% while baseline models range from 1.60% to 2.10%.
Figures & tables
Fig. 1: Feature engineering pipeline. Raw OHLCV data is processed through multiple technical indicator computations, normalized using z-score standardization, and concatenated into the final feature vector.
Cashtag
Company
Cashtag
Company
$AAPL
Apple
$KO
Coca-Cola
$MSFT
Microsoft
$CVX
Chevron
$JPM
JPMorgan Chase
$HD
Home Depot
$JNJ
Johnson & Johnson
$VZ
Verizon
$WMT
Walmart
$BA
Boeing
$XOM
ExxonMobil
$MCD
McDonald’s
TABLE I: Search tags used for CSS dataset collection.
Characteristic
FMD
CSS
Time period
1982-2025
2007-2025
Number of stocks
20
20
Trading days
≈ 10,300
≈ 4,100
Features per stock-day
17
3
Total observations
≈ 206,000
≈ 4.2M posts
TABLE II: Dataset summary statistics.
Fig. 2: System architecture. Price data, volume, and technical indicators are jointly processed through normalization, graph construction, and temporal encoding before the node transformer. Social media posts are processed through BERT and sentiment aggregation. Both streams combine through attention-based multimodal fusion.
Fig. 3: Single transformer layer architecture. Input passes through multi-head self-attention with residual connection and layer normalization, followed by a position-wise feed-forward network with another residual connection and normalization.
Fig. 4: BERT sentiment extraction pipeline. Raw social media posts are preprocessed, tokenized, encoded through BERT, pooled via [CLS] token, and classified to sentiment scores.
Fig. 5: Adaptive fusion mechanism. The weighting coefficient αt is computed from volatility and sentiment magnitude, then used to blend node transformer and sentiment-based predictions.
Parameter
Value
Node transformer layers
6
Attention heads
8
Model dimension
512
FFN dimension
2048
Dropout rate
0.1
Input sequence length
252 days
TABLE III: Model hyperparameters.
Class / Metric
Precision
Recall
F1-Score
Negative
0.83
0.80
0.81
Neutral
0.89
0.92
0.90
Positive
0.85
0.82
0.83
Macro Average
0.86
0.85
0.85
Overall Accuracy
0.87
Cohen’s κ
0.79
TABLE IV: BERT sentiment classification performance on the MSE test set.
Model
1-Day
5-Day
20-Day
Theil’s U
Naïve Random Walk
1.35 (0.09)
3.40 (0.18)
5.80 (0.28)
1.00
ARIMA
1.20 (0.08)
2.80 (0.15)
4.50 (0.22)
0.89
VAR
1.15 (0.07)
2.60 (0.14)
4.20 (0.20)
0.85
LSTM
1.00 (0.06)
2.30 (0.12)
3.80 (0.18)
0.74
Random Forest
1.10 (0.07)
2.50 (0.13)
4.00 (0.19)
0.81
SVR
1.05 (0.06)
2.40 (0.12)
3.90 (0.18)
0.78
TABLE V: MAPE (%) and Theil’s U across prediction horizons. Best results in bold. Standard errors in parentheses.
Model
DA (%)
Random Baseline
50.0
ARIMA
55.0 [53.2, 56.8]
VAR
56.0 [54.2, 57.8]
LSTM
58.0 [56.2, 59.8]
Random Forest
57.0 [55.2, 58.8]
XGBoost
60.0 [58.2, 61.8]
TABLE VI: Directional accuracy (%) for 1-day predictions with 95% confidence intervals.
Configuration
MAPE
Δ vs Full
Full Model
0.80
–
Without Sentiment
0.88
+10.0%
Without Graph Structure
0.92
+15.0%
Without Temporal Encoding
0.95
+18.8%
Without Feature Gating
0.84
+5.0%
Price Features Only
1.02
+27.5%
TABLE VII: Ablation study: MAPE (%) for 1-day predictions.
Model
Low VIX
Medium VIX
High VIX
ARIMA
1.00
1.30
2.10
LSTM
0.85
1.10
1.80
XGBoost
0.80
1.00
1.60
Proposed
0.70
0.90
1.50
TABLE VIII: MAPE (%) by market volatility regime.
Baseline
t -statistic
p -value
Cohen’s d
ARIMA
−4.25
<0.0001
0.34
LSTM
−3.12
0.0018
0.25
XGBoost
−2.45
0.0145
0.19
Simple Transformer
−2.18
0.0295
0.17
BERT + LSTM
−3.90
0.0001
0.31
TABLE IX: Paired t-test results ( n=1,580 test days). Negative t indicates proposed model superiority.
Baseline
DM-statistic
p -value
Naïve Random Walk
−5.82
<0.0001
ARIMA
−3.58
0.0002
LSTM
−2.67
0.0038
XGBoost
−2.08
0.0189
Simple Transformer
−1.84
0.0330
BERT + LSTM
−3.31
0.0005
TABLE X: Diebold-Mariano test results ( n=1,580 test days, MAE loss). Negative DM-statistic indicates proposed model superiority.
Statistic
Gross
Net (10 bps)
Annualized Return
22.8%
14.6%
Annualized Sharpe Ratio
1.35
0.84
Maximum Drawdown
12.1%
14.3%
Average Daily Turnover
32%
S&P 500 Buy-and-Hold Return
13.9%
S&P 500 Maximum Drawdown
23.9%
TABLE XI: Portfolio backtest statistics (January 2017 to March 2025).
Stock i
Stock j
Weight
Same Sector
Apple
Microsoft
0.86
Yes
ExxonMobil
Chevron
0.82
Yes
Microsoft
Salesforce
0.78
Yes
J&J
UnitedHealth
0.76
Yes
ExxonMobil
Caterpillar
0.67
No
Netflix
Chevron
0.19
No
TABLE XII: Top-5 and bottom-5 learned edge weights after training.
Predicting stock price movements during Earnings Announcements (EAs) is a significant challenge due to market noise and high-impact price discontinuities. In this study, we evaluate whether pre-announcement news sentiment, firm fundamentals, and recent market dynamics jointly predict the directional price movement of equities on EA days. We construct a multi-modal feature space combining 15 fundamental metrics, 3 price-based technical indicators and sentiment scores derived from financial news articles processed using FinBERT. We compare a Long Short-Term Memory (LSTM) network and a Transformer-based architecture against a logistic regression baseline, and further assess all models with and without sentiment features to quantify their incremental value. Our results indicate that while the LSTM demonstrates higher precision through a conservative safe-bet strategy, the Transformer model exhibits superior sensitivity in identifying volatile movements, achieving a higher macro F1-score, with ablation experiments showing a consistent benefit from incorporating news sentiment.
Accurate stock price forecasting has consistently remained a pivotal yet challenging FinTech task that underpins quantitative trading and investment decision making. Recent efforts have been dedicated to modeling various complex relationships among stocks in the stock market toward more reliable stock price forecasting.These methods depend heavily on strong static prior assumptions by modeling either temporal dependencies within individual stocks or spatial dependencies across different stocks based on predefined structures, while the complex market dynamics that drive stock price movements remain unexplored. To alleviate this issue, we propose a novel game-theoretic modeling method that captures heterogeneous investor interactions for stock price forecasting. The core idea is to embed game-theoretic mechanisms into the heterogeneous graph structure to finely model the dynamic strategic interactions among heterogeneous investors with respect to target stocks. Additionally, temporal positional encoding is adopted to reflect the differentiated influences of each game event at different time steps within the time window on future stock price movements. Leveraging heterogeneous graph networks, we proxy the intricate dynamics of the stock market through investor games and enable real-time information propagation and node updates among all nodes. Extensive experiments conducted on two real-world benchmark dataset demonstrate that our method effectively outperforms state-of-the-art stock price forecasting methods.
Yong Zhang, Xinxiao Wu, Yunde Jia +1
School of Computer Science, Beijing Institute of Technology · Faculty of Engineering, Shenzhen MSU-BIT University
This research aims to leverage machine learning to improve stock price prediction and support informed investment decisions related to buying, selling, and holding assets. Specifically, this work investigates transformer-based models for stock prediction and examines the impact of pre-training strategies on forecasting performance. A transformer model was first pre-trained on the Toronto Stock Exchange Index (TSX) to predict intra-day return direction and subsequently fine-tuned on individual TSX stocks. The model was further adapted for return-value regression tasks. Performance was benchmarked against Long Short-Term Memory (LSTM) and XGBoost models. Pre-training on the market index improved the binary cross-entropy loss for individual stock prediction from 0.69 to 0.64. The fine-tuned transformer regression model achieved lower mean squared error than the benchmark models, although the ensemble and XGBoost models achieved higher average daily returns. In addition, a practical application was developed to deliver real-time stock predictions for trading support. Future work will focus on increasing transformer model capacity, incorporating broader global technical indicators, and filtering out stocks with low predictability.
Marie Soehl Coolsaet, Roberto Gallardo, Zhen Gao
Faculty of Engineering, McMaster University · Department* of Economics, Universidad Veracruzana, Mexico