Stock Market Prediction Using Node Transformer Architecture Integrated with BERT Sentiment Analysis

Mohammad Al Ridhawi; Mahtab Haj Ali; Hussein Al Osman

cs.LGMar 6, 2026

Stock Market Prediction Using Node Transformer Architecture Integrated with BERT Sentiment Analysis

Authors: Mohammad Al Ridhawi, Mahtab Haj Ali, Hussein Al Osman

Organizations: School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

Abstract

Stock market prediction presents considerable challenges for investors, financial institutions, and policymakers operating in complex market environments characterized by noise, non-stationarity, and behavioral dynamics. Traditional forecasting methods, including fundamental analysis and technical indicators, often fail to capture the intricate patterns and cross-sectional dependencies inherent in financial markets. This paper presents an integrated framework combining a node transformer architecture with BERT-based sentiment analysis for stock price forecasting. The proposed model represents the stock market as a graph structure where individual stocks form nodes and edges capture relationships including sectoral affiliations, correlated price movements, and supply chain connections. A fine-tuned BERT model extracts sentiment information from social media posts and combines it with quantitative market features through attention-based fusion mechanisms. The node transformer processes historical market data while capturing both temporal evolution and cross-sectional dependencies among stocks. Experiments conducted on 20 S&P 500 stocks spanning January 1982 to March 2025 demonstrate that the integrated model achieves a mean absolute percentage error (MAPE) of 0.80% for one-day-ahead predictions, compared to 1.20% for ARIMA and 1.00% for LSTM. The inclusion of sentiment analysis reduces prediction error by 10% overall and 25% during earnings announcements, while the graph-based architecture contributes an additional 15% improvement by capturing inter-stock dependencies. Directional accuracy reaches 65% for one-day forecasts. Statistical validation through paired t-tests confirms the significance of these improvements (p < 0.05 for all comparisons). The model maintains lower error during high-volatility periods, achieving MAPE of 1.50% while baseline models range from 1.60% to 2.10%.

Figures & tables

Panel of Figure 1 — Fig. 1: Feature engineering pipeline. Raw OHLCV data is processed through multiple technical indicator computations, normalized using z-score standardization, and concatenated into the final feature vector.

Cashtag	Company	Cashtag	Company
$AAPL	Apple	$KO	Coca-Cola
$MSFT	Microsoft	$CVX	Chevron
$JPM	JPMorgan Chase	$HD	Home Depot
$JNJ	Johnson & Johnson	$VZ	Verizon
$WMT	Walmart	$BA	Boeing
$XOM	ExxonMobil	$MCD	McDonald’s

TABLE I: Search tags used for CSS dataset collection.

Characteristic	FMD	CSS
Time period	1982-2025	2007-2025
Number of stocks	20	20
Trading days	$\approx$ 10,300	$\approx$ 4,100
Features per stock-day	17	3
Total observations	$\approx$ 206,000	$\approx$ 4.2M posts

TABLE II: Dataset summary statistics.

Panel of Figure 4 — Fig. 2: System architecture. Price data, volume, and technical indicators are jointly processed through normalization, graph construction, and temporal encoding before the node transformer. Social media posts are processed through BERT and sentiment aggregation. Both streams combine through attention-based multimodal fusion.

Panel of Figure 5 — Fig. 3: Single transformer layer architecture. Input passes through multi-head self-attention with residual connection and layer normalization, followed by a position-wise feed-forward network with another residual connection and normalization.

Panel of Figure 6 — Fig. 4: BERT sentiment extraction pipeline. Raw social media posts are preprocessed, tokenized, encoded through BERT, pooled via [CLS] token, and classified to sentiment scores.

Panel of Figure 7 — Fig. 5: Adaptive fusion mechanism. The weighting coefficient $\alpha_{t}$ is computed from volatility and sentiment magnitude, then used to blend node transformer and sentiment-based predictions.

Parameter	Value
Node transformer layers	6
Attention heads	8
Model dimension	512
FFN dimension	2048
Dropout rate	0.1
Input sequence length	252 days

TABLE III: Model hyperparameters.

Class / Metric	Precision	Recall	F1-Score
Negative	0.83	0.80	0.81
Neutral	0.89	0.92	0.90
Positive	0.85	0.82	0.83
Macro Average	0.86	0.85	0.85
Overall Accuracy			0.87
Cohen’s $\kappa$			0.79

TABLE IV: BERT sentiment classification performance on the MSE test set.

Model	1-Day	5-Day	20-Day	Theil’s U
Naïve Random Walk	1.35 (0.09)	3.40 (0.18)	5.80 (0.28)	1.00
ARIMA	1.20 (0.08)	2.80 (0.15)	4.50 (0.22)	0.89
VAR	1.15 (0.07)	2.60 (0.14)	4.20 (0.20)	0.85
LSTM	1.00 (0.06)	2.30 (0.12)	3.80 (0.18)	0.74
Random Forest	1.10 (0.07)	2.50 (0.13)	4.00 (0.19)	0.81
SVR	1.05 (0.06)	2.40 (0.12)	3.90 (0.18)	0.78

TABLE V: MAPE (%) and Theil’s U across prediction horizons. Best results in bold. Standard errors in parentheses.

Model	DA (%)
Random Baseline	50.0
ARIMA	55.0 [53.2, 56.8]
VAR	56.0 [54.2, 57.8]
LSTM	58.0 [56.2, 59.8]
Random Forest	57.0 [55.2, 58.8]
XGBoost	60.0 [58.2, 61.8]

TABLE VI: Directional accuracy (%) for 1-day predictions with 95% confidence intervals.

Configuration	MAPE	$\Delta$ vs Full
Full Model	0.80	–
Without Sentiment	0.88	+10.0%
Without Graph Structure	0.92	+15.0%
Without Temporal Encoding	0.95	+18.8%
Without Feature Gating	0.84	+5.0%
Price Features Only	1.02	+27.5%

TABLE VII: Ablation study: MAPE (%) for 1-day predictions.

Model	Low VIX	Medium VIX	High VIX
ARIMA	1.00	1.30	2.10
LSTM	0.85	1.10	1.80
XGBoost	0.80	1.00	1.60
Proposed	0.70	0.90	1.50

TABLE VIII: MAPE (%) by market volatility regime.

Baseline	$t$ -statistic	$p$ -value	Cohen’s $d$
ARIMA	$-4.25$	$<0.0001$	0.34
LSTM	$-3.12$	0.0018	0.25
XGBoost	$-2.45$	0.0145	0.19
Simple Transformer	$-2.18$	0.0295	0.17
BERT + LSTM	$-3.90$	0.0001	0.31

TABLE IX: Paired t-test results (

n=1,580

test days). Negative

t

indicates proposed model superiority.

Baseline	DM-statistic	$p$ -value
Naïve Random Walk	$-5.82$	$<0.0001$
ARIMA	$-3.58$	0.0002
LSTM	$-2.67$	0.0038
XGBoost	$-2.08$	0.0189
Simple Transformer	$-1.84$	0.0330
BERT + LSTM	$-3.31$	0.0005

TABLE X: Diebold-Mariano test results (

n=1{,}580

test days, MAE loss). Negative DM-statistic indicates proposed model superiority.

Statistic	Gross	Net (10 bps)
Annualized Return	22.8%	14.6%
Annualized Sharpe Ratio	1.35	0.84
Maximum Drawdown	12.1%	14.3%
Average Daily Turnover	32%
S&P 500 Buy-and-Hold Return	13.9%
S&P 500 Maximum Drawdown	23.9%

TABLE XI: Portfolio backtest statistics (January 2017 to March 2025).

Stock $i$	Stock $j$	Weight	Same Sector
Apple	Microsoft	0.86	Yes
ExxonMobil	Chevron	0.82	Yes
Microsoft	Salesforce	0.78	Yes
J&J	UnitedHealth	0.76	Yes
ExxonMobil	Caterpillar	0.67	No
Netflix	Chevron	0.19	No

TABLE XII: Top-5 and bottom-5 learned edge weights after training.

Stock Price Forecasting

See all 20 papers

Predicting Stock Price Direction on Earnings Announcement Days using Multi-modal Deep Learning
May 25, 2026Manuel Noseda, Nathan Soldati, Marco PainaTime Series Classification Time Series Forecasting
ETH ZÜrich
Game-Theoretic Modeling of Heterogeneous Investor Interactions for Stock Price Forecasting
May 11, 2026Yong Zhang, Xinxiao Wu, Yunde Jia +1Quantitative Finance Stock Price Forecasting
School of Computer Science, Beijing Institute of Technology · Faculty of Engineering, Shenzhen MSU-BIT University
From Index to Equity: Pre-Training Transformers for Stock Return Prediction
May 12, 2026Marie Soehl Coolsaet, Roberto Gallardo, Zhen GaoTime Series Forecasting Transfer Learning
Faculty of Engineering, McMaster University · Department* of Economics, Universidad Veracruzana, Mexico

May 25, 2026J/K move · Enter open · S save

Predicting Stock Price Direction on Earnings Announcement Days using Multi-modal Deep Learning

Manuel Noseda, Nathan Soldati, Marco Paina

ETH ZÜrich

Time Series Classification Time Series Forecasting

Open paper AskPDF arXiv

Predicting stock price movements during Earnings Announcements (EAs) is a significant challenge due to market noise and high-impact price discontinuities. In this study, we evaluate whether pre-announcement news sentiment, firm fundamentals, and recent market dynamics jointly predict the directional price movement of equities on EA days. We construct a multi-modal feature space combining 15 fundamental metrics, 3 price-based technical indicators and sentiment scores derived from financial news articles processed using FinBERT. We compare a Long Short-Term Memory (LSTM) network and a Transformer-based architecture against a logistic regression baseline, and further assess all models with and without sentiment features to quantify their incremental value. Our results indicate that while the LSTM demonstrates higher precision through a conservative safe-bet strategy, the Transformer model exhibits superior sensitivity in identifying volatile movements, achieving a higher macro F1-score, with ablation experiments showing a consistent benefit from incorporating news sentiment.

Abstract

Stock Price Forecasting

Explore similar work

Predicting Stock Price Direction on Earnings Announcement Days using Multi-modal Deep Learning

Game-Theoretic Modeling of Heterogeneous Investor Interactions for Stock Price Forecasting

From Index to Equity: Pre-Training Transformers for Stock Return Prediction