Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts

Alina Khaybullina

cs.LGOct 15, 2026

Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts

Authors: Alina Khaybullina

Abstract

Which inputs drive a language model's financial forecast? We construct synthetic earnings announcements that independently reverse 20-day momentum, non-informational commentary tone, and whether fixed actual EPS beats a manipulated consensus. Six instruction/chat checkpoint conversions from four families (0.49-7.61B base parameters) receive 2,016 prompts each under a direct-answer Q8_0 protocol. The design includes 24 numerical blocks, six commentary pairs, profits and losses, rising and falling annual EPS, and two surprise sizes. Both A/B answer assignments and U/D cover all 384 primary scenarios per conversion. We estimate matched changes in conditional answer-prefix scores, not calibrated market probabilities. Across these checkpoints, the primary EPS contrast ranges from -0.19 to 86.80 percentage points. The magnitude of the primary signed tone contrast exceeds that of EPS in 5 of six models, and momentum exceeds it in 5. Natural-word numerical diagnostic accuracy ranges from 51.0 to 95.8 percent. Answer-encoding, wording, context, and conversion controls qualify these results. The experiment measures observable input reliance and numerical recognition; it establishes neither an internal reasoning mechanism nor realized return predictability. Public benchmark prompts, recorded scores, and an interactive recorded-results explorer accompany the study.

Figures & tables

Checkpoint	Params (B)	Family	Tokenizer rev.	GGUF rev.
Qwen2.5-0.5B	0.49	Qwen	7ae55760	9217f5db
Qwen2.5-1.5B	1.54	Qwen	989aa798	91cad511
SmolLM2-1.7B	1.71	SmolLM2	31b70e2e	1f034647
TinyLlama-1.1B	1.10	TinyLlama	fe8a4ea1	52e7645b
Phi-3.5-mini	3.82	Phi	2fe19245	6d70da17
Qwen2.5-7B	7.61	Qwen	a09a3545	bb5d59e0

Table 1: Six instruction/chat checkpoints, all evaluated as Q8_0 conversions. Revision columns are abbreviated; complete hashes and file digests are distributed. Parameter counts describe the base checkpoints and are rounded.

Checkpoint	Momentum	Tone	EPS surprise
Qwen2.5-0.5B	1.24 [0.816, 1.667]	4.02 [3.089, 5.027]	$-0.15$ [ $-0.506$ , 0.192]
Qwen2.5-1.5B	16.55 [15.224, 18.028]	30.39 [26.558, 34.113]	$-0.19$ [ $-0.444$ , 0.065]
SmolLM2-1.7B	6.27 [5.740, 6.822]	6.46 [5.551, 7.355]	$-0.08$ [ $-0.187$ , 0.029]
TinyLlama-1.1B	1.01 [0.859, 1.169]	0.22 [ $-0.134$ , 0.587]	$-0.10$ [ $-0.194$ , $-0.003$ ]
Phi-3.5-mini	20.77 [16.174, 26.089]	45.01 [33.089, 57.099]	6.57 [4.566, 8.772]
Qwen2.5-7B	10.65 [6.702, 14.726]	9.84 [5.359, 14.531]	86.80 [81.027, 92.038]

Table 2: Primary positive-minus-negative input effects in percentage points (pp), averaging both A/B assignments on every case. Brackets show 95% percentile intervals from 10,000 resamples of 24 earnings blocks. These are conditional benchmark intervals, not uncertainty about realized stock returns.

Figure 1: Primary input contrasts. The three colored, shape-coded series share one horizontal scale; each row represents a checkpoint conversion. Intervals resample 24 earnings blocks conditional on the fixed conversion and wording set. Printed values give the signed contrasts in pp.

Checkpoint	Label MAE	Flip (%)	AB–UD	Order	Office
Qwen2.5-0.5B	12.06	43.0	11.43	3.26	7.50
Qwen2.5-1.5B	9.79	6.8	15.75	11.12	2.64
SmolLM2-1.7B	9.21	0.0	3.52	1.97	1.69
TinyLlama-1.1B	63.32	83.3	24.71	7.66	3.38
Phi-3.5-mini	17.65	17.7	9.46	6.01	6.02
Qwen2.5-7B	3.02	2.3	2.69	7.56	5.74

Table 3: Forecast format and irrelevant-content controls. MAE columns are mean absolute score differences in pp. Label reversal and AB–UD use all 384 primary scenarios; order compares 192 matched cases; office compares 96 A/B-averaged cases. Flip is the share of A/B reversals changing whether the conditional positive-answer score exceeds 0.5.

Figure 2: Matched effects under AB0, AB1, their casewise score average, and UD on all 384 primary scenarios per model. Individual mappings are not treated as independent replications.

Checkpoint	A=positive	B=positive	A/B mean	U/D
Qwen2.5-0.5B	Tone	Tone	Tone	Tone
Qwen2.5-1.5B	Tone	Tone	Tone	Tone
SmolLM2-1.7B	Tone	Momentum	Tone	Momentum
TinyLlama-1.1B	Momentum	Tone	Momentum	Momentum
Phi-3.5-mini	Tone	Tone	Tone	Tone
Qwen2.5-7B	EPS	EPS	EPS	EPS

Table 4: Input with the largest absolute mean contrast in each encoding, on the same 384 scenarios. Ranking is descriptive and ignores interval overlap; negative effects can also have large absolute magnitude. Full signed effects are shown in Figure 2 .

Checkpoint	A=beat	B=beat	Mean score	Mean accuracy	Words
Qwen2.5-0.5B	43.8	50.0	50.0	46.9	55.2
Qwen2.5-1.5B	60.4	70.8	63.5	65.6	63.5
SmolLM2-1.7B	50.0	50.0	50.0	50.0	51.0
TinyLlama-1.1B	50.0	50.0	50.0	50.0	54.2
Phi-3.5-mini	78.1	75.0	77.1	76.6	84.4
Qwen2.5-7B	90.6	94.8	91.7	92.7	95.8

Table 5: Numerical EPS beat/miss accuracy (%) on the same 96 scenarios per format. Mean score classifies the casewise average of A/B scores; mean accuracy averages the two assignment-specific accuracies. Words score the natural beat/miss prefixes. The methods estimate distinct quantities.

Figure 3: Tone and EPS effects within each of six commentary pairs, using four numerical blocks per pair. The two stacked matrices have separate signed color scales and explicit pp values. Between-pair variation can include the numerical blocks assigned to each pair.

Figure 4: Numerical beat/miss diagnostic on 96 balanced scenarios. The dashed line marks 50% for this balanced task. Intervals on A/B-averaged and natural-word accuracy bootstrap 24 blocks; individual assignments are point estimates. The formats are distinct measurements; none is realized forecast accuracy.

Checkpoint	M+S	+E	+context	EPS gain	Held pair
Qwen2.5-0.5B	0.160	0.160	0.661	$-0.000515$	0.107
Qwen2.5-1.5B	0.717	0.717	0.834	$-0.000000$	0.702
SmolLM2-1.7B	0.767	0.767	0.772	$-0.000005$	0.753
TinyLlama-1.1B	$-0.045$	$-0.045$	0.944	0.000033	$-0.052$
Phi-3.5-mini	0.417	0.424	0.324	0.007589	0.414
Qwen2.5-7B	0.019	0.829	0.820	0.810390	0.830

Table 6: Out-of-sample

R^{2}

of linear surrogates of primary A/B-averaged scores. M+S uses momentum and tone; +E adds surprise; +context adds section order, commentary-pair and introduction indicators. First four numeric columns use leave-one-block-out fitting. Held pair uses M+S+E while leaving one of six commentary pairs out. Negative

R^{2}

indicates worse performance than the pooled target-mean reference.

Figure 5: Left: EPS effects after commentary removal, with block-bootstrap intervals. These cases use facts-first order only. Right: mean absolute score differences induced by answer mapping and by changing A/B to UD on matched primary cases.

Figure 6: Leave-one-block-out linear score reproduction. Adding EPS tests the incremental value of that manipulated input to this specific linear surrogate; context features can capture authored template differences.

Figure 7: Identical saved pilot prompts under the old FP32 setup and the new Q8 conversion/runtime. The dashed line marks equality and solid lines mark the conditional 0.5 threshold. Cases are selected across task types and saved score ranges, not randomly. This is a joint setup comparison, not a pure quantization experiment.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Pair	Positive opinion	Negative opinion
1	This is an encouraging and impressive development; the outlook feels bright.	This is a disappointing and worrying development; the outlook feels bleak.
2	I feel optimistic about this announcement.	I feel pessimistic about this announcement.
3	My overall impression of this release is favorable.	My overall impression of this release is unfavorable.
4	My reaction to this announcement is enthusiastic.	My reaction to this announcement is discouraged.
5	I welcome this announcement and feel confident.	I regret this announcement and feel uneasy.
6	This announcement leaves me upbeat about the shares.	This announcement leaves me downbeat about the shares.

Appendix

Table 7: Six authored commentary pairs. The surrounding prompt explicitly states that the opinion contains no additional facts. These are wording variations from one construction process, not six independently sampled authors or validated human sentiment labels.

Checkpoint	Momentum logit	Tone logit	EPS logit	Neutral EPS (pp)
Qwen2.5-0.5B	0.054	0.170	$-0.006$	$-0.02$
Qwen2.5-1.5B	0.783	1.413	$-0.009$	$-0.23$
SmolLM2-1.7B	0.326	0.331	$-0.004$	$-0.31$
TinyLlama-1.1B	0.047	0.042	$-0.006$	$-0.04$
Phi-3.5-mini	5.377	10.075	1.719	5.80
Qwen2.5-7B	3.468	3.524	15.635	96.86

Appendix

Table 8: Marginal effects on the average of the two answer-assignment log-likelihood ratios, and the EPS score contrast after removing commentary. Logit effects use a different scale from the main probability-score contrasts. Neutral EPS uses all 24 blocks at facts-first order only.

Checkpoint	$\|\Delta M\|$	$\|\Delta S\|$	$\|\Delta E\|$	EPS 95th	EPS $>0$ (%)
Qwen2.5-0.5B	2.18	4.30	1.82	4.46	48.4
Qwen2.5-1.5B	16.55	30.39	1.35	3.26	50.0
SmolLM2-1.7B	6.27	6.47	0.86	2.07	42.7
TinyLlama-1.1B	1.09	1.00	0.60	1.38	44.3
Phi-3.5-mini	20.77	45.01	6.67	36.82	87.5
Qwen2.5-7B	10.69	10.94	86.80	99.99	100.0

Appendix

Table 9: Mean absolute matched score changes (pp), comparing 192 positive/negative input pairs per factor after A/B averaging. EPS 95th is the 95th percentile of absolute pair changes, not a confidence limit. EPS

>0

is the share of beat-minus-miss pairs with a positive score change. These exploratory summaries detect cancellation hidden by a small signed mean.

Checkpoint	Momentum	Tone	EPS surprise
Qwen2.5-0.5B	1.58 / 0.90	3.83 / 4.22	$-0.29$ / $-0.01$
Qwen2.5-1.5B	13.61 / 19.50	33.58 / 27.20	$-0.42$ / 0.05
SmolLM2-1.7B	6.82 / 5.71	6.64 / 6.29	$-0.18$ / 0.03
TinyLlama-1.1B	0.90 / 1.12	0.27 / 0.17	$-0.10$ / $-0.10$
Phi-3.5-mini	20.26 / 21.29	48.91 / 41.10	6.20 / 6.94
Qwen2.5-7B	8.94 / 12.37	10.77 / 8.91	86.88 / 86.72

Appendix

Table 10: Input contrasts (pp) in facts-first / facts-last section order, using the same 24 blocks in each order. These exploratory estimates preserve all other input matching and do not change the primary order-averaged estimand.

Checkpoint	Loss	Profit	Decline	Rise	2% gap	15% gap
Qwen2.5-0.5B	0.21	$-0.51$	$-0.08$	$-0.22$	$-0.31$	0.01
Qwen2.5-1.5B	$-0.44$	0.06	$-0.48$	0.10	$-0.28$	$-0.10$
SmolLM2-1.7B	$-0.15$	$-0.00$	$-0.01$	$-0.15$	$-0.16$	0.00
TinyLlama-1.1B	$-0.15$	$-0.05$	$-0.15$	$-0.05$	$-0.02$	$-0.18$
Phi-3.5-mini	2.50	10.65	6.08	7.06	4.79	8.35
Qwen2.5-7B	84.54	89.07	84.49	89.11	79.52	94.08

Appendix

Table 11: Primary EPS contrasts (pp) within profitability, annual EPS change, and surprise-size subgroups. Every subgroup has 12 earnings blocks. These exploratory comparisons span different numerical blocks and may also reflect differences in assigned wording; they do not isolate the causal effect of profitability or growth.

Checkpoint	Loss	Profit	Decline	Rise	2% gap	15% gap
Qwen2.5-0.5B	60.4	50.0	52.1	58.3	52.1	58.3
Qwen2.5-1.5B	52.1	75.0	62.5	64.6	64.6	62.5
SmolLM2-1.7B	50.0	52.1	45.8	56.2	52.1	50.0
TinyLlama-1.1B	56.2	52.1	52.1	56.2	54.2	54.2
Phi-3.5-mini	83.3	85.4	83.3	85.4	79.2	89.6
Qwen2.5-7B	91.7	100.0	95.8	95.8	91.7	100.0

Appendix

Table 12: Natural beat/miss prefix diagnostic accuracy (%) within numerical subgroups; 48 scenarios and 12 blocks per subgroup. This known-answer task is distinct from return prediction. The numerical blocks differ between subgroups, so comparisons are descriptive.

Checkpoint	Momentum	Tone	EPS surprise
Qwen2.5-0.5B	5.31 / 1.07	28.79 / 3.77	0.03 / $-0.12$
Qwen2.5-1.5B	33.40 / 16.82	50.91 / 30.85	$-0.14$ / $-0.51$
SmolLM2-1.7B	19.99 / 6.23	18.83 / 6.52	$-0.11$ / $-0.06$
TinyLlama-1.1B	0.12 / 0.95	2.31 / 0.18	$-0.07$ / $-0.08$
Phi-3.5-mini	23.56 / 21.63	35.78 / 45.37	4.98 / 5.52
Qwen2.5-7B	10.37 / 12.95	9.26 / 13.38	85.01 / 83.38

Appendix

Table 13: Natural positive/negative word-prefix effects / matched A/B-averaged effects (pp). Each format covers exactly the same 192 scenarios: one loss and one profitable numerical block for each commentary pair, both section orders. Profitable cases were added in the documented control-coverage amendment. This test is descriptive and is not interchangeable with the 384-scenario primary estimand.

Checkpoint	AB0 mass	top	AB1 mass	top	UD mass	top
Qwen2.5-0.5B	98.2	100.0	98.4	100.0	98.0	100.0
Qwen2.5-1.5B	99.8	100.0	99.7	100.0	99.9	100.0
SmolLM2-1.7B	94.6	100.0	93.7	100.0	94.0	100.0
TinyLlama-1.1B	38.5	88.3	57.9	100.0	11.9	35.2
Phi-3.5-mini	100.0	100.0	100.0	100.0	100.0	100.0
Qwen2.5-7B	100.0	100.0	100.0	100.0	100.0	100.0

Appendix

Table 14: Primary requested-prefix mass and first-token compliance (%). Mass is the average unconditioned probability assigned to either requested prefix. Top is the share whose highest-logit next token starts a requested candidate. Both are distinct from semantic comprehension or calibrated market probability.

Pilot case	FP32	Rebuilt F16	Rebuilt Q8	Stock Q8
b02_m1_s-1_e1_t0_o0_labels0	55.76	55.09	66.96	57.88
b08_e1_diagnostic1_reversed	15.13	15.27	20.32	17.75
b00_m1_s-1_e-1_t0_o1	39.31	39.83	44.58	72.10
b10_e1_diagnostic1	80.62	80.59	85.73	47.32
b07_e1_diagnostic1	84.54	84.84	86.12	51.04
b00_e-1_diagnostic1	78.57	78.67	83.97	37.83

Appendix

Table 15: Conditional positive/beat candidate scores (%) for six selected Qwen0.5 pilot stress cases. The rebuilt F16 and Q8 files use the same pinned HF source; stock Q8 is the downloaded first-party conversion used in the main experiment. Cases span tasks and are selected after observing bridge differences; this is an exploratory precision/provenance investigation, not an error-rate sample or a new full-precision factorial experiment.

Financial Forecasting

Financial forecasting uses market, company, and economic data to predict prices, returns, volatility, and business outcomes for decisions and risk management.

See all 86 papers

Can Language Models Learn to Forecast Stock Prices
Sep 29, 2026Jiacheng Guo, Suozhi Huang, Shuzhen Li +11Financial Forecasting LLM-Based Time Series Forecasting
Princeton University · InclusionAI · Harvard University +1
MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting
Sep 29, 2026Dong Shu, Yanguang Liu, Huopu Zhang +4Financial Forecasting Multimodal Large Language Models
Northwestern University · New Jersey Institute of Technology · Georgia Institute of Technology +2
From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts
Oct 16, 2026Shengkun Wang, Taoran Ji, Linhan Wang +3Financial Language Model Evaluation Financial Forecasting

Sep 29, 2026J/K move · Enter open · S save

Can Language Models Learn to Forecast Stock Prices

Jiacheng Guo, Suozhi Huang, Shuzhen Li, Yunlong Gao+10

Princeton University · InclusionAI · Harvard University+1

Financial Forecasting LLM-Based Time Series Forecasting Language Model Post-Training

Open paper AskPDF arXiv

Post-training has been shown to significantly improve language models' performance on tasks with verifiable outcomes, including mathematical reasoning, software engineering, and computer use. However, whether the same approach can improve forecasting in financial markets is much less clear. Compared with tasks with verifiable outcomes, not only are realized returns noisy, but even what constitutes a relevant information set for making effective predictions is not obvious a priori: the model must decide which observations to gather and then commit to a numerical judgment before the outcome is known. We study this question in a chronological stock-price sandbox, where a language model gathers price, volume, relative-performance, and market-context evidence and predicts a future return. We post-train Qwen3-4B with supervised fine-tuning (SFT) on tool-use demonstrations, then proximal policy optimization (PPO) with a terminal reward given by the forecast score against the realized return. The resulting AURA-4B more than doubles the starting direction--magnitude score, from 20.94 to 43.31, and is comparable to frontier language models on this benchmark. Conditional magnitude agreement rises from 33.3 to 66.2, while directional accuracy changes from 62.9 to 65.4. SFT expands tool use, and PPO further increases the share of ranking and market-context queries. These results show that post-training can substantially improve financial forecasting performance, together with changes in how the model investigates the market, on this outcome-selected benchmark.

Abstract

Financial Forecasting

Explore similar work

Can Language Models Learn to Forecast Stock Prices

MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts