FinAnchor: Aligned Multi-Model Representations for Financial Prediction

Zirui He; Huopu Zhang; Yanguang Liu; Sirui Wu; Mengnan Du

cs.CLFeb 24, 2026

FinAnchor: Aligned Multi-Model Representations for Financial Prediction

Authors: Zirui He, Huopu Zhang, Yanguang Liu, Sirui Wu, Mengnan Du

Organizations: New Jersey Institute of Technology · Georgia Institute of Technology · Chinese University of Hong Kong, Shenzhen

Abstract

Financial prediction from long documents involves significant challenges, as actionable signals are often sparse and obscured by noise, and the optimal LLM for generating embeddings varies across tasks and time periods. In this paper, we propose FinAnchor(Financial Anchored Representations), a lightweight framework that integrates embeddings from multiple LLMs without fine-tuning the underlying models. FinAnchor addresses the incompatibility of feature spaces by selecting an anchor embedding space and learning linear mappings to align representations from other models into this anchor. These aligned features are then aggregated to form a unified representation for downstream prediction. Across multiple financial NLP tasks, FinAnchor consistently outperforms strong single-model baselines and standard ensemble methods, demonstrating the effectiveness of anchoring heterogeneous representations for robust financial prediction.

Figures & tables

Figure 1: (a) Three different LLM independently predict next-day stock movement from the same earnings call transcript: Models A and B predict negative return, while Model C predicts positive return. Highlighted phrases denote text snippets deemed salient by each model. In this case, Model A and B emphasize financially relevant cues (e.g., margins, demand, guidance), while Model C is dominated by less relevant content. (b) Multi-Model Representation Alignment Procedure. We select an anchor LLM space as a common coordinate system and learn ridge regression maps that project source-model representations into the anchor space. The aligned representations are then aggregated in the anchor space to form an aligned representation for downstream prediction.

Method	Conference Call		10-Q		FNSPID Nasdaq News		Stock Movement		FOMC
Method	Accuracy $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	F1 $\uparrow$
Zero-shot Prompting	0.650	0.676	0.596	0.628	0.647	0.669	0.481	0.301	0.506	0.508
Few-shot Prompting	0.694	0.695	0.611	0.642	0.685	0.689	0.520	0.602	0.542	0.556
Longformer	0.744	0.718	0.721	0.704	0.758	0.743	0.530	0.692	0.636	0.617
Hierarchical FinBERT	0.772	0.721	0.735	0.712	0.771	0.754	0.538	0.619	0.636	0.608
Llama3.1-8B	0.758	0.716	0.722	0.669	0.767	0.734	0.578	0.646	0.607	0.603

Table 1: Accuracy and F1 comparison across five datasets (red/blue: best/second best per column).

	Gemma	Llama	Qwen
Gemma	1.000	0.811	0.802
Llama	0.782	1.000	0.819
Qwen	0.791	0.838	1.000

Table 2: Error overlap (%) between model pairs on the test set. Entry

(A,B)

denotes

P(B\ \text{error}\mid A\ \text{error})

Figure 2: Decision transitions after alignment on Stock Movement Prediction task. Counts of label transitions from Gemma to the FinAnchor under validation-chosen thresholds. The FinAnchor corrects a substantial number of false positives (FP

\rightarrow

TN), while also introducing smaller regressions.

Figure 3: FinAnchor increases confidence on corrected cases. We measure the change in confidence assigned to the ground-truth label,

\Delta P(y)=P_{\text{Aligned}}(y)-P_{\text{Gemma}}(y)

, on the test set and stratify examples into three groups: Corrected (Gemma incorrect, Aligned correct), Degraded (Gemma correct, Aligned incorrect), and Unchanged . Boxplots show the distribution of

\Delta P(y)

within each group (median and interquartile range; whiskers denote 1.5

\times

IQR; outliers omitted). Positive values indicate that the FinAnchor assigns higher probability to the true label than Gemma.

Source $\rightarrow$ Anchor	Train $R^{2}$
Llama-3.1-8B $\rightarrow$ Gemma-2-9B	0.97
Qwen3-8B $\rightarrow$ Gemma-2-9B	0.98

Table 3: Linear representation alignment quality measured by train-set

R^{2}

on the anchor space.

Idx	Sentence (full)	Gemma	Llama	FinAnchor
205	First quarter, as the industry is having more promotional activity and we were price, we saw fairly significant increase in demand and also a corresponding reduction in gross margin .	$\downarrow\,-0.040$	$\downarrow\,-0.380$	$\downarrow\,-0.807$
62	The heightened promotional environment that we experienced in the first quarter moderated over the last several months, resulting in comparable gross margin rates versus last year.	$\downarrow\,-0.031$	$\downarrow\,-0.365$	$\downarrow\,-0.795$
160	We continue to see that business growing in the mid- to high-20s range, and so that’s the continuing funding of that business.	$\downarrow\,-0.033$	$\downarrow\,-0.358$	$\downarrow\,-0.780$
156	I wanted to follow up on the growth in inventory this quarter and specifically, the planned growth in full-line.	$\downarrow\,-0.025$	$\downarrow\,-0.363$	$\downarrow\,-0.772$
61	Our gross profit rate decreased 7 basis points from last year primarily due to higher occupancy expenses related to Rack’s growth.	$\downarrow\,-0.023$	$\downarrow\,-0.351$	$\downarrow\,-0.770$

Table 4: Sentence-level occlusion on a false-positive example for Gemma-only that is corrected by the FinAnchor (ground truth: down ). For each sentence

i

, we report

\Delta z_{i}=z_{\text{base}}-z_{-i}

, where

z

is the up logit and

z_{-i}

is the logit after removing sentence

i

. Arrows indicate direction:

\uparrow

supports up (

\Delta z>0

), while

\downarrow

supports down (

\Delta z<0

). Highlighted spans mark salient financial cues.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: ROC-AUC comparison across five datasets. We report ROC-AUC for six methods (Zero-shot, Few-shot, FinBERT, Longformer, Anchor (Gemma), and FinAnchor), grouped by dataset.

Model / Setting	F1	Acc	ROC-AUC
Source (Llama)	0.603	0.607	0.807
Source (Qwen)	0.604	0.611	0.811
Anchor (Gemma)	0.640	0.638	0.823
Concatenation (no alignment)	0.619	0.613	0.805
FinAnchor	0.656	0.643	0.824

Appendix

Table 5: FOMC results for a lightweight ablation study (F1/Acc/ROC-AUC).

Financial Forecasting

Financial forecasting uses market, company, and economic data to predict prices, returns, volatility, and business outcomes for decisions and risk management.

See all 88 papers

Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning
May 28, 2026Yujin Jeong, Noelle Jung, Brian Y. C. LeungSentence Embeddings Time Series Forecasting
Mike
A Foundation Model for Multimodal Event Sequences in Financial Applications
Jul 10, 2026Nikita Rusakov, Vladislav Meshkov, Konstantin Zorin +4Financial Services Multimodal Pretraining
Sber Moscow, Russian Federation · Sber AI Lab Moscow, Russian Federation
Relational Probing: LM-to-Graph Adaptation for Financial Prediction
Apr 11, 2026Yingjie Niu, Changhong Jin, Rian Dolphin +1LLMs for Graph Learning Financial Forecasting
University College Dublin, Belfield, Dublin, Ireland · Massive, Dublin, Ireland

May 28, 2026J/K move · Enter open · S save

Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning

Yujin Jeong, Noelle Jung, Brian Y. C. Leung

Mike

Sentence Embeddings Time Series Forecasting Siamese Neural Networks Multimodal Embedding

Open paper AskPDF arXiv

Traditional multi-modal financial forecasting often relies on scalar sentiment scores, which fail to capture the nuances of financial news. To address this information loss, this paper explores high-dimensional representation learning by replacing discrete polarity ratings with dense FinBERT embeddings within a Transformer-based forecasting architecture. We benchmarked various embedding strategies on the FNSPID dataset, including raw embeddings, attention-weighted aggregation, and a custom Siamese network. While the attention-based mechanism struggled with the low signal-to-noise ratio typical of financial data, the integration of Siamese-optimized embeddings outperformed both the scalar baseline and raw embedding approaches, demonstrating that preserving high-dimensional narrative context yields improved predictive accuracy for short-term stock price movements.

Abstract

Financial Forecasting

Explore similar work

Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning

A Foundation Model for Multimodal Event Sequences in Financial Applications

Relational Probing: LM-to-Graph Adaptation for Financial Prediction