Bias in financial language models constitutes a major obstacle to their adoption in real-world applications. Detecting such bias is challenging, as it requires identifying inputs whose predictions change when varying properties unrelated to the decision, such as demographic attributes. Existing approaches typically rely on exhaustive mutation and pairwise prediction analysis over large corpora, which is effective but computationally expensive-particularly for large language models and can become impractical in continuous retraining and releasing processes. Aiming at reducing this cost, we conduct a large-scale study of bias in five financial language models, examining similarities in their bias tendencies across protected attributes and exploring cross-model-guided bias detection to identify bias-revealing inputs earlier. Our study uses approximately 17k real financial news sentences, mutated to construct over 125k original-mutant pairs. Results show that all models exhibit bias under both atomic (0.58%-6.05%) and intersectional (0.75%-5.97%) settings. Moreover, we observe consistent patterns in bias-revealing inputs across models, enabling substantial reuse and cost reduction in bias detection. For example, up to 73% of FinMA's biased behaviours can be uncovered using only 20% of the input pairs when guided by properties derived from DistilRoBERTa outputs.
Figures & tables
Model
Atomic (Body)
Inter. (Body)
Atomic (Gender)
Inter. (Gender)
Atomic (Race)
Inter. (Race)
Total (Atomic)
Total (Inter.)
Total Hidden (Inter.)
FinMA
9.23%
7.48%
2.77%
2.25%
3.25%
3.29%
3.99%
3.23%
4.05%
FinGPT
5.39%
2.77%
6.10%
6.55%
6.13%
6.07%
6.05%
5.97%
31.29%
FinBERT
1.89%
1.88%
0.69%
0.88%
0.25%
0.41%
0.58%
0.75%
30.34%
DeBERTa-v3
1.69%
1.67%
0.70%
0.89%
0.30%
0.46%
0.60%
0.75%
29.95%
DistilRoBERTa
1.69%
1.67%
0.70%
0.89%
0.30%
0.46%
0.60%
0.75%
29.95%
Table 1: Models bias rates on FinSen. Total tested pairs, per attribute: Body = 22,972; Gender = 105,962; Race = 121,490, overall = 125,161.
Figure 3: Comparison of FinMA bias-revealing and non-bias-revealing inputs, based on the JSD distance between original and mutated statements computed on their prediction probabilities, obtained from lightweight and large models.
Prioritization of inputs by decreasing JSD of their DistilRoBERTa prediction shifts.
FinMA
73,01
+53,01
O(10−18)
1
89,64
+49,64
O(10−18)
1
95,49
+35,49
O(10−18)
1
97,5
+17,5
O(10−18)
1
Table 3: Comparison of the efficiency and cost efficiency of a guided input selection with a conventional (random) one, in detecting large model bias based on DistilRoBERTa predictions. We note respectively BD%, Δ , p-val, and A^12 , the bias detection ratio, the BD% difference with random, Wilcoxon p-values, and A^12 values.
Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already observed the outcomes it is asked to predict. To handle this issue, point-in-time (PIT) language models are pretrained on chronologically filtered corpora and released as one checkpoint per calendar year, each with a documented cutoff. However, each additional year costs another pretraining run, and whether that run is necessary has never been tested. In this paper, we show that a newer checkpoint is not detectably better on the decision it informs, as its information coefficient on a fixed evaluation window shows no decay with age that we can detect. Motivated by this observation, we propose PALM (Point-in-time Adaptation for financial Language Models), a simple yet effective alternative to annual pretraining that fits a low-rank adapter on financial text the checkpoint is allowed to read, without modifying any pretrained weight. We further find that the gain comes from the financial domain rather than from recency, which leaves the point-in-time constraint without a detectable cost. We validate PALM on a decade of financial news and on three families of PIT models, whose cutoffs span two decades. The pooled gain keeps its sign across experimental settings and comes at a small fraction of the cost of pretraining. Code is available at: https://github.com/seunghan96/palm.
Language models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale. Such preferential biases can be introduced by any actor in the model's supply chain and are most dangerous when the model reveals its preference only on the relevant topic while behaving identically to its unmodified base on all other inputs. Recent work has shown that these biases can transfer through context distillation on semantically unrelated data, with the signal residing entirely in the soft logit distribution and remaining invisible to text-based inspection. However, the defender faces a fundamental asymmetry: without knowing the bias topic, no detection method can reliably surface a stealth preferential bias, regardless of whether it examines generated text, internal representations, or model weights. Here we introduce Distill to Detect (D2D), a method that surfaces hidden biases by distilling the distributional shift between a suspected model and its base into a cartridge (a KV-cache prefix adapter), concentrating the dominant divergence and amplifying the bias signal into generated text. We show that D2D successfully amplifies the hidden biases of stealth models to the extent that they can be reliably detected across multiple bias types. We also propose a theoretical framework that explains the efficacy of D2D through the lens of Fisher-weighted projection of the logit distribution shift, supported by empirical observations. By turning the capacity bottleneck of prefix-tuning adapters into a detection tool, D2D provides a practical building block for auditing hidden behaviors in deployed language models.
Shayan Talaei, Abhinav Chinta, Devvrit Khatri +3
1Stanford University · 3Foundation AI–Cisco Systems Inc. · University of Texas at Austin
Large language models (LLMs) are increasingly deployed in financial contexts, raising critical concerns about reliability, alignment, and susceptibility to adversarial manipulation. While prior finance-related benchmarks assess LLMs' capabilities in stock trading, they are often restricted to small sample and fail to demonstrate LLM susceptibility to context with potential human bias. We introduce Fin-Bias (financial herding under long and uncertain financial context), a benchmark for evaluating LLM investment decision-making when faced with uncertainty and possible human-biased opinions. Fin-Bias includes 8868 long firm-specific analyst reports, including firm aspects summarized and analyzed by sophisticated analysts with investment ratings (Bullish/Neutral/Bearish) spanning from various industries. We present large language models with firm analyst reports with/without analyst investment ratings and even with 'fake' rating, to get investment ratings generated by LLMs. Our results reveal that LLMs tend to herd the explicit bias in context. We also develop a method to detect potential human opinions, which can encourage LLMs to think independently, some models even exceed human performance in predicting future stock return.
Xiaoyu Hu, Jinman Zhao
Rutgers University · Department of Computer Science, University of Toronto