Applying LLMs to predictive tasks in finance is challenging due to look-ahead bias resulting from their training on long time-series data. This precludes the backtests typically employed in finance since retraining frontier models from scratch with a specific knowledge cutoff is prohibitive. In this paper, we introduce a fast, effective, and low-cost alternative. Our method guides generation at inference time by adjusting the logits of a large base model using a pair of smaller, specialized models -- one fine-tuned on information to be forgotten and another on information to be retained. We demonstrate that our method effectively removes both verbatim and semantic knowledge, corrects biases, and outperforms prior methods.
Figures & tables
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Initial LR
Best Verbatim
Best Q&A
Stupid Backoff Trigram
TopK=1
Alpha=10
princeton-nlp/Sheared-LLaMA-1.3B
5e-5
TopK=100
Alpha=0.8
princeton-nlp/Sheared-LLaMA-2.7B
4e-5
TopK=200
Alpha=1.0
Appendix
Table 1 : Configuration MUSE
Method
Epochs
Method-Specific Hyperparameters
GradDiff
1 ∗
α=1.0,γ=1.0
NPO
10
β=0.1,α=1.0,γ=1.0
SimNPO
10
δ=0,β=4.5,α=1.0,γ=0.125
Appendix
Table 2 : MUSE Configurations
Name
Score
google/gemma-3-27b-it
77.94
Unlearning Split B α =2
77.58
Unlearning Split B TopK=250
75.09
Appendix
Table 3 : MMLU CoT 0-Shot
Set A
Set B
Bucyrus International <> Caterpillar
Allegheny Energy <> FirstEnergy
El Paso <> Kinder Morgan
Goodrich <> United Technologies
Hillshire Brands <> Tyson Foods
Biomet <> Zimmer Holdings
Rockwood Holdings <> Albemarle
Family Dollar Stores <> Dollar Tree
Rock-Tenn <> MeadWestva
Bally Technologies <> Scientific Games
Bright House Networks LLC <> Charter Communications
Pharmacyclics <> AbbVie
Appendix
Table 4 : Final Deal Sets
Tell me about the acquisition of {target name} by {acquirer name}
When did {acquirer name} announce the acquisition of {target name}?
What firm bought {target name} in {year}?
Why did {acquirer name} acquire {target name}?
What strategic benefits did {acquirer name} gain from acquiring {target name}?
How did the {target name} acquisition help {acquirer name}’s business strategy?
What synergies were expected from the {acquirer name}-{target name} deal?
Appendix
Table 5 : Prompts used to distill data for the M&A (Temperature=0.4)
Ran for 2014-2016 and 2022-2024 for both Southwest Airlines and United Airlines.
Summarize the financial performance of {airline} in {year}
How well did {airline} do in {year}?
Summarize the operational performance of {airline} in {year}
What was the outlook for {airline} going into {year}
Was {year} a good year for {airline}?
Appendix
Table 6 : Prompts used to distill data for Airlines (Temperature=0, Gemma API)