A Financial Brain Scan of the LLM
Organizations: MIT Sloan and NBER. · University of Melbourne. · Yale SOM. · ESSEC Business School.
Abstract
We use sparse autoencoders (SAEs) to "brain scan" large language models, mapping internal activations to interpretable concepts and enabling targeted "steering" of specific features. We show that prompting models to adopt specific preferences activates many unintended concepts, complicating causal inference. In contrast, SAEs permit smooth, more isolated steering of target concepts. Furthermore, our scans reveal a stark disconnect between the model's internal feature contributions and its self-generated explanations, showing that these explanations are unreliable guides to its reasoning. Finally, SAEs provide a lightweight intervention for existing models, producing interpretable embeddings and allowing researchers to attenuate specific economic biases.
Figures & tables
| Theme | Topics |
| Corporate Finance | Leverage; Financial Distress; CAPEX; Investments; Revenues growth; Mergers & Acquisitions (M&A); Dividend Policy; Cost of Capital; Working Capital; Inventory; Governance; Capital Budgeting; Human resources |
| Investment & Portfolio Management | Risk Aversion; Asset Allocation; Alpha; Beta; Sharpe Ratio; Passive investing; Arbitrage |
| Risk Management & Derivatives | Value at Risk (VaR); Hedging; Credit Default Swaps (CDS); Counterparty Risk; Systemic Risk; Stress Testing; Operational Risk; Regulation |
| Financial Markets & Instruments | Equity Markets; Fixed Income; Derivatives; Liquidity; Initial Public Offering (IPO); Short Selling; High-Frequency Trading (HFT); Foreign Exchange (FX); Commodities |
| Macro-Finance & Economics | Inflation; Unemployment; Monetary Policy; Fiscal Deficit; Gross Domestic Product (GDP); Quantitative Easing (QE); Yield Curve; Balance of Payments; Tariffs |
| Variable | Corr. (Level) | Corr. (Revision) |
| Employment | 0.277** | 0.256** |
| Federal Gov. Consumption Growth | 0.197* | 0.064 |
| Housing Starts | 0.246** | 0.396*** |
| Industrial Production | -0.155 | 0.356*** |
| Real Consumption Growth | 0.418*** | 0.326*** |
| Real Nonresidential Inv. Growth | 0.590*** | 0.465*** |
| Default (Unsteered) | Steered (Quantitative Easing) |
| Prompt: Here is a piece of news: The US economy is experiencing a crisis because of the housing bubble. Do you think this news will increase or decrease the 3-month treasury bill rate? Please include a very short explanation. | |
| This news would likely increase the 3-month Treasury bill rate. Here’s why: Risk Aversion: A housing crisis signals economic instability, making investors more risk-averse. They flock to safe-haven assets like Treasury bills, increasing demand and pushing up their rates. Let me know if you have any other questions! | This news would likely decrease the 3-month Treasury bill rate. Here’s why: Crisis signals low growth: A housing bubble crisis suggests weak economic conditions and potential recession. Central bank response: To stimulate the economy, central banks (like the Federal Reserve) typically lower interest rates, including short-term rates like Treasury bills. Lower rates encourage borrowing and investment, hoping to boost economic activity. Let me know if you have any other questions! |
| SAE Steering | Prompt Steering | ||||||
| Avg. Return | Avg. Return | ||||||
| Steering | Pos. Class. | Positive | Negative | Steering | Pos. Class. | Positive | Negative |
| 58.4% | 0.0105 | 0.0% | 0.0066 | 0.0033 | |||
| 61.6% | 0.0101 | 0.2% | 0.0282 | 0.0033 | |||
| 64.2% | 0.0098 | 0.1% | 0.0336 | 0.0033 | |||
| 67.6% | 0.0095 | 67.6% | 0.0095 | ||||
| Sparse features | |||||||||
| Benchmark | 10 | 50 | 100 | 500 | 1000 | 3000 | 5000 | ALL | |
| EW Sharpe | 3.67 | 3.44 | 3.84 | 4.15 | 4.50 | 4.45 | 4.95 | 4.78 | 5.22 |
| VW Sharpe | 1.55 | 1.73 | 1.71 | 1.76 | 1.90 | 1.92 | 1.92 | 1.91 | 1.99 |
| Accuracy | 50.90% | 50.92% | 51.14% | 51.33% | 51.38% | 51.33% | 51.42% | 51.38% | 51.45% |
| Year | Number of Firms | Number of News | % Return 0 | News length |
| 2015 | 3,610 | 209,447 | 49.2% | 92 |
| 2016 | 3,481 | 203,412 | 50.1% | 90 |
| 2017 | 3,468 | 203,972 | 48.4% | 91 |
| 2018 | 3,455 | 199,420 | 47.9% | 87 |
| 2019 | 3,460 | 215,949 | 51.1% | 88 |
| 2020 | 3,550 | 225,337 | 46.5% | 88 |
LLM Interpretability
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
See all 824 papers