FinCutBench: A Continually Updated Benchmark for Measuring the Temporal Boundaries of LLM Financial Knowledge
Organizations: The Hong Kong University of Science and Technology (Guangzhou)
Abstract
In financial backtesting and other point-in-time tasks, external inputs can be restricted by date, but model parameters may retain financial facts disclosed later, introducing parametric look-ahead bias. We introduce FinCutBench, a continually updated monthly financial knowledge benchmark for measuring how far a fixed model's financial knowledge extends over calendar time. The current dataset contains 9,000 Chinese and English four-option multiple-choice questions from January 2023 to September 2026, covering Chinese and U.S. markets and four categories of financial information. We construct monthly knowledge profiles using the approximate months in which the tested facts became public. After a profile passes the decline test and knot checks, we define the FinCut Boundary as the first month when the fitted profile reaches its lower level, and report the stability of the estimated date under resampling. Boundary dates and localization precision vary across models. Using the independently measured Transition Start and Boundary as fixed references for downstream sample grouping, we find significant relative declines in U.S. asset-ranking performance and in earnings recall for Sol and Luna. Downstream effects vary across markets and tasks. The FinCut Boundary thus provides an empirical reference for the temporal scope of parametric knowledge in point-in-time financial evaluation, supporting evaluation-window selection and result interpretation.
Figures & tables
| Axis | Reports | Indicators | Market | Total |
|---|---|---|---|---|
| Macro / policy | 30 | 20 | 0 | 50 |
| Sector / supply | 40 | 10 | 0 | 50 |
| Market path | 0 | 0 | 50 | 50 |
| Asset moves | 0 | 0 | 50 | 50 |
| Total | 70 | 30 | 100 | 200 |
| Model | Release Date | Official Knowledge Cutoff | Decline Detected | Overall FinCut Boundary | Localization Stability | Leading-Axis Boundary |
|---|---|---|---|---|---|---|
| Open-Weight Models | ||||||
| DeepSeek family | ||||||
| DeepSeek-V4.1-Flash-1017 a d | 2026-10-17 | - | Yes | 2026-06 | 96.2% located 2026-06–2026-07 (Tight) | 2026-08 |
| Qwen family | ||||||
| Qwen3.8-27B | 2026-08-14 | - | Yes | 2026-03 | 100.0% located 2026-02–2026-06 (Tight) | 2026-03 |
| Qwen3.5-4B d | 2026-03-02 | - | No | ND | – | ND |
| Model | China | US | ||||
|---|---|---|---|---|---|---|
| Pre | Post | Rel. | Pre | Post | Rel. | |
| Qwen3.8-27B | 0.106 | -0.038 | 0.054 | -0.261 | ||
| GPT-5.6-Sol | 0.147 | -0.031 | 0.129 | -0.326 | ||
| GPT-5.6-Luna | -0.011 | 0.063 | -0.108 | -0.328 | ||
| Claude-Haiku-5.5 | -0.027 | 0.017 | 0.114 | -0.219 | ||
| Model | Earnings recall (%) | Forecast loss ( ) | ||||
|---|---|---|---|---|---|---|
| Disc. Pre | Disc. Post | Rel. (pp) | Disc. Pre | Disc. Post | Rel. | |
| Qwen3.8-27B | 20.67 | 13.67 | 16.12 | 31.01 | ||
| GPT-5.6-Sol | 87.33 | 31.33 | 13.83 | 30.25 | ||
| GPT-5.6-Luna | 51.67 | 25.33 | 14.22 | 31.79 | ||
| Claude-Haiku-5.5 | 40.33 | 29.67 | 16.16 | 31.90 | ||
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Market | Group | Covered Series |
|---|---|---|
| CN | Macro | Manufacturing PMI; Manufacturing PMI New Orders; CPI YoY; PPI YoY; M2 YoY and level; Total Social Financing Stock YoY; Exports YoY; Industrial Value Added YoY; Retail Sales YoY; Real-Estate Development Investment YoY; Commercial-Building Sales Area YoY; Manufacturing Fixed-Asset Investment YoY. |
| Sector | Auto Sales; New-Energy Vehicle Sales and Output; Aluminum and LME Copper Inventories; Polysilicon, A00 Aluminum Ingot, Lithium Carbonate, Rebar, and Thermal-Coal Prices; Road Logistics and Coastal Container Freight Indices. | |
| US | Macro | CPI and Core CPI; PCE and Core PCE Price Indices; Unemployment Rate; Nonfarm Payroll Employment; Average Hourly Earnings; Retail Sales; Industrial Production; Capacity Utilization; M2 Money Stock; Federal Funds Rate; 3-Month Treasury Bill Rate; 2-, 5-, and 10-Year Treasury Yields; Aaa and Baa Corporate Bond Yields; 10-Year–2-Year Treasury Yield Spread. |
| Sector | WTI and Brent Crude-Oil Prices; Henry Hub Natural-Gas Price; Total Vehicle and Heavy-Truck Sales; Housing Starts; Building Permits; Durable-Goods Orders; Manufacturers’ New Orders. |
| Market | Group | Covered Assets |
|---|---|---|
| CN | Indices | CSI 300; SSE Composite; Shenzhen Component; ChiNext; CSI 500; CSI 1000; STAR 50; SSE 50; CSI 800. |
| ETFs | CSI 300; CSI 500; ChiNext; SSE 50; STAR 50; CSI 1000; Semiconductor; Securities; Photovoltaic; New-Energy Vehicle; Dividend; Dividend Low-Volatility; Gold; Treasury Bond; 10-Year Treasury Bond; Convertible Bond; Hang Seng; Hang Seng TECH; Hong Kong Connect Internet; China Internet; Nasdaq; S&P 500. | |
| Stocks | Kweichow Moutai; CATL; China Ping An; China Merchants Bank; Wuliangye Yibin; Midea Group; BYD; Hikvision; Luxshare Precision; Zijin Mining; CITIC Securities; Hengrui Medicine; China Yangtze Power; China Tourism Group Duty Free; WuXi AppTec; SMIC; Hygon Information Technology; LONGi Green Energy; East Money Information; Mindray Bio-Medical Electronics; iFLYTEK; COSCO Shipping Holdings. | |
| US | Indices | S&P 500; Nasdaq Composite; Nasdaq 100; Dow Jones Industrial Average; Russell 2000; VIX. |
| ETFs | SPY; QQQ; IWM; DIA; GLD; TLT; HYG; USO; XLK; XLF; XLE; SMH. | |
| Stocks | Apple; Microsoft; Nvidia; Amazon; Meta Platforms; Alphabet; Tesla; JPMorgan Chase; Berkshire Hathaway; Exxon Mobil; UnitedHealth Group; Broadcom; Costco Wholesale; Eli Lilly; AMD. |
| Axis / market | Question and options | Answer and evidence summary |
|---|---|---|
| Macro / policy US | From 2023-07 to 2024-07, what was the year-over-year percentage change in U.S. CPI for all urban consumers (seasonally adjusted)? A. 2.2% B. 2.0% C. 2.6% D. 2.9% | D . Dividing the 2024-07 observation by the 2023-07 observation and subtracting 1 gives 2.9%. |
| Sector / supply CN | According to China Passenger Car Association data released in August 2024, what were domestic retail sales of new energy passenger vehicles in July 2024? A. 838,000 vehicles B. 878,000 vehicles C. 798,000 vehicles D. 758,000 vehicles | B . The association reported July retail sales of 878,000 new energy passenger vehicles. This figure uses the monthly retail-sales measure, which differs from wholesale, export, and cumulative sales measures. |
| Market path US | In 2024-08, on which trading day did the S&P 500 reach its lowest index closing value? A. 2024-08-08 B. 2024-08-06 C. 2024-08-05 D. 2024-08-07 | C . Comparing daily index closing values within the target month, the lowest value was 5186.3301 on 2024-08-05. |
| Asset moves CN | Based on adjusted closing prices, which option is closest to the cumulative return of COSCO SHIPPING Holdings (601919.SH) from the close on 2024-08-01 to the close on 2024-08-30? A. 0.3% B. -2.1% C. -16.9% D. -5.6% | D . Recalculating the cumulative return from adjusted closing prices between the close on 2024-08-01 and the close on 2024-08-30 gives -5.58%. |
| Criterion | Majority pass | Unanimous pass |
|---|---|---|
| Overall usability | 126 / 126 | 122 / 126 |
| Temporal validity | 126 / 126 | 126 / 126 |
| Financial relevance | 126 / 126 | 125 / 126 |
| Dependence on new information | 126 / 126 | 126 / 126 |
| Evidence sufficiency | 126 / 126 | 125 / 126 |
| Question and option design | 126 / 126 | 124 / 126 |
| Quantity | Resampling design | Replicates | Output |
|---|---|---|---|
| Monthly accuracy | Within each month: 200 draws from all 200 groups, or 50 from the selected axis’s 50 groups | Exact distribution | Pointwise 95% percentile interval |
| Overall Boundary | Within each month-by-axis stratum: 50 draws from 50 groups; shared across nested windows | 1,000 | Localization Rate and Boundary Range |
| Axis Boundary | Same draws as above; each axis evaluated in the full-data overall stopping window | Same 1,000 | Axis Localization Rate and Boundary Range |
| Matched difficulty control | Within each month: draws from groups in the analyzed subset; shared across target and control models | 5,000 | Paired early-minus-late contrasts and pointwise 95% intervals |
| Model | Trials | Localization Rate | Exact Boundary | Within month | Boundary Range | Width |
|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash-1017 | 1000 | 96.2% | 58.2% | 95.6% | 2026-06–2026-07 | Tight |
| Qwen3.8-27B | 1000 | 100.0% | 72.0% | 87.0% | 2026-02–2026-06 | Tight |
| Qwen3.5-4B | 1000 | 7.2% | – | – | – | – |
| Qwen3.5-122B-A10B | 1000 | 99.6% | 15.3% | 38.2% | 2025-02–2026-01 | Wide |
| Qwen3.5-397B-A17B | 1000 | 100.0% | 23.9% | 50.1% | 2025-02–2025-07 | Tight |
| GPT-6-Astra | 1000 | 100.0% | 97.4% | 100.0% | 2026-06–2026-06 | Tight |
| Model | Target drop | Control drop | Difference [95% CI] |
|---|---|---|---|
| GPT-5.6-Sol | 46.8 | 3.3 | 43.5 [39.2, 47.7] |
| GPT-6-Astra | 50.2 | 3.9 | 46.3 [42.2, 50.5] |
| Gemini-3.8-Flash | 10.1 | 3.9 | 6.2 [2.0, 10.5] |
| DeepSeek-V4.1-Flash-1017 | 34.8 | 4.5 | 30.3 [26.1, 34.4] |
| Qwen3.8-27B | 13.9 | 3.7 | 10.2 [6.3, 14.0] |
| Model | Release Date | Input Window |
|---|---|---|
| DeepSeek-V4.1-Flash-1017 | 2026-10-17 | 2025-04–2026-09 |
| Qwen3.8-27B | 2026-08-14 | 2025-08–2026-08 |
| Qwen3.5-4B | 2026-03-02 | 2023-01–2026-03 |
| Qwen3.5-122B-A10B | 2026-02-24 | 2024-02–2026-02 |
| Qwen3.5-397B-A17B | 2026-02-16 | 2024-08–2026-02 |
| GPT-6-Astra | 2026-09-03 | 2025-09–2026-09 |
| Model | Knowledge cutoff | Reliable knowledge cutoff | Training data cutoff | FinCut Boundary | (mo.) |
|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash-1017 | – | – | Partial | 2026-06 | – |
| Qwen3.8-27B | – | – | – | 2026-03 | – |
| Qwen3.5-4B | – | – | – | ND | – |
| Qwen3.5-122B-A10B | – | – | – | 2025-12 | – |
| Qwen3.5-397B-A17B | – | – | – | 2025-04 | – |
| GPT-6-Astra | 2026-04-30 | – | Partial | 2026-06 | +2 |
| Model | Macro | Width | Sector | Width | Path | Width | Asset | Width |
|---|---|---|---|---|---|---|---|---|
| Qwen3.5-122B-A10B | 2025-03 | Tight | Unresolved | – | Unresolved | – | ND | – |
| Qwen3.5-397B-A17B | 2025-04 | Tight | 2025-08 | Wide | Unresolved | – | ND | – |
| Qwen3.8-27B | 2026-03 | Tight | ND | – | ND | – | ND | – |
| DeepSeek-V4.1-Flash-1017 | 2026-08 b | Tight | 2026-07 b | Tight | 2026-04 | Tight | Unresolved | – |
| GPT-5.6-Sol | 2026-06 b | Tight | 2026-03 | Tight | 2026-04 | Tight | 2026-06 b | Tight |
| GPT-6-Astra | 2026-07 b | Tight | 2026-06 | Tight | 2026-05 | Tight | 2026-06 | Tight |
| Model | Market | Acc. | Overall FinCut Boundary | Width | Leading-Axis Boundary | Axis Width | Acc. | |
| Open-Weight Models | ||||||||
| DeepSeek-V4.1-Flash-1017 a | CN | 44.11 | 2026-07 | Tight | 2026-06 | Macro/Path: Tight | ||
| US | 45.36 | 2026-06 | Tight | 2026-08 | Macro: Tight | |||
| Qwen3.8-27B | CN | 31.46 | 2026-03 | Tight | 2026-03 | Macro: Tight | ||
| US | 31.31 | 2026-02 | Tight | ND | – | |||
| Qwen3.5-122B-A10B | CN | 28.10 | 2025-08 | Wide | 2025-02 | Macro: Tight | ||
| CC | IDK-Enabled | |||
|---|---|---|---|---|
| Model | Acc. | Acc. | IDK | Sel. |
| GPT-5.6-Sol | 41.7 | 9.5 | 87.4 | 75.6 |
| GPT-5.6-Terra | 38.0 | 18.9 | 63.7 | 52.2 |
| GPT-5.6-Luna | 35.6 | 9.8 | 82.9 | 57.0 |
| GPT-6-Astra | 53.8 | 3.5 | 96.4 | 98.4 |
| DeepSeek-V4.1-Flash-1017 | 37.5 | 10.8 | 84.6 | 69.9 |
| Model | Start | Boundary | Pre span | Post span | |
|---|---|---|---|---|---|
| Qwen3.8-27B | 2025-12 | 2026-03 | 2025-06–2025-11 | 2026-04–2026-09 | 6/6 |
| GPT-5.6-Sol | 2025-11 | 2026-06 | 2025-05–2025-10 | 2026-07–2026-09 | 6/3 |
| GPT-5.6-Luna | 2025-10 | 2026-06 | 2025-04–2025-09 | 2026-07–2026-09 | 6/3 |
| Claude-Haiku-5.5 | 2025-10 | 2026-06 | 2025-04–2025-09 | 2026-07–2026-09 | 6/3 |
| (A) Market-level inference | |||||
|---|---|---|---|---|---|
| Market | Pre | Post | [95% CI] | ||
| China | 14 | 0.054 | 0.003 | ||
| US | 14 | 0.047 | -0.284 | ||
| Model | Task | Target Disc. Pre Post | 72B Disc. Pre Post | Rel. [95% CI] | |
|---|---|---|---|---|---|
| Qwen3.8-27B | Recall | 20.67 13.67 | 11.00 7.00 | ||
| Plain | 16.12 31.01 | 18.42 36.31 | |||
| Accuracy-emphasis | 16.22 30.30 | 18.85 35.54 | |||
| GPT-5.6-Sol | Recall | 87.33 31.33 | 13.33 4.00 | ||
| Plain | 13.83 30.25 | 18.75 35.93 | |||
| Accuracy-emphasis | 12.85 29.32 | 19.49 35.29 |
| Model | Prompt | Target Disc. Pre Post | 72B Disc. Pre Post | Rel. [95% CI] | |
|---|---|---|---|---|---|
| Panel A: EPS amount hit rate (%) | |||||
| Qwen3.8-27B | Plain | 54.33 36.33 | 44.67 29.00 | ||
| Accuracy-emphasis | 52.00 36.67 | 42.67 29.00 | |||
| GPT-5.6-Sol | Plain | 60.22 32.33 | 44.00 24.00 | ||
| Accuracy-emphasis | 64.78 37.00 | 42.33 25.00 | |||
| GPT-5.6-Luna | Plain | 56.67 26.67 | 41.67 24.00 | ||