Financial agents must form forecasts and investment judgments from decision-time evidence, with predictions evaluated against subsequent outcomes. Existing benchmarks assess financial knowledge, research workflows, or forecasting, but evaluating the full research process requires a strictly frozen information set and professional evaluation criteria. We introduce Investor's Last Exam (ILE), an equity-research benchmark led by experienced buy-side investors and fund managers. Its three components are a versioned financial database spanning 3.43 million document versions and nine source categories, supplemented by structured market data, with validated timestamps governing retrieval and computation; a task taxonomy organized around professional analyst deliverables; and an expert-led framework linking scoring criteria to sources, calculations, and assumptions through financial dependency graphs. Evaluation spans 86 assignments and jointly assesses research quality, forecast accuracy, and simulated investment performance. GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100, yet even Astra scores only 52.5 on Forecast & Model. Our results show that even state-of-the-art models fail to translate overall research capability into predictive performance. ILE shifts the evaluation unit from isolated capabilities to the evidence--assumption--judgment chain, allowing researchers to locate where analytical advantages fail to translate into forecasts and investment judgments, and providing a traceable, diagnostic foundation for training and evaluating financial agents.
Figures & tables
Figure 1: Overall performance and relative strengths. (a) Overall scores; (b) answer-quality dimensions; (c) professional task categories. Scores use GPT-6 Astra grading. Each radar axis spans the lowest (inner ring) to highest (outer ring) score among highlighted models. Table reports the detailed scores.
Figure 2: Cutoff-constrained financial research and private evaluation. The agent analyzes sources available before t0 . In the refinancing example, drawable credit depends on loan covenants. Private references and rubrics assess research quality and designated outcomes; later disclosures are accessible only to evaluation.
Data sources
Agent tasks
Evaluation
Benchmark
Year
Issuer docs
News / web
Market data
Agent loop
Forecast tasks
Source trace
Cutoff control
Process rubric
FinQA [ 7 ]
2021
✓
–
–
–
–
✓
–
–
FinanceBench [ 17 ]
2023
✓
–
–
–
–
✓
–
–
Finance Agent [ 2 ]
2025
✓
✓
–
✓
–
✓
–
✓
FinSearchComp [ 16 ]
2025
✓
✓
✓
✓
–
–
–
–
StockBench [ 6 ]
2025
–
✓
✓
✓
–
–
✓
–
Table 1: Design and evaluation features of financial benchmarks. Year denotes the first public release.
Figure 3: Expert-led construction with AI-assisted search and review. Experts frame questions, build evidence-linked financial dependencies, and refine questions and rubrics. The refinancing graph illustrates a covenant constraint on drawable credit. Microsoft excerpts and the abbreviated O58 question and rubric show how reporting definitions enter task design. Reference analyses and rubrics remain private.
Task category
Representative deliverable
Tasks
Discover & Frame
Opportunity screen, catalyst map, or testable thesis
10
Research & Evidence
Comparable historical series or reconciled facts
13
Forecast & Model
Operating forecast, earnings bridge, or peer read-through
13
Evaluate & Decide
Valuation, implied expectations, or investment action
14
Monitor & Update
Event assessment, revised forecast, or thesis update
17
Integrated Research
Report linking forecasts, stance, risks, and thesis conditions
11
Table 2: Tasks organized by analyst deliverable. Categories specify the work assigned; scoring dimensions assess requirements within its answer.
Figure 4: Information validity and financial interpretation. Left: in Tesla task O74, GPT-6 Astra’s public-web Codex search returns realized Q2 deliveries from a post-cutoff source despite a dated query. Right: higher AEP operating cash inflow coexists with a wider pre-financing shortfall. AEP amounts are USD millions; investing use and construction denote outflows. Each AEP badge counts how many of the 11 evaluated systems earn full credit on every criterion in the indicated group of this task’s rubric, under GPT-6 Astra grading.
(a) Scoring dimensions: quality within answers
Rubric dimensions (across applicable tasks)
System
Overall
Result
Evidence
Logic
Calc.
Recon.
Decision
Uncert.
GPT-6 Astra
68.1
59.9
79.0
77.8
89.4
86.8
84.4
83.6
Claude Opus 5.5
66.9
60.6
78.8
77.5
90.1
79.8
77.8
80.1
DeepSeek V4.1 Flash
63.0
58.5
76.2
72.0
82.5
78.8
74.5
81.0
GLM 5.3
62.3
56.9
79.0
71.3
82.1
84.3
74.1
76.3
Table 3: Answer-based performance by scoring dimension (a) and task category (b). Final answers are graded independently of submission acceptance, using GPT-6 Astra for semantic criteria. Panel (a) averages task-normalized dimension credit over applicable tasks; (b) averages complete task scores within each category. Overall weights tasks equally. Cost is answering-model API expenditure in USD per task; bold marks the highest displayed score.
Figure 5: Score, cost, and tool use. (a) Overall score versus API cost per task. (b) Mean tool calls, grouping table operations.
System
Return
Excess
MDD
Hunyuan 4 Preview
+8.50
+7.32
2.72
DeepSeek V4.1 Flash
+7.98
+6.80
3.65
GPT-6 Astra
+7.18
+6.00
1.50
Kimi K3
+7.15
+5.97
4.02
Claude Opus 5.5
+6.39
+5.21
2.94
GLM 5.3
+6.19
+5.01
3.53
Table 4: Forecast-driven stock returns, March–August 2026. Return and maximum drawdown (MDD) are reported in percent, and excess return relative to always long in percentage points.
System
Answer- based
Strict
Change
Step 5 Preview
55.12
34.46
−20.66
Hunyuan 4 Preview
60.39
47.73
−12.66
DeepSeek V4.1 Flash
62.99
53.03
−9.96
Claude Opus 5.5
66.87
62.69
−4.18
Ling 3.0 Flash
42.59
40.93
−1.66
Table 5: Score drops under strict submission requirements. Only affected systems are shown; change is strict minus answer-based score.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Data type
Document versions
Financial reports
253,055
Financial statements
876,903
Earnings calls
136,651
Investor materials
49,601
Issuer events
356,574
Analyst estimates
425,380
Appendix
Table 6: Document-version counts by source category, 15 October 2026.
Category / format
n
Analyst deliverable
Professional judgment
Discover & Frame
10
Screened opportunities, catalyst calendars, a thesis, or a driver map.
Identify economically material questions and testable investment hypotheses.
Research & Evidence
13
Reconciled facts, historical series, unit economics, or supply-chain relationships.
Resolve definitions and accounting scope; establish which evidence supports a claim.
Forecast & Model
13
Operating forecasts, earnings bridges, financial statements, or peer read-throughs.
Link drivers to financial outputs; distinguish observed signals from assumptions.
Evaluate & Decide
14
Valuation, implied expectations, surprise probabilities, or an action under stated payoffs.
Compare consistent measures and translate uncertainty into an investment decision.
Monitor & Update
17
Material-event assessments, revised forecasts, thesis tests, or position adjustments.
Identify incremental information and revise a prior without double-counting.
Integrated Research
11
A research report connecting forecasts, investment stance, risks, and thesis conditions.
Maintain consistency from evidence through model to recommendation.
Appendix
Table 7: Professional deliverables and judgments in each task category. The first five categories contain twenty subtypes; integrated research combines them, and Market Trading evaluates realized market outcomes.
Professional subtype
Analytical output
Discover & Frame
1.1 Screening & ranking
A ranked opportunity set under explicit investment criteria.
1.2 Catalyst calendars
Dated events and their possible effects on the investment thesis.
1.3 Thesis evidence
An investment hypothesis with supporting and opposing evidence.
1.4 Drivers & sensitivity
Material operating or financial drivers and comparable sensitivities.
Research & Evidence
Appendix
Table 8: Twenty analyst-work subtypes, integrated research, and three market-forecasting subtypes. Descriptions specify the intended analytical output.
Dimension
Assessed requirement
Example from a task rubric
Result
Accuracy or completeness of the requested numerical targets, rankings, or other specified outputs.
AWS O15: the revenue point forecast is scored against realized segment net sales.
Evidence
Support from eligible sources using the required entity, period, unit, and reporting basis.
Screening O01: growth inputs use each issuer’s as-reported rate and latest completed fiscal year.
Financial logic
Financial definitions, economic mechanisms, and assumptions that justify an inference or forecast.
AWS O15: each forecast adjustment has a magnitude justified through demand, capacity, pricing, or timing.
Calculation
Numerical execution of a stated transformation or formula using the specified inputs and units.
AEP N27: the pre-financing shortfall equals investing cash use less operating cash flow.
Reconciliation
Agreement among linked quantities under accounting identities, component totals, or financial bridges.
AEP N27: net cash flows reconcile to the cash increase, and opening cash plus that increase equals closing cash.
Decision consistency
Agreement of conclusions, rankings, or actions with the analysis, stated rules, and investment mandate.
Screening O01: the written shortlist matches the qualification flags and ranking in the screen.
Appendix
Table 9: Rubric dimensions: assessed requirements and examples. Dimensions describe requirements within answers; professional task categories describe whole assignments.
Model
Report n=37
Market n=8
Combined n=45
Strict zeros
Strict score
Hunyuan 4 Preview
16 (43.2%)
1 (12.5%)
17 (37.8%)
17
47.7
DeepSeek V4.1 Flash
14 (37.8%)
0
14 (31.1%)
14
53.0
Ling 3.0 Flash
5 (13.5%)
2 (25.0%)
7 (15.6%)
10
40.9
Step 5 Preview
1 (2.7%)
0
1 (2.2%)
32
34.5
GPT-6 Astra
0
0
0
0
68.1
Claude Opus 5.5
0
0
0
5
62.7
Appendix
Table 10: Submission omissions and strict scores. Report and Market columns count tasks with no submit_answer call among 37 report and eight market tasks. Strict zeros cover omitted and rejected submissions across all tasks.
(a) Scoring dimensions: quality within answers
System
Overall
Result
Evidence
Logic
Calc.
Recon.
Decision
Uncert.
GPT-6 Astra
68.1
59.9
79.0
77.8
89.4
86.8
84.4
83.6
Claude Opus 5.5
62.7
57.8
71.2
69.7
81.6
71.3
71.4
73.2
GLM 5.3
62.3
56.9
79.0
71.3
82.1
84.3
74.1
76.3
Qwen 3.8 Max
62.1
55.8
81.1
74.6
86.5
80.3
70.0
71.9
DeepSeek V4 Pro
59.3
54.3
70.1
69.2
81.5
82.1
71.7
75.1
Appendix
Table 11: Strict submission evaluation. An accepted submit_answer call is required; submission failures receive zero. Both panels use the same tasks, criteria, and aggregation as Table . Bold marks the highest displayed score.
Figure 6: Scores across expert revisions. Equal-task means for two fixed solvers on matched assignments. The first two revisions regrade existing answers; the final stage obtains answers to revised questions.
Solver
Initial
Validity repair
Forecast scoring
Question revision
GPT-6 Astra
68.93
69.85
66.70
66.12
Qwen 3.8 27B
63.81
64.20
60.86
58.99
Panel mean
66.37
67.03
63.78
62.56
Appendix
Table 12: Matched revision scores on the benchmark’s structured research assignments. All stages use the same tasks and solvers, with interval credit computed from the specified endpoints.
Figure 7: Per-model responses to expert refinement. All curves follow the same benchmark assignments through four versions, using the same Astra judging protocol.
Figure 8: Mean rubric-score profiles of the remaining three systems. The underlying dimension scores use the aggregation in Table (a); this panel shows absolute scores on a radial scale of 40–95 (center = 40).
System
Combined
Point
Interval
Probability
Market
DeepSeek V4.1 Flash
40.3
31.0
28.2
62.0
58.5
Hunyuan 4 Preview
37.9
27.9
39.4
55.7
61.7
GPT-6 Astra
37.7
27.7
30.8
78.5
57.0
Claude Opus 5.5
37.6
27.8
33.6
80.3
56.3
Kimi K3
37.2
27.9
21.1
49.9
58.7
DeepSeek V4 Pro
35.0
23.4
39.4
61.3
64.6
Appendix
Table 13: Credit on realized forecast outcomes. Scores use the final answers and declared numerical scoring rules. Each column averages task-normalized outcome credit over its applicable tasks. Bold marks the highest score.
Figure 9: Forecast errors despite correct supporting analysis. Absolute Tesla forecast errors divided by each metric’s zero-credit error limit. Every marker lies beyond one. All three systems earn full supporting-analysis credit but zero forecast-outcome credit.
Case
GPT-6 Astra
Qwen 3.8 27B
Microsoft
73.4→71.0
78.1→75.2
JPMorgan
38.5→48.7
69.2→48.7
AWS
49.1→18.2
64.9→13.4
Appendix
Table 14: Matched scores for three financial cases under Astra grading. Each cell is before → after for the same solver. Microsoft and JPMorgan retain their scoring clauses; AWS revises both question and rubric.
Benchmark
System
Rounds
Tool calls
Finance Agent Benchmark [ 2 ]
o3
13.5
12.5
FrontierFinance [ 36 ]
Kimi K3
10.4
19.6
ILE
Kimi K3
26.7
41.6
FrontierFinance [ 36 ]
DeepSeek V4 Pro
21.3
41.1
ILE
DeepSeek V4 Pro
34.1
52.5
ILE
DeepSeek V4.1 Flash
73.8
113.8
Appendix
Table 15: Observed interaction length across financial benchmarks. Entries are mean rounds and tool calls per task. Published values come from Finance Agent Benchmark Figure 11 [ 2 ] and FrontierFinance Table 5 [ 36 ] .
Company
Assignments
Disclosure dates (2026)
Alphabet
O19, O21, O22
Apr 29; Jul 22
Apple
O05, O07, O09
Jul 30
Amazon
O12
Jul 30
Clean Harbors
N40
Jul 29
JPMorgan
O26
Jul 14
McDonald’s
O43, O44, O45, O46
Aug 4
Appendix
Table 16: Forecast opportunities used in the stock-return simulation. Dates identify the disclosures; multiple assignments can share one event. Tesla includes deliveries and earnings.
Solver
Astra judge
OSS judge
GPT-6 Astra
68.1
70.3
Claude Opus 5.5
66.9
68.6
DeepSeek V4.1 Flash
63.0
70.1
GLM 5.3
62.3
65.8
Qwen 3.8 Max
62.1
68.6
Hunyuan 4 Preview
60.4
67.9
Appendix
Table 17: Judge model comparison on fixed answers. Columns report equal-task mean scores over the shared tasks, using Astra or OSS for semantic grading under answer-based evaluation. Systems are ordered by the primary Astra scores.
Overall
Result
Forecast outcomes
System
ILE
Codex
ILE
Codex
ILE
Codex
GPT-6 Astra
68.1
68.3
59.9
60.5
37.7
40.9
Qwen 3.8 Max
62.1
58.5
55.8
52.8
34.5
33.9
Hunyuan 4 Preview
60.4
58.8
56.7
54.7
37.9
38.9
Appendix
Table 18: Controlled-database and public-web research. ILE enforces source availability; Codex uses public-web access with cutoffs specified in the prompt. Scores use the 86 shared questions, rubrics, and Astra semantic judge.
Model
Tasks
Partial credit
All-pass
DeepSeek V4 Flash
27
59.9
44.4
Appendix
Table 19: Shared-tool performance on public FAB v2 tasks. Scores are percentages.
Tool
Arguments, behavior, and returned information
search
Required: query (string). Optional: mode ( text or metadata ), datasets , subtypes , markets , filing_forms , period , published_from , version_policy , limit , cursor . Metadata search matches company names, symbols, and titles; text search uses a two-to-ten-word content phrase. Period filters describe reporting periods, while published_from is a publication-time lower bound. version_policy selects the latest visible revision or all visible revisions; limit is 1–100 (default 20). Results contain document_ref , formats, snippets, and recommended next actions.
read
Required: query and either document_ref or sources . Optional per source: format , unit , start , count , cursor . A sources array groups up to eight documents or ranges under a shared reading budget. Formats are Markdown, plain text, JSON, or PNG; units are pages, blocks, chunks, or rows. Omitting ranges requests budgeted full-document reading. Explicit ranges start at 1 and allow up to 10 pages/chunks, 100 blocks, or 200 rows; a PNG read covers one page separately. Results identify covered ranges and continuation fields and provide evidence-focused reading for the supplied query.
download
Required: document_ref . Optional: format , page_number , filename . Downloads an available source representation into the task workspace. Parquet is preferred for tabular analysis. A PDF-to-PNG request identifies one page; unsupported readable formats can be inspected through the downloaded source. The returned workspace path can be used by bash .
bash
Required: command (string). Optional: timeout (1–300 seconds, default 300). Runs file inspection, Python analysis, arithmetic, or deterministic checks in the isolated workspace, with network access disabled. Available libraries include NumPy, SciPy, pandas, openpyxl, PyArrow, and DuckDB. Returns command output and execution status.
Appendix
Table 20: Document and computation tools. Required arguments are listed first; optional arguments follow. Data access is limited to the task’s authorized sources and information cutoff.
Interface
Arguments, behavior, and returned information
tables
Optional: table_id . With no argument, lists available tables and their datasets and titles. A returned table identifier requests fields, units, keys, and revision semantics before a query is written. Existing results keep their original catalogue snapshot.
table_lookup
Required: query , dataset . Optional: market , limit (1–100, default 50). Resolves a registered company name or code in financial statements or analyst estimates. Returns dated identity candidates; provider identifiers are distinguished from exchange symbols.
table_query
Required: sql . Optional: parameters , count (1–200, default 50). Executes one read-only SELECT over discovered tables, supporting screening, joins, aggregates, and windows. Named value placeholders use $name ; table and column names come from the discovered schema. Returns a table_ref and query status. The preview count does not limit the complete result.
table_read
Required: table_ref . Optional: cursor , count (1–200). Checks query status or reads one page of the frozen result. Continuation uses the returned cursor unchanged. Decimal values and large integers remain strings; total_rows counts query rows.
table_cancel
Required: table_ref . Cancels an unfinished query from the current run and returns its cancellation status.
table_download
Required: table_ref . Optional: format (Parquet or JSON, default Parquet), filename . Downloads the complete frozen result into the workspace used by bash .
Appendix
Table 21: Structured-table interfaces. Queries and their results retain the task’s authorization and availability boundary. A returned table_ref identifies a frozen result shared by reading, cancellation, and download operations.
Deliverable
Required and optional fields
Structured analysis
Required: nonempty answer_summary and a nonempty structured_output object with the question’s specified fields. Optional: evidence , calculations , artifacts , and uncertainties . Each evidence item records document_ref , locator , excerpt , and claim . Each calculation records description , expression , and result , with optional inputs . Each artifact records workspace_path , artifact_type , and description ; uncertainties are strings. Evidence uses actual document or completed table identifiers.
Research report
A single nonempty answer string containing the complete report.
Price forecast
A single positive numeric forecast_close_usd value.
Return or volatility ranking
A ranking array containing every member of the task’s fixed universe exactly once, in the requested order.
Appendix
Table 22: Task-specific schemas for submit_answer . Each schema accepts the fields declared for that deliverable. The question supplies the required financial content.
AI agents are increasingly deployed for professional investment research, yet no benchmark captures the complexity of the full investor workflow. Existing benchmarks mainly target financial data extraction, a narrow slice that current models have largely saturated, while reference-based metrics and generic LLM-as-a-judge scoring fall short on the open-ended, long-form answers that real analyst queries demand. We introduce FrontierFinance, a fully open benchmark of 220 expert-crafted queries and 11,543 source-attributed rubrics spanning six crucial use cases across the full investor workflow. FrontierFinance is both broader and harder than existing public finance benchmarks. Evaluating frontier models and agent systems under a common harness restricted to publicly available data, we find that the tool harness, not the model alone, strongly shapes quality and efficiency; that Samaya's in-house system leads at 56.0%, ahead of the strongest frontier model (Claude Fable 5, 49.2%) at roughly 2.2x lower cost; and that the best open-weight model (Kimi K3, 46.4%) nearly matches the best proprietary model at 4.5x lower cost. Screening & Discovery and Sector, Industry & Macro remain the hardest use cases across all systems, where even the best systems reach only 33% and 39%. We make the dataset and grading code publicly available.
Yuhao Zhang, O. Ozan Koyluoglu, Thejas Venkatesh +4
We introduce Deep FinResearch Bench, a practical and comprehensive evaluation framework for deep research (DR) agents in financial investment research. The benchmark assesses three dimensions of report quality: qualitative rigor, quantitative forecasting and valuation accuracy, and claim credibility and verifiability. Particularly, we define corresponding qualitative and quantitative evaluation metrics and implement an automated scoring procedure to enable scalable assessment. Applying the benchmark to financial reports from frontier DR agents and comparing them with reports authored by financial professionals, we find that AI-generated reports still fall short across these dimensions. These findings underscore the need for domain-specialized DR agents tailored to finance, and we hope the work establishes a foundation for standardized benchmarking of DR agents in financial research.
Mirazul Haque, Antony Papadimitriou, Samuel Mensah +6
We introduce Frontier Financial Judgement, a challenging new benchmark developed in collaboration with professional equity analysts to assess agents' ability to replicate expert human judgements. Rapidly identifying new information, evaluating its implications and determining its valuation impact is one of the most time-consuming and challenging aspects of real-world equity coverage. This is becoming ever more difficult and important as AI rapidly increases the quantity of new information to process. The strongest agent we evaluate on Frontier Financial Judgement matches all expert labels in only 52.4% of cases. We also find significant divergence in estimated false-positive rates among frontier agents, ranging from ~1% for GPT-5.6 Sol to ~32% for Claude Sonnet 4.6. To construct the benchmark and make it representative of real-world settings, we combine human-designed and labelled synthetic articles with live news articles and historical documents, creating 656 items for assessment. The resulting task requires agents to distinguish genuinely new, valuation-relevant financial information from stale, immaterial or misleading news under realistic conditions. We find substantial trade-offs among agent accuracy, cost, false positives and reliability that continue to hinder the reliable deployment of news-flow filtering in practice.