cs.AIOct 28, 2026
SaveInvestor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?
Organizations: Prophetlab · RethinkLab · Antai College of Economics and Management, SJTU · USTC ·
Abstract
Financial agents must form forecasts and investment judgments from decision-time evidence, with predictions evaluated against subsequent outcomes. Existing benchmarks assess financial knowledge, research workflows, or forecasting, but evaluating the full research process requires a strictly frozen information set and professional evaluation criteria. We introduce Investor's Last Exam (ILE), an equity-research benchmark led by experienced buy-side investors and fund managers. Its three components are a versioned financial database spanning 3.43 million document versions and nine source categories, supplemented by structured market data, with validated timestamps governing retrieval and computation; a task taxonomy organized around professional analyst deliverables; and an expert-led framework linking scoring criteria to sources, calculations, and assumptions through financial dependency graphs. Evaluation spans 86 assignments and jointly assesses research quality, forecast accuracy, and simulated investment performance. GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100, yet even Astra scores only 52.5 on Forecast & Model. Our results show that even state-of-the-art models fail to translate overall research capability into predictive performance. ILE shifts the evaluation unit from isolated capabilities to the evidence--assumption--judgment chain, allowing researchers to locate where analytical advantages fail to translate into forecasts and investment judgments, and providing a traceable, diagnostic foundation for training and evaluating financial agents.
Figures & tables
Figure 1: Overall performance and relative strengths. (a) Overall scores; (b) answer-quality dimensions; (c) professional task categories. Scores use GPT-6 Astra grading. Each radar axis spans the lowest (inner ring) to highest (outer ring) score among highlighted models. Table reports the detailed scores.
Figure 2: Cutoff-constrained financial research and private evaluation. The agent analyzes sources available before . In the refinancing example, drawable credit depends on loan covenants. Private references and rubrics assess research quality and designated outcomes; later disclosures are accessible only to evaluation.
| Data sources | Agent tasks | Evaluation | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Benchmark | Year | Issuer docs | News / web | Market data | Agent loop | Forecast tasks | Source trace | Cutoff control | Process rubric |
| FinQA [ 7 ] | 2021 | – | – | – | – | – | – | ||
| FinanceBench [ 17 ] | 2023 | – | – | – | – | – | – | ||
| Finance Agent [ 2 ] | 2025 | – | – | – | |||||
| FinSearchComp [ 16 ] | 2025 | – | – | – | – | ||||
| StockBench [ 6 ] | 2025 | – | – | – | – | ||||
Table 1: Design and evaluation features of financial benchmarks. Year denotes the first public release.
Figure 3: Expert-led construction with AI-assisted search and review. Experts frame questions, build evidence-linked financial dependencies, and refine questions and rubrics. The refinancing graph illustrates a covenant constraint on drawable credit. Microsoft excerpts and the abbreviated O58 question and rubric show how reporting definitions enter task design. Reference analyses and rubrics remain private.
| Task category | Representative deliverable | Tasks |
|---|---|---|
| Discover & Frame | Opportunity screen, catalyst map, or testable thesis | 10 |
| Research & Evidence | Comparable historical series or reconciled facts | 13 |
| Forecast & Model | Operating forecast, earnings bridge, or peer read-through | 13 |
| Evaluate & Decide | Valuation, implied expectations, or investment action | 14 |
| Monitor & Update | Event assessment, revised forecast, or thesis update | 17 |
| Integrated Research | Report linking forecasts, stance, risks, and thesis conditions | 11 |
Table 2: Tasks organized by analyst deliverable. Categories specify the work assigned; scoring dimensions assess requirements within its answer.
Figure 4: Information validity and financial interpretation. Left: in Tesla task O74, GPT-6 Astra’s public-web Codex search returns realized Q2 deliveries from a post-cutoff source despite a dated query. Right: higher AEP operating cash inflow coexists with a wider pre-financing shortfall. AEP amounts are USD millions; investing use and construction denote outflows. Each AEP badge counts how many of the 11 evaluated systems earn full credit on every criterion in the indicated group of this task’s rubric, under GPT-6 Astra grading.
| (a) Scoring dimensions: quality within answers | ||||||||
|---|---|---|---|---|---|---|---|---|
| Rubric dimensions (across applicable tasks) | ||||||||
| System | Overall | Result | Evidence | Logic | Calc. | Recon. | Decision | Uncert. |
| GPT-6 Astra | 68.1 | 59.9 | 79.0 | 77.8 | 89.4 | 86.8 | 84.4 | 83.6 |
| Claude Opus 5.5 | 66.9 | 60.6 | 78.8 | 77.5 | 90.1 | 79.8 | 77.8 | 80.1 |
| DeepSeek V4.1 Flash | 63.0 | 58.5 | 76.2 | 72.0 | 82.5 | 78.8 | 74.5 | 81.0 |
| GLM 5.3 | 62.3 | 56.9 | 79.0 | 71.3 | 82.1 | 84.3 | 74.1 | 76.3 |
Table 3: Answer-based performance by scoring dimension (a) and task category (b). Final answers are graded independently of submission acceptance, using GPT-6 Astra for semantic criteria. Panel (a) averages task-normalized dimension credit over applicable tasks; (b) averages complete task scores within each category. Overall weights tasks equally. Cost is answering-model API expenditure in USD per task; bold marks the highest displayed score.
Figure 5: Score, cost, and tool use. (a) Overall score versus API cost per task. (b) Mean tool calls, grouping table operations.
| System | Return | Excess | MDD |
| Hunyuan 4 Preview | 2.72 | ||
| DeepSeek V4.1 Flash | 3.65 | ||
| GPT-6 Astra | 1.50 | ||
| Kimi K3 | 4.02 | ||
| Claude Opus 5.5 | 2.94 | ||
| GLM 5.3 | 3.53 |
Table 4: Forecast-driven stock returns, March–August 2026. Return and maximum drawdown (MDD) are reported in percent, and excess return relative to always long in percentage points.
| System | Answer- based | Strict | Change |
|---|---|---|---|
| Step 5 Preview | 55.12 | 34.46 | |
| Hunyuan 4 Preview | 60.39 | 47.73 | |
| DeepSeek V4.1 Flash | 62.99 | 53.03 | |
| Claude Opus 5.5 | 66.87 | 62.69 | |
| Ling 3.0 Flash | 42.59 | 40.93 |
Table 5: Score drops under strict submission requirements. Only affected systems are shown; change is strict minus answer-based score.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Data type | Document versions |
|---|---|
| Financial reports | 253,055 |
| Financial statements | 876,903 |
| Earnings calls | 136,651 |
| Investor materials | 49,601 |
| Issuer events | 356,574 |
| Analyst estimates | 425,380 |
Table 6: Document-version counts by source category, 15 October 2026.
| Category / format | Analyst deliverable | Professional judgment | |
|---|---|---|---|
| Discover & Frame | 10 | Screened opportunities, catalyst calendars, a thesis, or a driver map. | Identify economically material questions and testable investment hypotheses. |
| Research & Evidence | 13 | Reconciled facts, historical series, unit economics, or supply-chain relationships. | Resolve definitions and accounting scope; establish which evidence supports a claim. |
| Forecast & Model | 13 | Operating forecasts, earnings bridges, financial statements, or peer read-throughs. | Link drivers to financial outputs; distinguish observed signals from assumptions. |
| Evaluate & Decide | 14 | Valuation, implied expectations, surprise probabilities, or an action under stated payoffs. | Compare consistent measures and translate uncertainty into an investment decision. |
| Monitor & Update | 17 | Material-event assessments, revised forecasts, thesis tests, or position adjustments. | Identify incremental information and revise a prior without double-counting. |
| Integrated Research | 11 | A research report connecting forecasts, investment stance, risks, and thesis conditions. | Maintain consistency from evidence through model to recommendation. |
Table 7: Professional deliverables and judgments in each task category. The first five categories contain twenty subtypes; integrated research combines them, and Market Trading evaluates realized market outcomes.
| Professional subtype | Analytical output |
|---|---|
| Discover & Frame | |
| 1.1 Screening & ranking | A ranked opportunity set under explicit investment criteria. |
| 1.2 Catalyst calendars | Dated events and their possible effects on the investment thesis. |
| 1.3 Thesis evidence | An investment hypothesis with supporting and opposing evidence. |
| 1.4 Drivers & sensitivity | Material operating or financial drivers and comparable sensitivities. |
| Research & Evidence | |
Table 8: Twenty analyst-work subtypes, integrated research, and three market-forecasting subtypes. Descriptions specify the intended analytical output.
| Dimension | Assessed requirement | Example from a task rubric |
|---|---|---|
| Result | Accuracy or completeness of the requested numerical targets, rankings, or other specified outputs. | AWS O15: the revenue point forecast is scored against realized segment net sales. |
| Evidence | Support from eligible sources using the required entity, period, unit, and reporting basis. | Screening O01: growth inputs use each issuer’s as-reported rate and latest completed fiscal year. |
| Financial logic | Financial definitions, economic mechanisms, and assumptions that justify an inference or forecast. | AWS O15: each forecast adjustment has a magnitude justified through demand, capacity, pricing, or timing. |
| Calculation | Numerical execution of a stated transformation or formula using the specified inputs and units. | AEP N27: the pre-financing shortfall equals investing cash use less operating cash flow. |
| Reconciliation | Agreement among linked quantities under accounting identities, component totals, or financial bridges. | AEP N27: net cash flows reconcile to the cash increase, and opening cash plus that increase equals closing cash. |
| Decision consistency | Agreement of conclusions, rankings, or actions with the analysis, stated rules, and investment mandate. | Screening O01: the written shortlist matches the qualification flags and ranking in the screen. |
Table 9: Rubric dimensions: assessed requirements and examples. Dimensions describe requirements within answers; professional task categories describe whole assignments.
| Model | Report | Market | Combined | Strict zeros | Strict score |
|---|---|---|---|---|---|
| Hunyuan 4 Preview | 16 (43.2%) | 1 (12.5%) | 17 (37.8%) | 17 | 47.7 |
| DeepSeek V4.1 Flash | 14 (37.8%) | 0 | 14 (31.1%) | 14 | 53.0 |
| Ling 3.0 Flash | 5 (13.5%) | 2 (25.0%) | 7 (15.6%) | 10 | 40.9 |
| Step 5 Preview | 1 (2.7%) | 0 | 1 (2.2%) | 32 | 34.5 |
| GPT-6 Astra | 0 | 0 | 0 | 0 | 68.1 |
| Claude Opus 5.5 | 0 | 0 | 0 | 5 | 62.7 |
Table 10: Submission omissions and strict scores. Report and Market columns count tasks with no submit_answer call among 37 report and eight market tasks. Strict zeros cover omitted and rejected submissions across all tasks.
| (a) Scoring dimensions: quality within answers | ||||||||
|---|---|---|---|---|---|---|---|---|
| System | Overall | Result | Evidence | Logic | Calc. | Recon. | Decision | Uncert. |
| GPT-6 Astra | 68.1 | 59.9 | 79.0 | 77.8 | 89.4 | 86.8 | 84.4 | 83.6 |
| Claude Opus 5.5 | 62.7 | 57.8 | 71.2 | 69.7 | 81.6 | 71.3 | 71.4 | 73.2 |
| GLM 5.3 | 62.3 | 56.9 | 79.0 | 71.3 | 82.1 | 84.3 | 74.1 | 76.3 |
| Qwen 3.8 Max | 62.1 | 55.8 | 81.1 | 74.6 | 86.5 | 80.3 | 70.0 | 71.9 |
| DeepSeek V4 Pro | 59.3 | 54.3 | 70.1 | 69.2 | 81.5 | 82.1 | 71.7 | 75.1 |
Table 11: Strict submission evaluation. An accepted submit_answer call is required; submission failures receive zero. Both panels use the same tasks, criteria, and aggregation as Table . Bold marks the highest displayed score.
Figure 6: Scores across expert revisions. Equal-task means for two fixed solvers on matched assignments. The first two revisions regrade existing answers; the final stage obtains answers to revised questions.
| Solver | Initial | Validity repair | Forecast scoring | Question revision |
|---|---|---|---|---|
| GPT-6 Astra | 68.93 | 69.85 | 66.70 | 66.12 |
| Qwen 3.8 27B | 63.81 | 64.20 | 60.86 | 58.99 |
| Panel mean | 66.37 | 67.03 | 63.78 | 62.56 |
Table 12: Matched revision scores on the benchmark’s structured research assignments. All stages use the same tasks and solvers, with interval credit computed from the specified endpoints.
Figure 7: Per-model responses to expert refinement. All curves follow the same benchmark assignments through four versions, using the same Astra judging protocol.
Figure 8: Mean rubric-score profiles of the remaining three systems. The underlying dimension scores use the aggregation in Table (a); this panel shows absolute scores on a radial scale of 40–95 (center = 40).
| System | Combined | Point | Interval | Probability | Market |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | 40.3 | 31.0 | 28.2 | 62.0 | 58.5 |
| Hunyuan 4 Preview | 37.9 | 27.9 | 39.4 | 55.7 | 61.7 |
| GPT-6 Astra | 37.7 | 27.7 | 30.8 | 78.5 | 57.0 |
| Claude Opus 5.5 | 37.6 | 27.8 | 33.6 | 80.3 | 56.3 |
| Kimi K3 | 37.2 | 27.9 | 21.1 | 49.9 | 58.7 |
| DeepSeek V4 Pro | 35.0 | 23.4 | 39.4 | 61.3 | 64.6 |
Table 13: Credit on realized forecast outcomes. Scores use the final answers and declared numerical scoring rules. Each column averages task-normalized outcome credit over its applicable tasks. Bold marks the highest score.
Figure 9: Forecast errors despite correct supporting analysis. Absolute Tesla forecast errors divided by each metric’s zero-credit error limit. Every marker lies beyond one. All three systems earn full supporting-analysis credit but zero forecast-outcome credit.
| Case | GPT-6 Astra | Qwen 3.8 27B |
|---|---|---|
| Microsoft | ||
| JPMorgan | ||
| AWS |
Table 14: Matched scores for three financial cases under Astra grading. Each cell is before after for the same solver. Microsoft and JPMorgan retain their scoring clauses; AWS revises both question and rubric.
| Benchmark | System | Rounds | Tool calls |
|---|---|---|---|
| Finance Agent Benchmark [ 2 ] | o3 | 13.5 | 12.5 |
| FrontierFinance [ 36 ] | Kimi K3 | 10.4 | 19.6 |
| ILE | Kimi K3 | 26.7 | 41.6 |
| FrontierFinance [ 36 ] | DeepSeek V4 Pro | 21.3 | 41.1 |
| ILE | DeepSeek V4 Pro | 34.1 | 52.5 |
| ILE | DeepSeek V4.1 Flash | 73.8 | 113.8 |
Table 15: Observed interaction length across financial benchmarks. Entries are mean rounds and tool calls per task. Published values come from Finance Agent Benchmark Figure 11 [ 2 ] and FrontierFinance Table 5 [ 36 ] .
| Company | Assignments | Disclosure dates (2026) |
|---|---|---|
| Alphabet | O19, O21, O22 | Apr 29; Jul 22 |
| Apple | O05, O07, O09 | Jul 30 |
| Amazon | O12 | Jul 30 |
| Clean Harbors | N40 | Jul 29 |
| JPMorgan | O26 | Jul 14 |
| McDonald’s | O43, O44, O45, O46 | Aug 4 |
Table 16: Forecast opportunities used in the stock-return simulation. Dates identify the disclosures; multiple assignments can share one event. Tesla includes deliveries and earnings.
| Solver | Astra judge | OSS judge |
|---|---|---|
| GPT-6 Astra | 68.1 | 70.3 |
| Claude Opus 5.5 | 66.9 | 68.6 |
| DeepSeek V4.1 Flash | 63.0 | 70.1 |
| GLM 5.3 | 62.3 | 65.8 |
| Qwen 3.8 Max | 62.1 | 68.6 |
| Hunyuan 4 Preview | 60.4 | 67.9 |
Table 17: Judge model comparison on fixed answers. Columns report equal-task mean scores over the shared tasks, using Astra or OSS for semantic grading under answer-based evaluation. Systems are ordered by the primary Astra scores.
| Overall | Result | Forecast outcomes | ||||
|---|---|---|---|---|---|---|
| System | ILE | Codex | ILE | Codex | ILE | Codex |
| GPT-6 Astra | 68.1 | 68.3 | 59.9 | 60.5 | 37.7 | 40.9 |
| Qwen 3.8 Max | 62.1 | 58.5 | 55.8 | 52.8 | 34.5 | 33.9 |
| Hunyuan 4 Preview | 60.4 | 58.8 | 56.7 | 54.7 | 37.9 | 38.9 |
Table 18: Controlled-database and public-web research. ILE enforces source availability; Codex uses public-web access with cutoffs specified in the prompt. Scores use the 86 shared questions, rubrics, and Astra semantic judge.
| Model | Tasks | Partial credit | All-pass |
|---|---|---|---|
| DeepSeek V4 Flash | 27 | 59.9 | 44.4 |
Table 19: Shared-tool performance on public FAB v2 tasks. Scores are percentages.
| Tool | Arguments, behavior, and returned information |
|---|---|
| search | Required: query (string). Optional: mode ( text or metadata ), datasets , subtypes , markets , filing_forms , period , published_from , version_policy , limit , cursor . Metadata search matches company names, symbols, and titles; text search uses a two-to-ten-word content phrase. Period filters describe reporting periods, while published_from is a publication-time lower bound. version_policy selects the latest visible revision or all visible revisions; limit is 1–100 (default 20). Results contain document_ref , formats, snippets, and recommended next actions. |
| read | Required: query and either document_ref or sources . Optional per source: format , unit , start , count , cursor . A sources array groups up to eight documents or ranges under a shared reading budget. Formats are Markdown, plain text, JSON, or PNG; units are pages, blocks, chunks, or rows. Omitting ranges requests budgeted full-document reading. Explicit ranges start at 1 and allow up to 10 pages/chunks, 100 blocks, or 200 rows; a PNG read covers one page separately. Results identify covered ranges and continuation fields and provide evidence-focused reading for the supplied query. |
| download | Required: document_ref . Optional: format , page_number , filename . Downloads an available source representation into the task workspace. Parquet is preferred for tabular analysis. A PDF-to-PNG request identifies one page; unsupported readable formats can be inspected through the downloaded source. The returned workspace path can be used by bash . |
| bash | Required: command (string). Optional: timeout (1–300 seconds, default 300). Runs file inspection, Python analysis, arithmetic, or deterministic checks in the isolated workspace, with network access disabled. Available libraries include NumPy, SciPy, pandas, openpyxl, PyArrow, and DuckDB. Returns command output and execution status. |
Table 20: Document and computation tools. Required arguments are listed first; optional arguments follow. Data access is limited to the task’s authorized sources and information cutoff.
| Interface | Arguments, behavior, and returned information |
|---|---|
| tables | Optional: table_id . With no argument, lists available tables and their datasets and titles. A returned table identifier requests fields, units, keys, and revision semantics before a query is written. Existing results keep their original catalogue snapshot. |
| table_lookup | Required: query , dataset . Optional: market , limit (1–100, default 50). Resolves a registered company name or code in financial statements or analyst estimates. Returns dated identity candidates; provider identifiers are distinguished from exchange symbols. |
| table_query | Required: sql . Optional: parameters , count (1–200, default 50). Executes one read-only SELECT over discovered tables, supporting screening, joins, aggregates, and windows. Named value placeholders use $name ; table and column names come from the discovered schema. Returns a table_ref and query status. The preview count does not limit the complete result. |
| table_read | Required: table_ref . Optional: cursor , count (1–200). Checks query status or reads one page of the frozen result. Continuation uses the returned cursor unchanged. Decimal values and large integers remain strings; total_rows counts query rows. |
| table_cancel | Required: table_ref . Cancels an unfinished query from the current run and returns its cancellation status. |
| table_download | Required: table_ref . Optional: format (Parquet or JSON, default Parquet), filename . Downloads the complete frozen result into the workspace used by bash . |
Table 21: Structured-table interfaces. Queries and their results retain the task’s authorization and availability boundary. A returned table_ref identifies a frozen result shared by reading, cancellation, and download operations.
| Deliverable | Required and optional fields |
|---|---|
| Structured analysis | Required: nonempty answer_summary and a nonempty structured_output object with the question’s specified fields. Optional: evidence , calculations , artifacts , and uncertainties . Each evidence item records document_ref , locator , excerpt , and claim . Each calculation records description , expression , and result , with optional inputs . Each artifact records workspace_path , artifact_type , and description ; uncertainties are strings. Evidence uses actual document or completed table identifiers. |
| Research report | A single nonempty answer string containing the complete report. |
| Price forecast | A single positive numeric forecast_close_usd value. |
| Return or volatility ranking | A ranking array containing every member of the task’s fixed universe exactly once, in the requested order. |
Table 22: Task-specific schemas for submit_answer . Each schema accepts the fields declared for that deliverable. The question supplies the required financial content.