Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines

Akshay Gulati; Kanha Singhania; Tushar Banga; Parth Arora; Anshul Verma; Vaibhav Kumar Singh; Agyapal Digra; Jayant Singh Bisht

cs.AIMar 9, 2026

Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines

Authors: Akshay Gulati, Kanha Singhania, Tushar Banga, Parth Arora, Anshul Verma, Vaibhav Kumar Singh, Agyapal Digra, Jayant Singh Bisht, +3 more

Organizations: The Future University

Abstract

Large language models are increasingly used for financial analysis and investment research, yet systematic evaluation of their financial reasoning capabilities remains limited. In this work, we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities across five dimensions: factual accuracy, analytical completeness, data recency, model consistency, and failure patterns. We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions derived from real-world equity research tasks. The results reveal substantial differences in performance across models. Within this benchmark setting, SuperInvesting achieves the highest aggregate performance, with an average factual accuracy score of 8.96/10 and the highest completeness score of 56.65/70, while also demonstrating the lowest hallucination rate among evaluated systems. Retrieval-oriented systems such as Perplexity perform strongly on data recency tasks due to live information access but exhibit weaker analytical synthesis and consistency. Overall, the results highlight that financial intelligence in large language models is inherently multi-dimensional, and systems that combine structured financial data access with analytical reasoning capabilities provide the most reliable performance for complex investment research workflows.

Figures & tables

Benchmark	Year	FAT	NR	MDE	RFD	FA	RE
FinQA [ 14 ]	2021	✗	✓	✗	✓	✗	✗
FinanceBench [ 15 ]	2023	✗	✓	✗	✓	✗	✗
FinBen [ 16 ]	2024	✓	✓	✗	✓	✗	✗
FinanceQA [ 17 ]	2025	✓	✓	✗	✓	✗	✗
FinAuditing [ 18 ]	2025	✓	✓	✗	✓	✗	✗
Fin-RATE [ 19 ]	2026	✓	✓	✓	✓	✗	✗

TABLE I: Comparison of AFIB with existing financial LLM benchmarks.

Fig. 1: Overview of the AI Financial Intelligence Benchmark (AFIB) evaluation pipeline. Financial analysis queries are constructed from multi-sector datasets and executed across multiple AI models. Model outputs are evaluated using a multi-dimensional scoring framework capturing accuracy, analytical depth, completeness, recency, consistency, and failure patterns.

Fig. 3: Model Capability Comparison Across Financial Benchmark Dimensions

Benchmark	1st	2nd	3rd	4th	5th
Valuation Depth	SuperInvesting	Gemini	GPT	Claude	Perplexity
Consistency	SuperInvesting	Claude	Gemini	GPT	Perplexity
Accuracy / Hallucination	SuperInvesting	Gemini	Perplexity	GPT	Claude
Data Recency	Perplexity	SuperInvesting	Gemini	GPT	Claude
Completeness	SuperInvesting	Gemini	Perplexity	Claude	GPT

TABLE II: Per-Benchmark Leaderboard Across Evaluation Dimensions

Fig. 4: Trade-off Between Data Recency and Analytical Depth

Fig. 5: Hallucination Frequency Across Models

Fig. 6: Heatmap of model performance across benchmark evaluation dimensions. Higher scores (green) indicate stronger performance.

Prompt	GPT	Gemini	Perplexity	Claude	SuperInvesting
Q1: Reliance EBITDA Contribution What percentage of Reliance’s consolidated EBITDA comes from Oil-to-Chemicals, Jio, and Retail in the latest financial year?	Provided segment percentages (43%, 28%, 29%) but incorrectly concluded the three segments represent 100% of EBITDA.	Estimated segment contributions and explained structural shift toward consumer businesses. Combined estimate around 88% .	Attempted calculation using annual report figures but produced inconsistent totals due to segment aggregation issues.	Provided historical estimates from earlier financial periods without latest-year verification.	Provided structured segment analysis with contextual explanation and estimated combined contribution around 88–90% .
Q2: SBI vs ICICI NPA Comparison Compare the gross NPA and net NPA ratios of SBI and ICICI Bank for the most recent quarter and explain the role of provision coverage ratio (PCR).	Provided illustrative values and conceptual explanation of PCR but acknowledged figures were approximate.	Used recent financial results with structured explanation of GNPA, NNPA, and provisioning impact.	Retrieved figures from financial sources and presented a comparative table with PCR explanation.	Used older financial period data but correctly explained the PCR mechanism linking GNPA and NNPA.	Provided structured comparison of GNPA, NNPA, and PCR with explanation of asset quality differences and provisioning strategies.
Q3: L&T Weighted Fundamental Score Construct a weighted fundamental score (out of 100) for Larsen & Toubro using: 40% order book-to-bill ratio 30% working capital days 30% EPC EBITDA margins. Show exact calculation.	Assumed sector benchmarks and normalized metrics to compute a score of 60/100 . Used hypothetical values.	Applied financial benchmarks and calculated weighted score of 83.7/100 with detailed reasoning.	Estimated financial inputs from disclosures and derived a score of 79.3/100 .	Used sector averages and derived a score of 82/100 with emphasis on working capital efficiency.	Calculated normalized benchmark scores and produced a weighted score of 87.2/100 with detailed breakdown.

TABLE III: Example benchmark prompts and model responses used in the AI Financial Intelligence Benchmark (AFIB). Responses are summarized for readability.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Response (Excerpt)
GPT	Reported segment contributions as Oil-to-Chemicals 43%, Jio 28%, and Retail 29%, and incorrectly concluded that these segments together account for 100% of consolidated EBITDA .
Gemini	Provided a structured financial analysis estimating segment contributions around O2C $\sim$ 43%, Jio $\sim$ 32%, and Retail $\sim$ 13%, concluding that these three segments contribute approximately 88% of consolidated EBITDA .
Perplexity	Attempted a calculation using annual report figures but produced inconsistent totals due to incorrect aggregation of segment EBITDA values and reporting overlaps.
Claude	Provided historical segment estimates from earlier financial periods and explained the role of segment diversification, but used outdated financial data.
SuperInvesting	Produced a structured breakdown with contextual explanation of segment contributions (O2C $\sim$ 42–44%, Jio $\sim$ 31–33%, Retail $\sim$ 13–15%), concluding that these segments contribute approximately 88–90% of consolidated EBITDA.

Appendix

TABLE IV: Example responses for Reliance Industries EBITDA composition prompt.

Model	Response (Excerpt)
GPT	Provided approximate illustrative values for GNPA and NNPA and explained the concept of provision coverage ratio (PCR), but acknowledged that the figures were not based on the latest financial results.
Gemini	Used recent financial data to compare asset quality metrics and explained the relationship between GNPA, NNPA, and PCR in determining bank balance sheet strength.
Perplexity	Retrieved financial ratios from web sources and presented a comparison table of GNPA and NNPA values, linking the difference to provisioning behaviour.
Claude	Used financial data from an earlier reporting period but provided a clear explanation of the relationship between PCR and the GNPA–NNPA gap.
SuperInvesting	Produced a structured comparison including GNPA, NNPA, and PCR metrics, explaining how higher provisioning buffers reduce net NPA exposure and improve asset quality resilience.

Appendix

TABLE V: Example responses for SBI vs ICICI Bank asset quality analysis prompt.

Model	Response (Excerpt)
GPT	Outlined the methodology for computing ROIC and WACC and provided illustrative estimates showing ROIC consistently exceeding WACC by roughly 9–11 percentage points.
Gemini	Declined to provide a detailed calculation due to insufficient data availability and instead discussed valuation context using ROE and ROCE indicators.
Perplexity	Presented a detailed financial analysis including approximate historical ROIC and WACC values and discussed the implications of a narrowing value spread.
Claude	Estimated historical ROIC and WACC values and concluded that Asian Paints maintains a significant positive ROIC–WACC spread supporting its premium valuation.
SuperInvesting	Produced a structured capital efficiency analysis showing ROIC significantly exceeding WACC but highlighted a declining spread due to increasing competition and margin pressure.

Appendix

TABLE VI: Example responses for Asian Paints ROIC–WACC valuation analysis prompt.

Explore similar work

FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting
Feb 25, 2026Yiyun Zhu, Yidong Jiang, Ziwen Xu +4LLM Evaluation LLM Reliability
School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China
Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows
Aug 25, 2026Miao Liu, Zhizhe LiuLLM Evaluation LLM Reliability
Carroll School of Management, Boston College · Columbia Business School, Columbia University
Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain
May 9, 2026Xiaoyu Hu, Jinman ZhaoLLM Evaluation Social Bias in Language Models
Rutgers University · Department of Computer Science, University of Toronto

Feb 25, 2026J/K move · Enter open · S save

FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting

Yiyun Zhu, Yidong Jiang, Ziwen Xu, Yinsheng Yao+3

School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China

LLM Evaluation LLM Reliability Language Model Auditing

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly deployed in financial research workflows, where their role is evolving from single-model assistance for human analysts toward autonomous collaboration among multiple agents. Yet real-world deployments still expose factual errors, numerical inconsistencies, and shallow analysis, which can distort assessments of corporate fundamentals and trigger severe economic losses. While existing benchmarks have begun to evaluate such failures, they score all aspects of the generated analysis in one pass, failing to distinguish whether a model fails at foundational stages like auditing and correction, or underperforms at generating research-grade insights. Consequently, it obscures capability bottlenecks and the specialized strengths essential for multi-agent role assignment. To address these gaps, we introduce FinReasoning, a hierarchical benchmark that decomposes the core capabilities of financial research into semantic consistency, data alignment, and deep insight. We further propose a fine-grained evaluation framework that strengthens hallucination-correction assessment and incorporates a 12-indicator rubric for core analytical skills. FinReasoning reveals clear capability stratification across model types. Closed-source models (like Doubao-Seed-1.8) perform strongly overall and are better suited for core reasoning agents in multi-agent financial systems; open-source general models (like Qwen3-235B) show clear capability divergence and consistently underperform on Semantic Consistency, making them less suited for quality-sensitive generation tasks; financial-domain models (like Fin-R1) generate moderate insights but lack foundational auditing skills. Our work has already been deployed in pilot tests across several real-world scenarios. The resource is available at https://github.com/TongjiFinLab/FinReasoning.

Abstract

Explore similar work

FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting

Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows

Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain