Document-Level Numerical Reasoning across Single and Multiple Tables in Financial Reports

Yi-Cheng Wang; Wei-An Wang; Chu-Song Chen

cs.CLApr 4, 2026

Document-Level Numerical Reasoning across Single and Multiple Tables in Financial Reports

Authors: Yi-Cheng Wang, Wei-An Wang, Chu-Song Chen

Organizations: Department of Computer Science and Information Engineering, National Taiwan University · FinTech Center, National Taiwan University

Abstract

Despite the strong language understanding abilities of large language models (LLMs), they still struggle with reliable question answering (QA) over long, structured documents, particularly for numerical reasoning. Financial annual reports exemplify this difficulty: financial statement analysis often hinges on accurate arithmetic, and analysts derive key indicators by integrating evidence scattered across multiple tables and narrative text. However, existing benchmarks focus largely on single-table settings, leaving cross-table document-level numerical reasoning underexplored. To address this gap, we introduce FinLongDocQA, a dataset for both single-table and cross-table financial numerical reasoning in long-context reports. Evaluating both closed-source and open-source LLMs on FinLongDocQA reveals two bottlenecks: (1) annual reports often exceed 129k tokens, exacerbating the context rot problem for locating relevant tables; and (2) even when relevant evidence is located, LLMs remain prone to errors in multi-step numerical reasoning. We propose FinLongDocAgent, a Multi-Agent Multi-Round Retrieval-Augmented Generation (RAG) approach that iteratively retrieves evidence, performs intermediate calculations, and verifies results across rounds. Experiments highlight the importance of iterative retrieval and verification for reliable numerical QA in long financial documents.

Figures & tables

Figure 1: An example QA instance from FinLongDocQA. The figure shows only the relevant tables and text for presentation; in practice, the model must retrieve them from the full annual report before computing the answer.

Dataset	Task	Input Modality			Doc RAG	Cross Table
Dataset	Task	Tab.	Text	Img.	Doc RAG	Cross Table
FiQA-SA (WWW’18)	Semantic analysis	✗	✓	✗	✗	✗
FinQA (EMNLP’21)	Numerical reasoning	✓	✓	✗	✗	✗
TAT-QA (ACL’21)	Numerical reasoning	✓	✓	✗	✗	✗
TAT-DQA (MM’22)	Numerical reasoning	✗	✓	✓	✗	✗
MultiHiertt (ACL’22)	Numerical reasoning	✓	✓	✗	✗	✓

Table 1: Comparison of financial QA datasets. Doc RAG denotes settings where QA is grounded in a full document and retrieval is required to find evidence.

Figure 2: Illustration of the construction steps for our FinLongDocQA dataset.

Figure 3: Distribution of evidence page occurrences across page ranges in annual reports.

Figure 4: A running example of FinLongDocAgent showing multi-agent interaction across retrieval rounds.

Error Analysis

Retrieval Error (62%)

Q: What was the 2023 net sales per facility in Europe, the Middle East & Africa? G: 518.75 Retrieved: 29, 32, 36, 37, 39 (+10 more) Failure cue: Missing required operands (EMEA net sales; # facilities). P: Retrieved pages do not contain (2023 EMEA Net Sales) / (2023 EMEA # Facilities).

Evidence Utilization Error (19%)

Q: What was the total lease liability per proved BoE of reserves at year-end 2022? G: 0.06 (GT: p.101, p.105) Retrieved: 101, 105, 104, 108, 107 (+10 more) Failure cue: Gold pages are retrieved, but the model uses reserves from the wrong page . P:

\frac{296+33+584+182\;({\color[rgb]{0,0.5,0}p.101})}{4{,}238\;({\color[rgb]{0.668,0.2734,0.2734}p.107})}=0.26

Value Extraction Error (11%)

Q: What is the absolute difference between 2022 underlying net income available to common stockholders and GAAP net income? G: 352 (GT: p.16, p.17) Retrieved: 16, 17, 15, 14, 18 (+10 more) Failure cue: Evidence is retrieved and used, but the model extracts a wrong GAAP value . P:

|2312-{\color[rgb]{0.668,0.2734,0.2734}2073}|={\color[rgb]{0.668,0.2734,0.2734}239}

(should be 352)

Table 9: Error analysis of FinLongDocAgent. Q : question, G : golden answer, P : model prediction.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Recall@k for different chunk sizes.

Methods	Prompting Design	FinQA †
FinQAPT ( Singh et al., 2024 )	Few-shot	62.0
FinLongDocAgent (ours)	Zero-shot	66.2

Appendix

Table 10: Comparison with a prior SOTA method on FinQA. † Modified setting where the full document is provided as input.

Panel of Figure 6 — Figure 6: Full prompt template used for QA generation.

Panel of Figure 7 — Figure 7: System prompts used in FinLongDocAgent.

Numerical Reasoning in Language Models

Numerical reasoning in language models studies how systems represent, calculate with, and explain numbers across mathematical and real-world tasks.

See all 150 papers

Agentic Retrieval-Augmented Generation for Financial Document Question Answering
May 6, 2026Yang Shu, Yingmin Liu, Zequn XieFinancial QA Agentic RAG
College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China
Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
Jul 22, 2026Xinke Tong, Xuanming Zhang, Tianyi Tang +10Numerical Reasoning in Language Models LLM Evaluation
Qwen Team, Alibaba Group · Tsinghua University · Alibaba Group
V-FiLLM: Verified Financial LLM Reasoning Benchmark
Aug 11, 2026Alicia Larsen, Victoire Laurent, Aulia Kharis Rakhmasari +2Numerical Reasoning in Language Models Synthetic Data Generation
ETH Zürich, Zürich · Aisot Technologies Ltd

May 6, 2026J/K move · Enter open · S save

Agentic Retrieval-Augmented Generation for Financial Document Question Answering

Yang Shu, Yingmin Liu, Zequn Xie

College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

Financial QA Agentic RAG Finance LLM Reasoning

Open paper AskPDF arXiv

Financial document question answering (QA) demands complex multi-step numerical reasoning over heterogeneous evidence--structured tables, textual narratives, and footnotes--scattered across corporate filings. Existing retrieval-augmented generation (RAG) approaches adopt a single-pass retrieve-then-generate paradigm that struggles with the compositional reasoning chains prevalent in financial analysis. We propose FinAgent-RAG, an agentic RAG framework that orchestrates iterative retrieval-reasoning loops with self-verification, specifically engineered for the precision requirements of financial numerical reasoning. The framework integrates three domain-specific innovations: (1) a Contrastive Financial Retriever trained with hard negative mining to distinguish semantically similar but numerically distinct financial passages, (2) a Program-of-Thought reasoning module that generates executable Python code for precise arithmetic rather than relying on error-prone LLM-based mental computation, and (3) an Adaptive Strategy Router that dynamically allocates computational resources based on question complexity, reducing API costs by 41.3% on FinQA while preserving accuracy. Extensive experiments on three benchmark datasets--FinQA, ConvFinQA, and TAT-QA--demonstrate that FinAgent-RAG achieves 76.81%, 78.46%, and 74.96% execution accuracy respectively, outperforming the strongest baseline by 5.62--9.32 percentage points. Ablation studies, cross-backbone evaluation with four LLMs, and deployment cost analysis confirm the framework's robustness and practical viability for financial institutions.

Abstract

Numerical Reasoning in Language Models

Explore similar work

Agentic Retrieval-Augmented Generation for Financial Document Question Answering

Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

V-FiLLM: Verified Financial LLM Reasoning Benchmark