Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis

Zhiyuan Cheng; Longying Lai; Yue Liu; Kai Cheng; Xiaoxi Qi

cs.CLFeb 18, 2026

Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis

Authors: Zhiyuan Cheng, Longying Lai, Yue Liu, Kai Cheng, Xiaoxi Qi

Organizations: School of Engineering Stanford University Stanford, CA, USA · Simon Business School University of Rochester Rochester, NY, USA · Accounting & Information Systems Rutgers University Newark, NJ, USA · Institute for Social and Economic Research and Policy Columbia University New York, NY, USA ·

Abstract

Financial analysts face significant challenges extracting information from lengthy 10-K reports, which often exceed 100 pages. This paper presents a Retrieval-Augmented Generation (RAG) system designed to answer questions about S&P 500 financial reports and evaluates the impact of neural reranking on system performance. Our pipeline employs hybrid search combining full-text and semantic retrieval, followed by an optional reranking stage using a cross-encoder model. We conduct systematic evaluation using the FinDER benchmark dataset, comprising 1,500 queries across five experimental groups. Results demonstrate that reranking significantly improves answer quality, achieving 49.0 percent correctness for scores of 8 or above compared to 33.5 percent without reranking, representing a 15.5 percentage point improvement. Additionally, the error rate for completely incorrect answers decreases from 35.3 percent to 22.5 percent. Our findings emphasize the critical role of reranking in financial RAG systems and demonstrate performance improvements over baseline methods through modern language models and refined retrieval strategies.

Figures & tables

Figure 1: Document Processing Pipeline. The system converts HTML reports to PDF, extracts and chunks text, generates embeddings, and stores data in SQLite (for keyword search) and FAISS (for semantic search).

Figure 2: Query Processing Pipeline with Reranking Ablation. The pipeline processes queries through rewriting, hybrid search (FTS + semantic), RRF fusion, optional reranking, and answer generation. The ablation study compares performance with and without the reranking stage.

Group	Config	Avg	=1	$\geq$ 8	=10
		Score	(%)	(%)	(%)
01	With	5.85	24.3	47.7	12.0
01	No	4.64	37.7	22.7	8.7
02	With	6.17	21.7	51.6	16.0
02	No	4.98	34.7	36.0	10.0
03	With	5.64	25.7	44.0	14.0

Table I: Performance Comparison: With vs. Without Reranking

Component	Parameter	Value
Chunking	Chunk size	2,500 chars
	Overlap	1,250 chars
Embedding	Model	text-embedding-3-small
	Dimension	1,024
	Batch size	32
FTS Retrieval	Top-k	20

Table II: Complete System Configuration

Financial QA

Financial question answering develops systems that answer questions about reports, markets, regulations, and other financial data with reliable evidence and reasoning.

See all 51 papers

Hierarchical Reranking for Scalable Financial RAG System
Jul 29, 2026Joohyun Lee, Sungwoo HongFinancial Services Retrieval-Augmented Generation
1Financial Security Institute · 2Hanyang University
Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering
Feb 20, 2026Amine Kobeissi, Philippe LanglaisLong-Context Retrieval Financial QA
Department of Computer Science and Operations Research, Université de Montréal · RALI
From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents
Apr 2, 2026Meftun Akarsu, Recep Kaan Karaman, Christopher MierbachRAG Evaluation Hybrid Retrieval
Technische Hochschule Ingolstadt · Uludag University · Radiate

Jul 29, 2026J/K move · Enter open · S save

Hierarchical Reranking for Scalable Financial RAG System

Joohyun Lee, Sungwoo Hong

1Financial Security Institute · 2Hanyang University

Financial Services Retrieval-Augmented Generation Learning to Rank Financial QA

Open paper AskPDF arXiv

Analyzing financial documents such as 10-K filings, tabular disclosures, and macroeconomic reports demands expert reasoning and extensive time. However, existing Retrieval-Augmented Generation systems often struggle to process hybrid text-table structures or the massive scale of financial documents. To address these challenges, we propose Hierarchical Reranker, a RAG framework designed to improve retrieval performance and generative reliability across large-scale financial datasets. The system integrates three key innovations: Pre-Retrieval Optimization, enhancing query clarity and search efficiency through normalization, keyword expansion, and table transformation; Hierarchical Reranker Architecture, improving retrieval precision through a two-stage ranking mechanism; and Long-Context Management, preserving reasoning accuracy through adaptive input partitioning and fusion under extensive contexts. Across multiple benchmarks, including FinQA, FinanceBench, and ConvFinQA, the proposed system achieved an NDCG@20 score of 0.7918 and demonstrated superior factual consistency. Its robustness was further validated by achieving second place in the ACM-ICAIF '24 FinanceRAG Challenge. This work presents a deployable, domain-optimized RAG pipeline that enhances both the accuracy and scalability of financial reasoning, paving the way for automated audit reporting and quantitative investment analysis. The source code will be made publicly available on GitHub upon acceptance.

Abstract

Financial QA

Explore similar work

Hierarchical Reranking for Scalable Financial RAG System

Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents