LAVA: Logic-Aware Validation and Augmentation Framework for Large-Scale Financial Document Auditing

Ruoqi Shu; Xuhui Wang; Isaac Wang; Yanming Mai; Bo Wan

cs.AIAug 17, 2026

LAVA: Logic-Aware Validation and Augmentation Framework for Large-Scale Financial Document Auditing

Authors: Ruoqi Shu, Xuhui Wang, Isaac Wang, Yanming Mai, Bo Wan

Organizations: BMO Financial Group

Abstract

Financial document validation in production, such as payroll auditing, tax compliance, and loan underwriting, demands exceptional accuracy, consistency, and reproducibility under strict enterprise constraints. In practice, documents arrive with heterogeneous layouts and formats, semantically rich and context-dependent content, and embedded business rules that current pipelines struggle to process reliably. We introduce LAVA (Logic-Aware Validation and Augmentation), a modular, backbone-agnostic pipeline built on multimodal large language models, that integrates a four-stage design: document-rule retrieval, layout-preserving information extraction, auxiliary metadata enrichment, and auditable symbolic/arithmetic verification. LAVA supports robust rule grounding, fine-grained error attribution, and consistent, traceable end-to-end execution, capabilities essential for high-stakes deployment. Evaluated on a large real-world benchmark with diverse financial documents and dozens of expert-curated validation rules, LAVA outperforms baselines in hallucination control and edge-case handling while maintaining efficient token usage, demonstrating practicality for high-volume, time-critical validation.

Figures & tables

Figure 1: Overview of LAVA. The architecture comprises two parallel pipelines—document processing and rule grounding—interacting via bidirectional constraints (solid arrows: main data flow; dashed arrows: auxiliary exchange). The document-processing track performs retrieval, extraction, and augmentation to produce layout-preserving structured content with enriched metadata. The rule-grounding track retrieves, classifies, and dispatches applicable validation rules to either the Symbolic Reasoner or Arithmetic Processor, depending on reasoning type.

Figure 2: Arithmetic Processor. Given a validation rule, the system generates a formula and delegates evaluation to an external calculator. A checker LLM validates alignment between the rule and result. Failures trigger a fallback loop with negative feedback.

Panel of Figure 3 — Figure 3: Prompt excerpt for Symbolic Reasoner, dynamically built from the retrieved validation rule, extracted knowledge, and augmentation components. Full versions for both Arithmetic Processor and Symbolic Reasoner are provided in Appendix A.1.1 and A.1.2 .

Metric	Rule Group	VLM + Field-Level OCR	LLM + Field-Level OCR	LLM + Enhanced OCR	LAVA
Factual	$C_{1}$	0.03	0.03	0.01	0.01
Hallucination	$C_{2}$	0.08	0.10	0.04	0.03
Rate	$C_{3}$	0.31	0.33	0.30	0.18
	$C_{4}$	0.05	0.08	0.05	0.03
	$C_{5}$	0.28	0.30	0.15	0.05
Numerical	$C_{1}$	N/A	N/A	N/A	N/A

Table 1: Performance of baselines across three metrics and five validation rule categories (

C_{1}

–

C_{5}

C_{1}

: content extraction;

C_{2}

: conditional logic reasoning;

C_{3}

: multi-step logic reasoning;

C_{4}

: unconstrained arithmetic consistency checking;

C_{5}

: constrained arithmetic consistency checking.

Pipeline	$C_{1}$	$C_{2}$	$C_{3}$	$C_{4}$	$C_{5}$
LAVA w/o KE	0.03	0.65	0.67	0.09	0.66
LAVA w/ Plain-text KE	0.02	0.63	0.58	0.08	0.48
LAVA w/ Markdown KE	0.01	0.25	0.54	0.05	0.55
LAVA w/o IA	0.01	0.15	0.45	0.05	0.12
LAVA w/o AP	0.01	0.03	0.29	0.10	0.56
LAVA	0.01	0.03	0.28	0.05	0.07

Table 2: Percentage of responses failing to exactly match ground-truth answers across five rule categories.

Panel of Figure 4 — Figure 4: Average input and output token counts across five rule categories.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Sample document shown for clarity and error illustrations; metrics are document-agnostic and applied unchanged across all document types (sensitive content redacted).

Document Information Extraction

Document Information Extraction turns text, tables, forms, and images into structured, searchable data.

See all 290 papers

FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks
Aug 24, 2026Hang Wang, Jin Zhang, Guoliang Xu +13Document Parsing Document Image Understanding
Ant Group, Hangzhou, China
FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
Mar 11, 2026Arun Vignesh Malarkkan, Manan Roy Choudhury, Guangwei Zhang +4Financial Reasoning in Language Models Benchmark Auditing
Arizona State University Tempe, Arizona, USA · Pine AI Singapore, Singapore · The College of William & Mary Williamsburg, Virginia, USA +1
Perception, Layout, and Validation: Calibrated Confidence for Reliable Straight-Through Processing of Financial Documents
Sep 17, 2026Yichao Jin, Yushuo Wang, Yuxuan Han +7Document Information Extraction VLM Calibration
OCBC, Singapore

Aug 24, 2026J/K move · Enter open · S save

FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks

Hang Wang, Jin Zhang, Guoliang Xu, Pengyue Lu+12

Ant Group, Hangzhou, China

Document Parsing Document Image Understanding Large Vision-Language Models

Open paper AskPDF arXiv

Financial document parsing requires accuracy, structural consistency, and verifiability that current benchmarks often fail to reflect. We present FinixDoc, an end-to-end agentic parsing system for real-world financial documents, with FinixDoc-VL, a 4B-scale vision-language model built on Qwen3-VL-4B, as its core parser. To characterize the gap between benchmark and deployment performance, we introduce a Document Parsing Capability Matrix organized along two practical axes: visual quality and document scale. Guided by this matrix, FinixDoc-VL is trained with a domain-adapted recipe combining homoglyph-aware contrastive learning and multi-stage reinforcement learning with composite domain-specific rewards. To better leverage our accumulated advantage in low-quality financial-document data and support large-scale, high-quality data production, we further build a human-in-the-loop Data Factory pipeline with confidence-aware expert review. For evaluation, we construct FinixDocBench, a financial-domain evaluation suite covering digital-native, camera-captured, ultra-large-page, and internal-workflow scenarios, with a compliance-reviewed subset released alongside this technical report. On its main subsets, FinixDoc-VL achieves the highest overall score (81.43) among evaluated baselines, outperforming the next-best open-source model by 5.13 points, with the largest gains on internal financial workflows (FinixInner: 84.08 vs. 78.73).

Abstract

Document Information Extraction

Explore similar work

FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks

FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles

Perception, Layout, and Validation: Calibrated Confidence for Reliable Straight-Through Processing of Financial Documents