Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector

Pedram Agand

cs.LGMar 4, 2026

Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector

Authors: Pedram Agand

Organizations: FactAI Lab, Vancouver, BC, Canada.

Abstract

Standard Retrieval-Augmented Generation (RAG) architectures fail in high-stakes financial domains due to two fundamental limitations: the inherent arithmetic incompetence of Large Language Models (LLMs) and the distributional semantic conflation of dense vector retrieval (e.g., mapping "Net Income" to "Net Sales" due to contextual proximity). In deterministic domains, a 99% accuracy rate yields 0% operational trust. To achieve zero-hallucination financial reasoning, we introduce the Verifiable Numerical Reasoning Agent (VeNRA). VeNRA shifts the RAG paradigm from retrieving probabilistic text to retrieving deterministic variables via a strictly typed Universal Fact Ledger (UFL). We mathematically bound this ledger using a novel Double-Lock Grounding algorithm. Coupled with deterministic Python execution, this neuro-symbolic routing compresses systemic hallucination rates to a near-zero 1.2%. Recognising that upstream parsing anomalies inevitably occur, we introduce the VeNRA Sentinel: a 3-billion parameter SLM trained to forensically audit candidate using a single-token inference budget with optional post-hoc reasoning. To train the Sentinel, we steer away from traditional hallucination datasets in favour of Adversarial Simulation, programmatically sabotaging financial records to simulate Ecological Errors. The compact Sentinel consequently outperforms 70B+ frontier models in error detection. Through Loss Dilution phenomenon in Reverse-CoT training, we present a novel Micro-Chunking loss algorithm to stabilise gradients under extreme verdict penalisation, yielding a 28x latency speedup without sacrificing forensic rigor.

Figures & tables

Figure 1: The VeNRA Neuro-Symbolic Paradigm. (Top) Standard RAG suffers from “Stochastic Inaccuracy” due to probabilistic arithmetic and vector conflation. (Bottom) VeNRA parses PDFs into a Universal Fact Ledger (UFL). A Query Navigator retrieves specific variables via a Lexical Gate. An Architect LLM generates a deterministic Python trace. Crucially, the $3$ B Sentinel model forensically audits the logic trace before surfacing the answer.

Split	Total	Supported	Unfounded	General
Training	6,673	3,421	3,153	99
Validation	835	425	396	14
Testing	812	420	382	10
Total	8,320	4,266	3,931	123

Table 1: VeNRA-Data distribution across splits. The topological split ensures all sabotage variants of the same source context reside in the same fold.

Run	Configuration	Retriever	Generator	EM $\uparrow$	Halluc. % $\downarrow$	Compile % $\uparrow$
1	Baseline RAG	Vector	Gemini CoT	$39.2\%$	$4.1\%$	—
2	Smart Ret., Dumb Math	VeNRA	Gemini CoT	$37.4\%$	$5.3\%$	—
3	Dumb Ret., Smart Math	Vector	VeNRA PAL	$40.4\%$	$1.8\%$	$51.7\%$
4	VeNRA Full (Ours)	VeNRA	VeNRA PAL	$\mathbf{49.1\%}$	$\mathbf{1.2\%}$	$\mathbf{68.8\%}$

Table 2: 2

\times

2 pipeline evaluation. Runs 1–2 use Gemini-3-flash CoT generation; Runs 3–4 use the VeNRA PAL Code Agent. Hallucination rate is computed as the fraction of incorrect answers containing a numerically ungrounded value. Compilation rate applies to PAL runs only; it measures first-attempt Python execution success.

Figure 2: Failure Type Transition: Run 1 (Baseline RAG) vs. Run 4 (VeNRA Full) . Type-1 Retrieval Blindness falls by 29 cases following Navigator rescue; Type-7 Generation Failures, which spiked under Run 2 (Smart Retrieval + CoT), return toward baseline levels once the PAL Code Agent replaces free-form generation. The net gain of 17 additional correct answers constitutes the

+9.9

pp EM improvement.

Metric	Count / Value	Share
Chunks processed	$899$	—
Text facts proposed (Lock 1)	$2{,}541$	—
Text facts rejected (Lock 2)	$754$	$29.7\%$
Text facts accepted	$1{,}787$	$70.3\%$
Numeric facts (deterministic)	$1{,}278$	—
UFL total rows	$\mathbf{3{,}065}$	—

Table 3: Double-Lock ingestion statistics over 899 document chunks. Text facts and numeric facts follow separate extraction paths; the Double-Lock filter applies only to the text fact track. Alignment tier and confidence score are assigned at Lock 2 verification.

Model	Valid%	FR S $\uparrow$	FR L $\uparrow$	FR global $\uparrow$	Recall nat $\uparrow$	TPR clean $\uparrow$	FPR clean $\downarrow$	$\mathcal{M}$ $\uparrow$
Base Qwen 3B (Zero-Shot)	$100.0\%$	$2.7\%$	$10.1\%$	$5.3\%$	$2.1\%$	$\mathbf{97.5\%}$	$\mathbf{2.5\%}$	$0.024$
VeNRA 3B (Ours)	$\mathbf{100.0\%}$	$\mathbf{92.6\%}$	$\mathbf{93.7\%}$	$\mathbf{93.0\%}$	$\mathbf{83.6\%}$	$81.5\%$	$18.5\%$	$\mathbf{0.630}$

Table 4: Full test-set evaluation (

n{=}812

\mathcal{M}

is computable only here, where all four metric pillars are populated. Axiom accuracy is indicative only (

n{=}10

held-out axioms). Bold denotes best per column;

\uparrow

\downarrow

indicate preferred direction.

Panel of Figure 3 — Figure 3: Full-distribution auditing performance. Left: $\mathcal{M}$ scores on the test set, demonstrating a $26.5\times$ improvement. Right: the FPR–TPR plane reveals the sycophancy trap, high TPR is trivially achievable by suppressing all detections, rendering the model operationally useless. The shaded green region marks the high-utility quadrant (low false-alarm rate, high valid-claim acceptance). The base model achieves high TPR only by never firing.

Model	Tokens	Valid%	Global FR $\uparrow$	Logic Code Lie	Numeric Neighbor	Irrelevancy RAG	Semantic Drift
Base Qwen 3B (Zero-Shot)	1	$100.0\%$	$8.7\%$	$6.1\%$	$0.0\%$	$8.0\%$	$0.0\%$
Base Qwen 3B ( + CoT)	150	$97.8\%$	$15.2\%$	$15.4\%$	$15.4\%$	$25.0\%$	$0.0\%$
Gemini-3-Flash-Preview †	100	$63.0\%$	$39.1\%$	$15.4\%$	$46.2\%$	$75.0\%$	$12.5\%$
GPT-OSS-120B †	100	$79.4\%$	$56.5\%$	$61.5\%$	$92.3\%$	$41.7\%$	$12.5\%$
Kimi K2.5	1	$93.96\%$	$69.6\%$	$69.2\%$	$92.3\%$	$91.7\%$	$0.0\%$
Llama 3.3 70B	1	$100.0\%$	$71.7\%$	$76.9\%$	$84.6\%$	$\mathbf{100.0\%}$	$0.0\%$

Table 5: Adversarial subsample evaluation (

n{=}92

rows, 46 pairs;

{\sim}13

per Logic/Numeric,

{\sim}12

Irrelevancy,

{\sim}8

Semantic Drift). Models marked

\dagger

were allocated 100 generation tokens budget to response; all others (except CoT) operate under a strict 1-token constraint. Global FR and per-type rates are computed only over valid responses. Bold : best per column.

Figure 4: Detection Rate Across Sabotage Typologies. The Sentinel achieves ${\geq}88\%$ detection across all four forensic failure modes. The base model’s near-zero rates across all typologies confirm that the discriminative capability is entirely attributable to fine-tunning, not to the base model’s pre-trained knowledge.

Condition	Median (ms)	Mean (ms)	P95 (ms)	Speedup
CoT ( ${\leq}150$ tok, capped)	$4{,}092.5$	$4{,}017.0$	$4{,}178.6$	$1.0\times$ (baseline)
VeNRA (1 tok)	$\mathbf{142.6}$	$\mathbf{150.8}$	$\mathbf{217.8}$	$\mathbf{28.7\times}$

Table 6: Time-to-Verdict latency benchmark (

n{=}100

prompts, NVIDIA A40). CoT was capped at 150 generation tokens; see text for uncapped extrapolation.

Panel of Figure 5 — Figure 5: Latency Efficiency. Standard CoT requires hundreds of reasoning tokens before committing to a verdict; VeNRA encodes the decision into the first generated token, achieving a $28.7\times$ reduction in median latency.

Configuration	$\mathcal{M}$ $\uparrow$	FR $\uparrow$	Found $\uparrow$	Fake $\uparrow$	Axiom $\uparrow$	Recall nat $\uparrow$	TPR $\uparrow$	FPR $\downarrow$	ECE $\downarrow$
Baseline (Standard CE)	0	$0.115$	$0.894$	$0.192$	—	—	—	—	—
$+$ Weighted ( $50{\times}$ ), No Clamp	0.002	$0.295$	$0.657$	$0.581$	—	—	—	—	—
$+$ Clamped, Uniform ( $50/50/50$ )	$0.007$	$0.229$	$0.739$	$0.299$	$0.286$	$0.150$	$0.700$	$0.063$	$0.439$
$+$ Differential ( $50/50/10$ )	$0.032$	$0.325$	$0.637$	$0.452$	$0.571$	$0.275$	$0.625$	$0.138$	$0.415$
$+$ Prompt Tuning (Final)	$\mathbf{0.201}$	$\mathbf{0.363}$	$\mathbf{0.777}$	$\mathbf{0.433}$	$\mathbf{0.643}$	$0.263$	$\mathbf{0.725}$	$\mathbf{0.125}$	$\mathbf{0.375}$

Table 7: Progressive ablation of the Micro-Chunking loss architecture, evaluated at a fixed intermediate checkpoint.

\mathcal{M}

, Flip Rate, per-class accuracies, TPR/FPR on clean data, and ECE are reported. Recall nat is included where estimable at this checkpoint. Rows without

\mathcal{M}

indicate configurations where training was terminated before reaching a measurable composite score (gradient explosion or class collapse).

Figure 6: Training Pathology Visualisation. (a) Garbage Bin Effect: Per-class accuracy under uniform vs. differential weighting. Uniform weighting causes probability mass to collapse into the minority GENERAL class; differential weighting (

50/50/10

) recovers the core binary boundary. (b) Prompt Strategy Ablation: Removing instruction text (“No Instruct”) marginally improves or matches full-prompt performance (Composite

0.200

vs.

0.190

), compare to full prompt due, while front-loading instruction degrades it significantly (

0.110

Figure 7: Training Dynamics under Micro-Chunking Loss. (a) Composite Score trajectory across prompt strategies; peak

\mathcal{M}{=}0.201

at this checkpoint, rising to

0.630

at full convergence. (b) Gradient norm (EMA) under the final clamped configuration; peak

452.3

confirms stable training. The unclamped crashed with gradient norms exploding.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 8: The VeNRA Refinement UI. Streamlit UI to faciliate human annotation that are ambiguous from teacher and the AI proxy refiner.

Financial Reasoning in Language Models

See all 13 papers

May 29, 2026

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA

Large Language Models (LLMs) have significantly advanced online data services, particularly in the domain of financial question answering (FinQA). However, such systems remain susceptible to numerical reasoning hallucinations, which critically undermine reliability in high-stakes financial applications. Although retrieval-augmented generation (RAG) has been widely adopted to ground responses in external knowledge, it introduces three persistent challenges: noise sensitivity, calculation fragility, and an auditability crisis. Existing model-centric approaches, which primarily focus on optimizing either the retriever or generator in isolation, still struggle to address these issues in an integrated manner. In this work, we pioneer a data-centric paradigm and propose a novel framework, the Data-centric Reasoning Compiler (DCRC). The framework operates through three cohesive phases: (1) adversarial data construction, which synthesizes training examples with controlled noise to teach robustness; (2) multi-stage training that cultivates a Data-centric Structuring Agent (DSA) capable of explicit evidence auditing and program synthesis; and (3) a compile-and-execute inference process, where the DSA transforms user queries and retrieved documents into verifiable, executable reasoning programs. This data-driven framework ensures faithful numerical reasoning by design. We conduct extensive experiments on established offline benchmarks and further validate our framework through deployment in a real-world online financial QA system.

Hao Chen, Xing Tang, Qirui Liu +6

Shenzhen Technology University FiT, Tencent Shenzhen, China · South China University of Technology Guangzhou, China · The Hong Kong University of Science and Technology Hong Kong SAR, China +2

Retrieval-Augmented Generation Hallucination in Language Models Financial QA Mathematical Reasoning Language Model Robustness LLM Reasoning

Ask PDF

May 6, 2026

Agentic Retrieval-Augmented Generation for Financial Document Question Answering

Financial document question answering (QA) demands complex multi-step numerical reasoning over heterogeneous evidence--structured tables, textual narratives, and footnotes--scattered across corporate filings. Existing retrieval-augmented generation (RAG) approaches adopt a single-pass retrieve-then-generate paradigm that struggles with the compositional reasoning chains prevalent in financial analysis. We propose FinAgent-RAG, an agentic RAG framework that orchestrates iterative retrieval-reasoning loops with self-verification, specifically engineered for the precision requirements of financial numerical reasoning. The framework integrates three domain-specific innovations: (1) a Contrastive Financial Retriever trained with hard negative mining to distinguish semantically similar but numerically distinct financial passages, (2) a Program-of-Thought reasoning module that generates executable Python code for precise arithmetic rather than relying on error-prone LLM-based mental computation, and (3) an Adaptive Strategy Router that dynamically allocates computational resources based on question complexity, reducing API costs by 41.3% on FinQA while preserving accuracy. Extensive experiments on three benchmark datasets--FinQA, ConvFinQA, and TAT-QA--demonstrate that FinAgent-RAG achieves 76.81%, 78.46%, and 74.96% execution accuracy respectively, outperforming the strongest baseline by 5.62--9.32 percentage points. Ablation studies, cross-backbone evaluation with four LLMs, and deployment cost analysis confirm the framework's robustness and practical viability for financial institutions.

Yang Shu, Yingmin Liu, Zequn Xie

College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

Financial QA Agentic RAG Finance LLM Reasoning

Ask PDF

Apr 26, 2026

FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification

Financial AI systems must produce answers grounded in specific regulatory filings, yet current LLMs fabricate metrics, invent citations, and miscalculate derived quantities. These errors carry direct regulatory consequences as the EU AI Act's high-risk enforcement deadline approaches (August 2026). Existing hallucination detectors treat all claims uniformly, missing 43% of computational errors that require arithmetic re-verification against structured tables. We present FinGround, a three-stage verify-then-ground pipeline for financial document QA. Stage 1 performs finance-aware hybrid retrieval over text and tables. Stage 2 decomposes answers into atomic claims classified by a six-type financial taxonomy and verified with type-routed strategies including formula reconstruction. Stage 3 rewrites unsupported claims with paragraph- and table-cell-level citations. To cleanly isolate verification value from retrieval quality, we propose retrieval-equalized evaluation as standard methodology for RAG verification research: when all systems receive identical retrieval, FinGround still reduces hallucination rates by 68% over the strongest baseline (

p < 0.01

). The full pipeline achieves a 78% reduction relative to GPT-4o. An 8B distilled detector retains 91.4% F1 at 18x lower per-claim latency, enabling $0.003/query deployment, supported by qualitative signals from a four-week analyst pilot.

Dongxin Guo, Jikun Wu, Siu Ming Yiu

1The University of Hong Kong · 2Stellaris AI Limited

LLM Answer Verification Financial QA LLM Hallucination Detection RAG Evaluation Evidence-Grounded Generation Finance

Ask PDF

Abstract

Financial Reasoning in Language Models

Explore similar work

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA

Agentic Retrieval-Augmented Generation for Financial Document Question Answering

FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification