Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector

Pedram Agand

cs.LGMar 4, 2026

Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector

Authors: Pedram Agand

Organizations: FactAI Lab, Vancouver, BC, Canada.

Abstract

Standard Retrieval-Augmented Generation (RAG) architectures fail in high-stakes financial domains due to two fundamental limitations: the inherent arithmetic incompetence of Large Language Models (LLMs) and the distributional semantic conflation of dense vector retrieval (e.g., mapping "Net Income" to "Net Sales" due to contextual proximity). In deterministic domains, a 99% accuracy rate yields 0% operational trust. To achieve zero-hallucination financial reasoning, we introduce the Verifiable Numerical Reasoning Agent (VeNRA). VeNRA shifts the RAG paradigm from retrieving probabilistic text to retrieving deterministic variables via a strictly typed Universal Fact Ledger (UFL). We mathematically bound this ledger using a novel Double-Lock Grounding algorithm. Coupled with deterministic Python execution, this neuro-symbolic routing compresses systemic hallucination rates to a near-zero 1.2%. Recognising that upstream parsing anomalies inevitably occur, we introduce the VeNRA Sentinel: a 3-billion parameter SLM trained to forensically audit candidate using a single-token inference budget with optional post-hoc reasoning. To train the Sentinel, we steer away from traditional hallucination datasets in favour of Adversarial Simulation, programmatically sabotaging financial records to simulate Ecological Errors. The compact Sentinel consequently outperforms 70B+ frontier models in error detection. Through Loss Dilution phenomenon in Reverse-CoT training, we present a novel Micro-Chunking loss algorithm to stabilise gradients under extreme verdict penalisation, yielding a 28x latency speedup without sacrificing forensic rigor.

Figures & tables

Figure 1: The VeNRA Neuro-Symbolic Paradigm. (Top) Standard RAG suffers from “Stochastic Inaccuracy” due to probabilistic arithmetic and vector conflation. (Bottom) VeNRA parses PDFs into a Universal Fact Ledger (UFL). A Query Navigator retrieves specific variables via a Lexical Gate. An Architect LLM generates a deterministic Python trace. Crucially, the $3$ B Sentinel model forensically audits the logic trace before surfacing the answer.

Split	Total	Supported	Unfounded	General
Training	6,673	3,421	3,153	99
Validation	835	425	396	14
Testing	812	420	382	10
Total	8,320	4,266	3,931	123

Table 1: VeNRA-Data distribution across splits. The topological split ensures all sabotage variants of the same source context reside in the same fold.

Run	Configuration	Retriever	Generator	EM $\uparrow$	Halluc. % $\downarrow$	Compile % $\uparrow$
1	Baseline RAG	Vector	Gemini CoT	$39.2\%$	$4.1\%$	—
2	Smart Ret., Dumb Math	VeNRA	Gemini CoT	$37.4\%$	$5.3\%$	—
3	Dumb Ret., Smart Math	Vector	VeNRA PAL	$40.4\%$	$1.8\%$	$51.7\%$
4	VeNRA Full (Ours)	VeNRA	VeNRA PAL	$\mathbf{49.1\%}$	$\mathbf{1.2\%}$	$\mathbf{68.8\%}$

Table 2: 2

\times

2 pipeline evaluation. Runs 1–2 use Gemini-3-flash CoT generation; Runs 3–4 use the VeNRA PAL Code Agent. Hallucination rate is computed as the fraction of incorrect answers containing a numerically ungrounded value. Compilation rate applies to PAL runs only; it measures first-attempt Python execution success.

Figure 2: Failure Type Transition: Run 1 (Baseline RAG) vs. Run 4 (VeNRA Full) . Type-1 Retrieval Blindness falls by 29 cases following Navigator rescue; Type-7 Generation Failures, which spiked under Run 2 (Smart Retrieval + CoT), return toward baseline levels once the PAL Code Agent replaces free-form generation. The net gain of 17 additional correct answers constitutes the

+9.9

pp EM improvement.

Metric	Count / Value	Share
Chunks processed	$899$	—
Text facts proposed (Lock 1)	$2{,}541$	—
Text facts rejected (Lock 2)	$754$	$29.7\%$
Text facts accepted	$1{,}787$	$70.3\%$
Numeric facts (deterministic)	$1{,}278$	—
UFL total rows	$\mathbf{3{,}065}$	—

Table 3: Double-Lock ingestion statistics over 899 document chunks. Text facts and numeric facts follow separate extraction paths; the Double-Lock filter applies only to the text fact track. Alignment tier and confidence score are assigned at Lock 2 verification.

Model	Valid%	FR S $\uparrow$	FR L $\uparrow$	FR global $\uparrow$	Recall nat $\uparrow$	TPR clean $\uparrow$	FPR clean $\downarrow$	$\mathcal{M}$ $\uparrow$
Base Qwen 3B (Zero-Shot)	$100.0\%$	$2.7\%$	$10.1\%$	$5.3\%$	$2.1\%$	$\mathbf{97.5\%}$	$\mathbf{2.5\%}$	$0.024$
VeNRA 3B (Ours)	$\mathbf{100.0\%}$	$\mathbf{92.6\%}$	$\mathbf{93.7\%}$	$\mathbf{93.0\%}$	$\mathbf{83.6\%}$	$81.5\%$	$18.5\%$	$\mathbf{0.630}$

Table 4: Full test-set evaluation (

n{=}812

\mathcal{M}

is computable only here, where all four metric pillars are populated. Axiom accuracy is indicative only (

n{=}10

held-out axioms). Bold denotes best per column;

\uparrow

\downarrow

indicate preferred direction.

Panel of Figure 3 — Figure 3: Full-distribution auditing performance. Left: $\mathcal{M}$ scores on the test set, demonstrating a $26.5\times$ improvement. Right: the FPR–TPR plane reveals the sycophancy trap, high TPR is trivially achievable by suppressing all detections, rendering the model operationally useless. The shaded green region marks the high-utility quadrant (low false-alarm rate, high valid-claim acceptance). The base model achieves high TPR only by never firing.

Model	Tokens	Valid%	Global FR $\uparrow$	Logic Code Lie	Numeric Neighbor	Irrelevancy RAG	Semantic Drift
Base Qwen 3B (Zero-Shot)	1	$100.0\%$	$8.7\%$	$6.1\%$	$0.0\%$	$8.0\%$	$0.0\%$
Base Qwen 3B ( + CoT)	150	$97.8\%$	$15.2\%$	$15.4\%$	$15.4\%$	$25.0\%$	$0.0\%$
Gemini-3-Flash-Preview †	100	$63.0\%$	$39.1\%$	$15.4\%$	$46.2\%$	$75.0\%$	$12.5\%$
GPT-OSS-120B †	100	$79.4\%$	$56.5\%$	$61.5\%$	$92.3\%$	$41.7\%$	$12.5\%$
Kimi K2.5	1	$93.96\%$	$69.6\%$	$69.2\%$	$92.3\%$	$91.7\%$	$0.0\%$
Llama 3.3 70B	1	$100.0\%$	$71.7\%$	$76.9\%$	$84.6\%$	$\mathbf{100.0\%}$	$0.0\%$

Table 5: Adversarial subsample evaluation (

n{=}92

rows, 46 pairs;

{\sim}13

per Logic/Numeric,

{\sim}12

Irrelevancy,

{\sim}8

Semantic Drift). Models marked

\dagger

were allocated 100 generation tokens budget to response; all others (except CoT) operate under a strict 1-token constraint. Global FR and per-type rates are computed only over valid responses. Bold : best per column.

Figure 4: Detection Rate Across Sabotage Typologies. The Sentinel achieves ${\geq}88\%$ detection across all four forensic failure modes. The base model’s near-zero rates across all typologies confirm that the discriminative capability is entirely attributable to fine-tunning, not to the base model’s pre-trained knowledge.

Condition	Median (ms)	Mean (ms)	P95 (ms)	Speedup
CoT ( ${\leq}150$ tok, capped)	$4{,}092.5$	$4{,}017.0$	$4{,}178.6$	$1.0\times$ (baseline)
VeNRA (1 tok)	$\mathbf{142.6}$	$\mathbf{150.8}$	$\mathbf{217.8}$	$\mathbf{28.7\times}$

Table 6: Time-to-Verdict latency benchmark (

n{=}100

prompts, NVIDIA A40). CoT was capped at 150 generation tokens; see text for uncapped extrapolation.

Panel of Figure 5 — Figure 5: Latency Efficiency. Standard CoT requires hundreds of reasoning tokens before committing to a verdict; VeNRA encodes the decision into the first generated token, achieving a $28.7\times$ reduction in median latency.

Configuration	$\mathcal{M}$ $\uparrow$	FR $\uparrow$	Found $\uparrow$	Fake $\uparrow$	Axiom $\uparrow$	Recall nat $\uparrow$	TPR $\uparrow$	FPR $\downarrow$	ECE $\downarrow$
Baseline (Standard CE)	0	$0.115$	$0.894$	$0.192$	—	—	—	—	—
$+$ Weighted ( $50{\times}$ ), No Clamp	0.002	$0.295$	$0.657$	$0.581$	—	—	—	—	—
$+$ Clamped, Uniform ( $50/50/50$ )	$0.007$	$0.229$	$0.739$	$0.299$	$0.286$	$0.150$	$0.700$	$0.063$	$0.439$
$+$ Differential ( $50/50/10$ )	$0.032$	$0.325$	$0.637$	$0.452$	$0.571$	$0.275$	$0.625$	$0.138$	$0.415$
$+$ Prompt Tuning (Final)	$\mathbf{0.201}$	$\mathbf{0.363}$	$\mathbf{0.777}$	$\mathbf{0.433}$	$\mathbf{0.643}$	$0.263$	$\mathbf{0.725}$	$\mathbf{0.125}$	$\mathbf{0.375}$

Table 7: Progressive ablation of the Micro-Chunking loss architecture, evaluated at a fixed intermediate checkpoint.

\mathcal{M}

, Flip Rate, per-class accuracies, TPR/FPR on clean data, and ECE are reported. Recall nat is included where estimable at this checkpoint. Rows without

\mathcal{M}

indicate configurations where training was terminated before reaching a measurable composite score (gradient explosion or class collapse).

Figure 6: Training Pathology Visualisation. (a) Garbage Bin Effect: Per-class accuracy under uniform vs. differential weighting. Uniform weighting causes probability mass to collapse into the minority GENERAL class; differential weighting (

50/50/10

) recovers the core binary boundary. (b) Prompt Strategy Ablation: Removing instruction text (“No Instruct”) marginally improves or matches full-prompt performance (Composite

0.200

vs.

0.190

), compare to full prompt due, while front-loading instruction degrades it significantly (

0.110

Figure 7: Training Dynamics under Micro-Chunking Loss. (a) Composite Score trajectory across prompt strategies; peak

\mathcal{M}{=}0.201

at this checkpoint, rising to

0.630

at full convergence. (b) Gradient norm (EMA) under the final clamped configuration; peak

452.3

confirms stable training. The unclamped crashed with gradient norms exploding.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 8: The VeNRA Refinement UI. Streamlit UI to faciliate human annotation that are ambiguous from teacher and the AI proxy refiner.

Financial Reasoning in Language Models

See all 13 papers

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA
May 29, 2026Hao Chen, Xing Tang, Qirui Liu +6Retrieval-Augmented Generation Hallucination in Language Models
Shenzhen Technology University FiT, Tencent Shenzhen, China · South China University of Technology Guangzhou, China · The Hong Kong University of Science and Technology Hong Kong SAR, China +2
Agentic Retrieval-Augmented Generation for Financial Document Question Answering
May 6, 2026Yang Shu, Yingmin Liu, Zequn XieFinancial QA Agentic RAG
College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China
FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLLM Answer Verification Financial QA
1The University of Hong Kong · 2Stellaris AI Limited

May 29, 2026J/K move · Enter open · S save

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA

Hao Chen, Xing Tang, Qirui Liu, Weijie Shi+5

Shenzhen Technology University FiT, Tencent Shenzhen, China · South China University of Technology Guangzhou, China · The Hong Kong University of Science and Technology Hong Kong SAR, China+2

Retrieval-Augmented Generation Hallucination in Language Models Financial QA Mathematical Reasoning Language Model Robustness LLM Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) have significantly advanced online data services, particularly in the domain of financial question answering (FinQA). However, such systems remain susceptible to numerical reasoning hallucinations, which critically undermine reliability in high-stakes financial applications. Although retrieval-augmented generation (RAG) has been widely adopted to ground responses in external knowledge, it introduces three persistent challenges: noise sensitivity, calculation fragility, and an auditability crisis. Existing model-centric approaches, which primarily focus on optimizing either the retriever or generator in isolation, still struggle to address these issues in an integrated manner. In this work, we pioneer a data-centric paradigm and propose a novel framework, the Data-centric Reasoning Compiler (DCRC). The framework operates through three cohesive phases: (1) adversarial data construction, which synthesizes training examples with controlled noise to teach robustness; (2) multi-stage training that cultivates a Data-centric Structuring Agent (DSA) capable of explicit evidence auditing and program synthesis; and (3) a compile-and-execute inference process, where the DSA transforms user queries and retrieved documents into verifiable, executable reasoning programs. This data-driven framework ensures faithful numerical reasoning by design. We conduct extensive experiments on established offline benchmarks and further validate our framework through deployment in a real-world online financial QA system.

Abstract

Financial Reasoning in Language Models

Explore similar work

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA

Agentic Retrieval-Augmented Generation for Financial Document Question Answering

FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification