cs.CLNov 3, 2026

When Self-Verification Hurts: Calibrated Answer Repair for Financial Question Answering

Authors: Celine Guo, Weida Liang, Yujian Fu, Anqi Dang, Yanzheng Jin, Kenji Kawaguchi, Wenya Wang

Organizations: National University of Singapore, Singapore · Nanyang Technological University, Singapore

Abstract

Financial question answering can fail even when a model extracts a plausible number from the supplied evidence: the answer may attach it to the wrong entity, period, unit, or accounting concept. Asking the model to verify and revise its draft can help, but open-ended revision may alter content that was already correct or inspect the wrong relation and reinforce the original error. We formulate repair as a decision that balances the information it may recover against the correct content it may damage. Based on this trade-off, we introduce Calibrated Answer Repair (CAR), which first estimates whether a draft is likely to benefit from repair, then selects a targeted check, localizes the relevant evidence or draft span, and limits the permitted edit. Across FinRank, Fin-RATE, and FinQA with three open-weight models, CAR outperforms generic verification in all dataset-model settings. At the system level, it recovers 14.6% of initially missed answer requirements while damaging 2.1% of initially correct ones. Its post-draft token cost is over 30 percent lower than the iterative revision alternatives. Ablations on two dataset-model settings show that evidence localization and bounded editing are the main contributors to CAR's gains.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Towards Expert Financial QA via Self-Improving RAG

    Aug 27, 2026Junjie Xiong, Shawheen Ghezavat, Aum HirparaLLM Self-CorrectionFinancial QA

  2. FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings

    Aug 7, 2026Sasan Mansouri, Daniel Saad, Mark Wahrenburg +2Data ProvenanceFinancial QA

  3. MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning

    Jun 10, 2026Abdelrahman Abdallah, AbdelRahim A. Elmadany, Sameh Al Natour +3Table QALLM Answer Verification