Financial question answering can fail even when a model extracts a plausible number from the supplied evidence: the answer may attach it to the wrong entity, period, unit, or accounting concept. Asking the model to verify and revise its draft can help, but open-ended revision may alter content that was already correct or inspect the wrong relation and reinforce the original error. We formulate repair as a decision that balances the information it may recover against the correct content it may damage. Based on this trade-off, we introduce Calibrated Answer Repair (CAR), which first estimates whether a draft is likely to benefit from repair, then selects a targeted check, localizes the relevant evidence or draft span, and limits the permitted edit. Across FinRank, Fin-RATE, and FinQA with three open-weight models, CAR outperforms generic verification in all dataset-model settings. At the system level, it recovers 14.6% of initially missed answer requirements while damaging 2.1% of initially correct ones. Its post-draft token cost is over 30 percent lower than the iterative revision alternatives. Ablations on two dataset-model settings show that evidence localization and bounded editing are the main contributors to CAR's gains.
Figures & tables
Figure 1: A value–concept binding error: both values appear in the filing, but the draft assigns them to the wrong accounting concepts. Generic verification checks presence without confirming what each value refers to.
Figure 2: Overview of CAR : diagnosis gates repair, predicted error subtypes guide the check and pointer, and an edit limit constrains one repair call.
Partition
FinRank
Fin-RATE
FinQA
Calibration
910
5,250
6,251
Development
140
750
883
Held-out test
135
1,500
1,147
Table 1: Partition sizes by benchmark.
Dataset / Metric
Model
Draft
Verify
SC-Verify
CoVe
RARR
CAR
FinRank / GAA
Qwen3-8B
23.70
22.96
25.19
25.93
28.15
29.63
Qwen3-14B
31.11
31.85
36.30
34.07
36.30
35.56
Mistral-24B
29.63
30.37
31.11
31.11
33.33
34.81
Fin-RATE / Macro CR
Qwen3-8B
5.53
5.73
6.13
6.67
7.47
8.20
Qwen3-14B
11.27
11.60
12.33
13.07
14.07
14.67
Mistral-24B
16.73
17.13
17.87
18.07
19.40
20.07
Table 2: Held-out benchmark scores (percent, with higher values indicating better performance). Bold marks the best method per setting.
Figure 3: Failure rates in the blinded 144 paired drafts.
Method
Recovery (%)
Damage (%)
Calls/q
Tokens per question
Rel. cost (%)
Draft
—
—
0.00
0
0.0
Verify
8.1
6.7
1.00
3,417
100.0
SC-Verify
9.8
5.5
3.00
10,467
306.3
CoVe
11.7
4.8
4.00
11,960
350.0
RARR
13.8
2.4
3.58
8,240
241.1
CAR
14.6
2.1
1.54
5,343
156.4
Table 3: Requirement transitions and post-draft cost, macro-averaged across settings. Relative cost is normalized to Verify at 100.0%.
CAR metric
Rate (%)
Overall recovery
14.6
Overall damage
2.1
Gate-open recovery
20.3
Gate-open damage
5.3
Changed-answer damage
8.4
Edit rate
35.0
Table 4: System-level and conditional repair outcomes.
Variant
GAA
Full CAR
35.56
Always repair
30.37
No pointer
28.89
Fixed checklist
31.85
Unbounded rewrite
27.41
Table 5: FinRank GAA for Qwen3-14B CAR variants.
Dataset
Always-repair f
Always-repair d
m⋆
Gate-open rate (%)
FinRank
0.142
0.095
0.401
62.7
Fin-RATE
0.115
0.089
0.436
55.4
FinQA
0.068
0.081
0.543
44.1
Table 6: Break-even quantities averaged across models.
Miss-rate input
MAE ↓
Pearson r↑
Dataset mean only
0.196
—
Text features only
0.171
0.344
Probe verdicts only
0.158
0.473
Text + probe
0.146
0.512
Table 7: Miss-rate prediction on grouped development folds.
Model
DR-QA
EC-QA
LT-QA
Qwen3-8B
4.4
1.4
2.2
Qwen3-14B
5.4
2.0
2.8
Mistral-24B
4.8
2.8
2.4
Table 8: Fin-RATE CAR minus Draft CR.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Family
Definition
Typical checks and localization
Omission
A requested entity, period, qualifier, comparison, or result is absent.
Retrieve the supporting source passage and append only the supported missing relation.
Binding
A stated value is attached to the wrong entity, period, unit, or accounting concept.
Trace the value to its row, column, and passage, then locally replace the incorrect binding.
Evidence
A claim is unsupported, weakly supported, or grounded in a distractor passage.
Compare the draft sentence with its strongest evidence and verify provenance.
Numerical
The selected operands, operation, scale, sign, or computed result is wrong.
Point to the calculation span and recompute from cited inputs.
Interpretation
The answer misstates a comparison, trend, aggregation, or requested relation.
Re-read the question relation and check the conclusion against the evidence.
Appendix
Table 9: The five fixed failure families. Fine-grained leaves are induced from calibration data and used for hierarchical smoothing and routing.
Family
Leaf
Definition
Omission
f_omit_narrative
A requested fact appears in narrative evidence but is absent from the answer.
f_omit_absent
The requested fact is not supported by the supplied evidence and cannot be recovered by verification.
f_omit_derivable
The required inputs are present, but the answer omits a value that must be derived.
f_omit_table
A requested value appears in a supplied table but is not reported.
f_omit_clause
A multi-part question is answered only in part.
Binding
f_entity
A value is attributed to the wrong company or entity.
Appendix
Table 10: The complete 25-leaf taxonomy. Frequencies are computed from the frozen calibration-label export rather than from test outputs. The pooled calibration corpus contains 5,019 labeled errors.
Setting
Metric
Pointer
Comparison
FinRank
Source Hit@3
72.8
46.9 random
Fin-RATE
Source Hit@3
52.4
20.1 random
Fin-RATE
Source Hit@3
52.4 relevance
29.2 novelty
FinQA
Span overlap
68.9
41.6 answer sent.
FinQA
All operands
51.7
27.4 answer sent.
Appendix
Table 11: Pointer localization results. Values are percentages. “Novelty” subtracts similarity to the draft from question relevance.
Dataset
Model
Always-repair f
Always-repair d
m⋆
Gate-open rate
FinRank
Qwen3-8B
0.132
0.091
0.408
66.7%
FinRank
Qwen3-14B
0.153
0.100
0.395
61.5%
FinRank
Mistral-24B
0.141
0.094
0.400
59.8%
Fin-RATE
Qwen3-8B
0.104
0.086
0.453
58.9%
Fin-RATE
Qwen3-14B
0.117
0.089
0.432
54.7%
Fin-RATE
Mistral-24B
0.125
0.093
0.427
52.6%
Appendix
Table 12: Setting-specific break-even quantities from always-repair calibration trials.
Model
Method
DR
EC
LT
Macro CR
Qwen3-8B
Draft
12.8
0.4
3.4
5.53
Verify
13.2
0.4
3.6
5.73
SC-Verify
13.8
0.6
4.0
6.13
CAR
17.2
1.8
5.6
8.20
Qwen3-14B
Draft
26.4
1.6
5.8
11.27
Verify
26.8
1.8
6.2
11.60
Appendix
Table 13: Official Fin-RATE Correctness Rate by subtask on the fixed test partition. Macro CR is the unweighted mean of DR-QA, EC-QA, and LT-QA. All values are percentages.
Variant
NAA (%)
CAR advantage (pp)
Full CAR
72.97
—
Always repair
70.53
2.44
No pointer
70.27
2.70
Fixed checklist
72.10
0.87
Unbounded rewrite
68.96
4.01
Appendix
Table 14: FinQA NAA and CAR ’s advantage over each variant on Mistral-24B (percentage points).
Figure 4: Four generic-verification failures: missed period binding, a wrong-denominator revision, an irrelevant adjusted value, and presence-only checking.
Figure 5: CAR examples: local correction, no-op on a correct draft, targeted addition, and unit correction.
Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.
University of Groningen, Groningen, Netherlands · Goethe University Frankfurt, Frankfurt am Main, Germany · DataNXT GmbH, Frankfurt am Main, Germany +2