Vision-Language Models (VLMs) have demonstrated strong performance in structured document extraction, yet their deployment in high-stakes applications requires reliable confidence estimates for individual fields. Existing uncertainty estimators often aggregate signals across an entire generation, obscuring field-specific reasoning uncertainty while overlooking visual degradations that may lead to confident extraction errors. To address these limitations, we introduce FiReIQ, a field-level confidence estimation framework that combines field-aligned reasoning signals from a single VLM generation with local and global image-quality features through quality-conditioned visual fusion. To facilitate systematic evaluation, we further introduce FiReIQ-Bench, a benchmark spanning financial receipts and legal contracts. Experiments with two reasoning VLMs show that FiReIQ consistently improves confidence discrimination, selective prediction, and probabilistic accuracy over existing estimators. These gains extend to private financial service data, where FiReIQ improves AUROC by 6.8% and coverage at 95% precision by 10.6% over an existing baseline. Together, these results highlight the potential of FiReIQ to reduce manual review while enabling more reliable document automation.
Figures & tables
Figure 1: Overview of FiReIQ . (a) A reasoning VLM extracts all fields in a single pass with structured decoding and returns, for each field, its value, an excerpt of its reasoning, and the token probabilities. The confidence model combines these with the page image to produce a calibrated confidence, which decides whether the field is accepted automatically or sent to human review. (b) Inside the confidence model, per-field reasoning signals and per-page image signals are encoded and projected. The image-quality profile modulates the visual features through FiLM before fusion, and an MLP followed by calibration yields the final confidence.
Dataset
VLM
# Pages
Train
Cal.
Test
Acc.
CORD
Gemma 26B
1,000
22,132
2,787
2,722
92.3
Nemotron 120B
776
13,271
1,709
1,671
89.9
CUAD
Gemma 26B
198
3,926
639
494
59.2
Nemotron 120B
198
5,182
755
450
58.4
Table 1: Statistics of the field-level confidence datasets. Train, Cal., and Test denote the number of field-level examples in each split. Acc. (%) denotes extraction accuracy.
Gemma 26B
Nemotron 3.5 Super VL 120B
Method
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
CORD
Sample Agreement
0.0367
0.6648
0.8635
0.2115
0.0264
0.0641
0.0503
0.7375
0.7115
0.0156
0.0254
0.0873
Answer LogProb
0.0544
0.4960
0.9869
0.0018
0.0179
0.0489
0.1083
0.4324
0.0000
0.0000
0.0384
0.0879
Reasoning LogProb
0.0545
0.5593
0.2661
0.0419
0.0271
0.0651
0.1051
0.4923
0.0030
0.0030
0.0374
0.1008
Reasoning Trajectory
0.0334
0.7316
0.9164
0.1832
0.0210
0.0583
0.0614
0.6752
0.4172
0.0355
0.0310
0.0919
Table 2: Comparison of field-level confidence estimation methods on FiReIQ-Bench . AURC measures selective prediction risk, AUROC measures confidence discrimination, Cov@95 and Cov@98 denote coverage at 95% and 98% precision, respectively, while ECE and Brier evaluate probability quality. Best results within each dataset and backbone are in bold.
Gemma 26B
Nemotron 3.5 Super VL 120B
Reasoning
AURC ↓
AUROC ↑
Cov@98 ↑
Brier ↓
AURC ↓
AUROC ↑
Cov@98 ↑
Brier ↓
CORD
Full Trajectory
0.0566
0.5395
0.0355
0.0631
0.0792
0.5662
0.0181
0.0834
Field-Specific (Ours)
0.0166
0.8736
0.8684
0.0387
0.0187
0.8907
0.6900
0.0460
CUAD
Full Trajectory
0.2976
0.6034
0.0175
0.2303
0.3350
0.5450
0.0172
0.2332
Table 3: Effect of field-specific reasoning attribution. Full-trajectory reasoning uses the complete generation trajectory, whereas field-specific reasoning uses the reasoning attributed to each extracted field. Best results within each dataset and VLM are in bold.
Figure 2: Effect of image-quality representations on CUAD. We compare no explicit quality features, a single sharpness metric, six global metrics, 384 tile-level measurements, and their combined 390-dimensional representation. Lower AURC is better.
Gemma 26B
Nemotron 3.5 120B
Method
AURC ↓
Cov@98 ↑
AURC ↓
Cov@98 ↑
Direct Fusion
0.0320
0.4900
0.0460
0.2120
FiLM (Ours)
0.0290
0.5910
0.0410
0.3670
Table 4: Comparison of image-quality integration strategies on pooled CORD and CUAD results. Lower AURC and higher Cov@98 are better.
Method
AUROC ↑
Cov@95 ↑
Answer LogProb
54.5
–
Reasoning LogProb
67.1
–
Learned Baseline
91.3
74.5
FiReIQ
98.1
85.1
Table 5: Confidence estimation on private financial production data. AUROC and Cov@95 are reported as percentages; higher is better.
Metric
Definition
Edge acutance
Mean Sobel gradient magnitude over the strongest 5% of pixels, reflecting the steepness of edges.
Frequency sharpness
Fraction of Fourier coefficients whose magnitude exceeds 1/1000 of the maximum [ 12 ] .
Entropy
Shannon entropy of the 256-bin intensity histogram.
Perceptual blur
No-reference blur score in [0,1] comparing intensity variations before and after a 9-pixel box blur [ 11 ] .
Contrast
Standard deviation of pixel intensities.
Noise level
Fast noise standard-deviation estimate based on a Laplacian-difference kernel [ 29 ] .
Table 6: Image quality metrics computed for each tile and for the full page.
Field
Definition
Match
Document Name
Title of the contract
Text
Parties
Parties who signed the contract
Parties
Agreement Date
Date on which the contract was signed
Date
Effective Date
Date on which the contract takes effect
Date
Expiration Date
Date on which the initial term expires
Date
Renewal Term
Renewal term after the initial term expires
Term
Table 7: CUAD extraction fields and matching criteria.
Field
Definition
Match
Purchased items (list-valued)
menu-nm
Item name
Text
menu-num
Item code
ID
menu-unitprice
Unit price
Numeric
menu-cnt
Quantity
Numeric
menu-discountprice
Item discount
Numeric
Table 8: CORD extraction fields and matching criteria. Field names follow the CORD v2 schema.
Gemma 26B
Nemotron 120B
Method
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
Sample Agreement
0.0367
0.6648
0.8635
0.2115
0.0264
0.0641
0.0503
0.7375
0.7115
0.0156
0.0254
0.0873
Answer LogProb (Mean)
0.0544
0.4960
0.9869
0.0018
0.0179
0.0489
0.1083
0.4324
0.0000
0.0000
0.0384
0.0879
Answer LogProb (Max)
0.0551
0.5236
0.9967
0.0050
0.0179
0.0491
0.0808
0.4718
0.0197
0.0173
0.0376
0.0878
Answer LogProb (Min)
0.0550
0.4879
0.9743
0.0018
0.0179
0.0489
0.1137
0.4037
0.0000
0.0000
0.0401
0.0881
Answer Entropy
0.0514
0.4950
0.9888
0.0000
0.0179
0.0489
0.1160
0.4320
0.0018
0.0018
0.0390
0.0880
Table 9: Complete field-level confidence baseline comparison on CORD. AURC, ECE, and Brier are lower-is-better; AUROC, Cov@95, and Cov@98 are higher-is-better. Best values are highlighted in bold.
Gemma 26B
Nemotron 120B
Method
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
AURC ↓
AUROC ↑
Cov@95 ↑
Cov@98 ↑
ECE ↓
Brier ↓
Sample Agreement
0.3998
0.4640
0.0076
0.0076
0.0902
0.2499
0.3842
0.5067
0.0038
0.0038
0.0898
0.2467
Answer LogProb (Mean)
0.2464
0.4292
0.0158
0.0090
0.1175
0.1824
0.3506
0.3877
0.0042
0.0042
0.0564
0.2024
Answer LogProb (Max)
0.2450
0.4921
0.0554
0.0501
0.1175
0.1824
0.2762
0.5032
0.0127
0.0127
0.0564
0.2025
Answer LogProb (Min)
0.2666
0.4024
0.0158
0.0090
0.1175
0.1824
0.3641
0.3718
0.0042
0.0042
0.0564
0.2027
Answer Entropy
0.2511
0.4253
0.0000
0.0000
0.1175
0.1824
0.3492
0.3888
0.0021
0.0021
0.0564
0.2024
Table 10: Complete field-level confidence baseline comparison on CUAD. AURC, ECE, and Brier are lower-is-better; AUROC, Cov@95, and Cov@98 are higher-is-better. Best values are highlighted in bold.