Information Extraction from Electricity Invoices with General-Purpose Large Language Models
Authors: Javier Gómez, Javier Sánchez
Organizations: Centro de Tecnologías de la Imagen (CTIM) Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES) Universidad de Las Palmas de Gran Canaria 35017 Las Palmas de Gran Canaria, Spain · Centro de Tecnologías de la Imagen (CTIM) Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES) University of Las Palmas de Gran Canaria 35017 Las Palmas de Gran Canaria, Spain
Information extraction from semi-structured business documents remains a critical challenge for enterprise management. This study evaluates the capability of general-purpose Large Language Models to extract structured information from Spanish electricity invoices without task-specific fine-tuning. Using a subset of the IDSEM dataset, we benchmark two architecturally distinct models, Gemini 1.5 Pro and Mistral-small, across 19 parameter configurations and 6 prompting strategies. Our experimental framework treats prompt engineering as the primary experimental variable, comparing zero-shot baselines against increasingly sophisticated few-shot approaches and iterative extraction strategies. Results demonstrate that prompt quality dominates over hyperparameter tuning: the F1-score variation across all parameter configurations is marginal, while the gap between zero-shot and the best few-shot strategy exceeds 19 percentage points. The best configuration (few-shot with cross-validation) achieves an F1-score of 97.61% for Gemini and 96.11% for Mistral-small, with document template structure emerging as the primary determinant of extraction difficulty. These findings establish that prompt design is the critical lever for maximizing extraction fidelity in LLM-based document processing, thereby providing an empirical framework for integrating general-purpose LLMs into business document automation.
Figures & tables
Figure 1: Examples of electricity invoices from the IDSEM dataset. Despite containing similar content, each template organizes information differently using tables, boxes, and columns. Source: Sánchez et al. [ 13 ] .
ID
Description
Labels
A
Customer receiving the invoice
6
B
Customer data as stated in contract
6
C
Marketer (retailer) information
14
D
Distributor information
5
E
Contract details
9
F
General invoice information
8
Table 1: IDSEM label categories and their semantic content. Each category groups related fields that appear together in invoice documents.
Template
T1
T2
T3
T4
T5
T6
Extractable labels
75
62
53
68
51
52
Table 2: Number of true labels per template after ground-truth refinement.
Strategy
Paradigm
Description
Zero-shot
Zero-shot
No examples; relies solely on task instructions and field definitions to measure inherent LLM extraction capability.
Few-shot_v1
Few-shot
One annotated example with explicit label markers ( #A1 text #A1 ) signaling text-to-label correspondences.
Few-shot_v2
Few-shot
One non-annotated example (plain text), testing whether the model can infer correspondences from the input-output pair alone.
Cross-valid_v1
Few-shot
Three examples from templates T1, T2, T3. and then tested on templates T4, T5, T6 (sequential grouping).
Cross-valid_v2
Few-shot
Three examples from templates T1, T2, T4, and then tested on templates T3, T5, T6; groups separated by visual similarity.
Iterative
Iterative
Sequential field-by-field extraction with tailored instructions; increases API calls but enables fine-grained control.
Table 3: Summary of prompting strategies evaluated.
Configuration
Temperature
Top-P
Top-K
Deterministic (ID 0)
0.0
–
–
Grid search
1.0, 1.5, 2.0
0.5, 0.7, 0.95
32, 64
(IDs 1–18)
(all combinations: 3×3×2=18 )
Table 4: Inference parameter configurations evaluated. The grid search explores stochastic settings, while the deterministic baseline (ID 0) uses greedy decoding.
Phase
Configuration
Model
API Calls
Parameter analysis
Zero-shot (19 configs × 2,400 invoices)
Gemini 1.5 Pro
45,600
Few-shot_v1 (19 configs × 800 invoices)
Gemini 1.5 Pro
15,200
Prompting strategies
Non-iterative (5 strategies × 2,400 invoices)
Gemini 1.5 Pro
12,000
Few-shot_v1 (1,600 remaining invoices)
Gemini 1.5 Pro
1,600
Iterative (83 calls/invoice × 2,400 invoices)
Gemini 1.5 Pro
199,200
Non-iterative (5 strategies × 2,400 invoices)
Mistral-small
12,000
Table 5: Summary of experimental configurations and API calls per model.
Temp.
Top-K
Top-P
Precision
Recall
F1-Score
0.0
–
–
80.87%
75.31%
77.99%
1.0
32
0.50
81.40%
75.86%
78.54%
1.0
32
0.70
81.33%
76.00%
78.57%
1.0
32
0.95
81.34%
75.59%
78.36%
1.0
64
0.50
81.40%
75.89%
78.55%
1.0
64
0.70
81.40%
75.84%
78.52%
Table 6: Results of parameter tuning experiments on Gemini 1.5 Pro, showing Precision, Recall, and F1-Score for each configuration.
Strategy
Precision
Recall
F1-score
Zero-shot
81.27%
75.71%
78.39%
Few-shot_v1*
92.44%
87.88%
90.10%
Few-shot_v2*
91.81%
87.30%
89.50%
Cross-valid_v1
96.18%
99.09%
97.61%
Cross-valid_v2
95.24%
98.72%
96.95%
Iterative (IP)
94.45%
96.51%
95.47%
Table 7: Global metrics for Gemini 1.5 Pro across prompting strategies. Strategies marked with * exclude template T1 (used as example); IP excludes T1 and T2.
Strategy
Precision
Recall
F1-Score
Zero-shot
76.59%
90.39%
82.92%
Few-shot_v1*
90.31%
100.00%
94.91%
Few-shot_v2*
89.55%
99.98%
94.48%
Cross-valid_v1
92.61%
99.88%
96.11%
Cross-valid_v2
91.50%
99.75%
95.45%
Iterative (IP)
92.55%
98.14%
95.26%
Table 8: Global metrics for Mistral-small across prompting strategies.
Precision Range
Gemini 1.5 Pro
Mistral-small
99–100%
61.9%
55.7%
90–99%
20.6%
12.4%
80–90%
7.2%
11.3%
70–80%
2.1%
10.3%
60–70%
2.1%
3.1%
50–60%
2.1%
3.1%
Table 9: Distribution of labels by precision range for Cross-valid_v2 experiments.
Approach
Method
Seen Templates
Unseen Templates
Sánchez & Cuervo (2024)
SVM (RBF kernel)
91.86%
67.20%
This work (Gemini)
Cross-valid_v2
—
95.24%
This work (Mistral)
Cross-valid_v2
—
91.50%
Table 10: Comparison with classical ML baseline on IDSEM dataset. Classical ML was trained on templates T1–T6 and tested on T7; LLM cross-validation used examples from one template family (T1, T2, T4) and tested on structurally different templates (T3, T5, T6).
Multimodal Large Language Models (MLLMs) enhance the potential of natural language processing. However, their actual impact on document information extraction remains unclear. In particular, it is unclear whether an MLLM-only pipeline--while simpler--can truly match the performance of traditional OCR+MLLM setups. In this paper, we conduct a large-scale benchmarking study that evaluates various out-of-the-box MLLMs on business-document information extraction. To examine and explore failure modes, we propose an automated hierarchical error analysis framework that leverages large language models (LLMs) to diagnose error patterns systematically. Our findings suggest that OCR may not be necessary for powerful MLLMs, as image-only input can achieve comparable performance to OCR-enhanced approaches. Moreover, we demonstrate that carefully designed schema, exemplars, and instructions can further enhance MLLMs performance. We hope this work can offer practical guidance and valuable insight for advancing document information extraction.
Whether an information extraction pipeline should process page images or parsed text depends on the document, and the answer flips across the layout spectrum. We study this trade-off under a constraint that rules out (closed) cloud services: privacy-sensitive documents processed on-premise by small (≤8B parameter) text-only and vision--language models, evaluated on both accuracy and energy over a design space spanning input representation, model family, and inference configuration. Benchmarking on the near-plain-text Kleister-NDA contracts and the layout-rich VRDU forms, we find that batching is the dominant energy lever, cutting energy per page by 38-85% at no cost in accuracy, while FP8 quantization saves 27-32% when requests are served one at a time but less than 1mWh per page (9-19%) once batching is applied. Preprocessing dominates what remains: neural OCR costs 17× more energy per page than classical OCR and never reaches the Pareto frontier. Which representation wins flips with the type of document: vision--language models on layout-rich documents and small text-only models with a cheap parser on near-plain text, where they are both more accurate and cheaper than any vision--language configuration. Our work yields concrete guidelines for energy-efficient, privacy-compliant local information extraction.
Christoph Walser, Mauricio Fadel Argerich, Jonathan Fürst
Zurich University of Applied Sciences, Switzerland · Universidad Politécnica de Madrid, Spain
Vision-language models (VLMs) are increasingly used to extract structured fields from business documents, yet most evaluations report accuracy on clean benchmarks and offer little guidance to practitioners choosing an approach for a given task complexity. We address this gap with a measurement-grounded study and an open-source release. Across eleven systems (three commercial, two reasoning, five open-source VLMs in pretrained and fine-tuned form, and a non-LLM OCR->regex floor) scored on a 750-document held-out pool of synthetic checks, fine-tuning on 3K samples lifts the best open-source VLMs above F1 0.98-above every zero-shot commercial system on this task-while GPT-5 leads the commercial pool on F1 and Claude Sonnet 4.5 collapses on Date. To turn these measurements into actionable choices, we introduce a practitioner-oriented selection framework that maps a task profile (quality, latency, governance, volume) to a recommended approach via filtering and total-cost minimization, illustrated on a hypothetical mid-volume document-extraction scenario.