Whether an information extraction pipeline should process page images or parsed text depends on the document, and the answer flips across the layout spectrum. We study this trade-off under a constraint that rules out (closed) cloud services: privacy-sensitive documents processed on-premise by small (≤8B parameter) text-only and vision--language models, evaluated on both accuracy and energy over a design space spanning input representation, model family, and inference configuration. Benchmarking on the near-plain-text Kleister-NDA contracts and the layout-rich VRDU forms, we find that batching is the dominant energy lever, cutting energy per page by 38-85% at no cost in accuracy, while FP8 quantization saves 27-32% when requests are served one at a time but less than 1mWh per page (9-19%) once batching is applied. Preprocessing dominates what remains: neural OCR costs 17× more energy per page than classical OCR and never reaches the Pareto frontier. Which representation wins flips with the type of document: vision--language models on layout-rich documents and small text-only models with a cheap parser on near-plain text, where they are both more accurate and cheaper than any vision--language configuration. Our work yields concrete guidelines for energy-efficient, privacy-compliant local information extraction.
Figures & tables
Figure 1: Overview of the benchmarking design space. We investigate the dimensions ( D,S,I,M,C ) to evaluate the trade-offs between extracted structured information ( E ) and resource consumption ( R ).
Model
Modality
Eval. Mode
Specialized Models
Arctic-TILT Borchmann et al. (2025)
OCR+Layout
In-domain
NuExtract-2.0-4B NuMind (2026)
VL
Zero-shot
General-Purpose Models
Qwen3-VL-Instruct (2B, 4B, 8B) Qwen Team (2025b)
VL
Zero-shot
Llama-3.2-Instruct (1B, 3B) Grattafiori et al. (2024) Ministral-3-3B Mistral AI (2025) Mistral-7B Jiang et al. (2023) Qwen3 (0.6B, 1.7B, 4B, 8B) Qwen Team (2025a)
Text
Zero-shot
Table 1: Evaluated models. Only Arctic-TILT saw a target dataset during training; all others are zero-shot.
Figure 2: A visual comparison between a low-layout document (from the Kleister-NDA dataset) and a highly structured document (from the VRDU dataset). Models must leverage 2D spatial features to successfully extract information from the document on the right.
Figure 3: Baseline extraction accuracy and energy consumption per page on VRDU (top) and Kleister-NDA (bottom), at batch size 1 without quantization, using Tesseract-based PDF-to-Markdown conversion for all text-only models. Both panels share the model axis and the energy scale, so the two datasets can be read against each other. Energy covers model inference only; parsing energy is added in the end-to-end analysis of Section 5.4 . Arctic-TILT’s Kleister-NDA result is in-domain (fine-tuned on that corpus), shown for reference only and not part of any zero-shot comparison.
Kleister-NDA
VRDU
Tool
Total
/Pg.
Total
/Pg.
Tesseract
12.91
0.0043
4.87
0.0047
Docling
4.38
0.0015
12.95
0.0141
DeepSeek-OCR 2
220.49
0.0751
76.41
0.0835
Table 2: Total and per-page energy consumption (Wh) for PDF to Markdown conversion, over 500 documents per dataset. Per-page figures use each tool’s own page count, which differs because Tesseract rasterizes the PDFs (3016 pages on Kleister-NDA, 1025 on VRDU) while Docling and DeepSeek-OCR 2 follow the PDF page structure (2934 and 915).
Figure 4: Average field exact match for each model under the four input representations (dataset-provided OCR, DeepSeek-OCR 2, Docling, Tesseract), at batch size 10 without quantization. Kleister-NDA ships no baseline OCR. Arctic-TILT’s Kleister-NDA bars are in-domain (fine-tuned on that corpus). Underlying values are in Tables 3 and 4 .
Figure 5: Inference energy per page on VRDU across models served with vLLM, unquantized, at batch sizes (BS) from 1 to 60; text-only models consume Tesseract output. Parsing energy is excluded. Data labels above the bars give the minimum and maximum energy per page observed for each model across the evaluated batch sizes.
Figure 6: Energy per page and exact match for Qwen3-4B and Qwen3-VL-4B on VRDU, FP16 versus FP8, across batch sizes. Bars show the mean over three runs per configuration and error bars the standard deviation, which is mostly thinner than the bar outline. Exact match is averaged over runs and batch sizes, within which it varies by less than a point. It is Bench360’s own field exact match rather than the recomputed metric of Tables 3 and 4 , and in these runs both models used the vision arm’s sampling settings (Appendix A ), so absolute values differ slightly from the tables while the FP16-versus-FP8 difference is unaffected. The FP8 energy advantage is concentrated at batch size 1 and shrinks roughly tenfold by each model’s best batch size.
Figure 7: End-to-end accuracy–energy trade-off on VRDU (left) and Kleister-NDA (right). Energy per page includes both parsing and inference. Pareto-optimal configurations are enlarged, outlined in black, labelled, and joined by the dashed step line; all other measured configurations are shown at the same strength. The frontier is held by vision-language models on the layout-rich dataset and by text-only models with a cheap parser on the text-heavy one; no DeepSeek-OCR 2 configuration reaches either frontier. Frontiers cover zero-shot configurations only: Arctic-TILT is fine-tuned on Kleister-NDA and appears on that panel solely as an in-domain reference point, excluded from the frontier and from every zero-shot comparison.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Energy (mWh/pg)
Model
BS
OCR
EM (%)
FM (%)
F1 (%)
OCR
E2E
Arctic-TILT
10
DeepSeek
54.8
74.5
55.0
83.5
96.8
Arctic-TILT
10
Docling
57.9
74.0
58.0
14.1
18.7
Arctic-TILT
10
Tesseract
64.6
74.7
64.6
5.3
11.4
Llama-3.2-1B
10
DeepSeek
11.2
13.4
9.8
83.5
86.0
Llama-3.2-1B
10
Docling
18.6
31.2
17.6
14.1
16.4
Appendix
Table 3: Detailed benchmarking results for the VRDU dataset (Pareto plot configurations), over 500 documents. OCR (mWh/pg) is the parsing energy per page and E2E (mWh/pg) additionally includes model inference; both are normalized by the 1.83 pages per document of this dataset. The OCR text shipped with the VRDU benchmark is used for accuracy comparisons in Figure 4 but excluded here, since the energy of producing it is not attributable to our pipeline.
Energy (mWh/pg)
Model
BS
OCR
EM (%)
FM (%)
F1 (%)
OCR
E2E
Arctic-TILT †
10
DeepSeek
81.5
91.2
84.8
75.1
81.3
Arctic-TILT †
10
Docling
91.9
98.0
93.8
1.5
9.2
Arctic-TILT †
10
Tesseract
92.4
98.1
93.8
4.4
11.8
Llama-3.2-1B
10
DeepSeek
47.6
70.7
58.4
75.1
76.7
Llama-3.2-1B
10
Docling
42.0
64.6
53.2
1.5
2.6
Appendix
Table 4: Detailed benchmarking results for the Kleister-NDA dataset (Pareto plot configurations). Extraction is evaluated over 337 documents; parsing energy is amortized over the 500 documents of the parsing benchmark. Energy is per page, normalized by the 5.87 pages per document of this dataset. Arctic-TILT is fine-tuned on Kleister-NDA by its authors and is therefore evaluated in-domain, unlike every other model in the table; its rows are reported for reference and are excluded from best-configuration and Pareto comparisons.
We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.
Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
A. Said Gurbuz, Ahmed Nassar, Christoph Auer +8
IBM Research Zurich · ETH Zurich · IBM Research Zurich, Rüschlikon, Switzerland +1
Vision-language models (VLMs) are increasingly used to extract structured fields from business documents, yet most evaluations report accuracy on clean benchmarks and offer little guidance to practitioners choosing an approach for a given task complexity. We address this gap with a measurement-grounded study and an open-source release. Across eleven systems (three commercial, two reasoning, five open-source VLMs in pretrained and fine-tuned form, and a non-LLM OCR->regex floor) scored on a 750-document held-out pool of synthetic checks, fine-tuning on 3K samples lifts the best open-source VLMs above F1 0.98-above every zero-shot commercial system on this task-while GPT-5 leads the commercial pool on F1 and Claude Sonnet 4.5 collapses on Date. To turn these measurements into actionable choices, we introduce a practitioner-oriented selection framework that maps a task profile (quality, latency, governance, volume) to a recommended approach via filtering and total-cost minimization, illustrated on a hypothetical mid-volume document-extraction scenario.