Document Information Extraction

Follow this topic Ask about this topic

Document information extraction studies how to convert documents into structured fields, entities, relations, events, and tables that can support search, analysis, and decision-making. Common approaches combine OCR and layout analysis with supervised sequence taggers, schema-guided language models, multimodal models, retrieval, and deterministic validation. Work spans receipts and forms as well as long clinical, financial, legal, and scientific documents, with benchmarks increasingly testing noisy inputs, layout understanding, source grounding, and output completeness. A central challenge is balancing flexible semantic interpretation with exact values, boundaries, and reliable evidence, while keeping systems efficient and robust across document types.

Latest papers

290 papers31 papers in the last four weeks, down 39% on the four weeks before. 0.2% of all new papers.

Mar 5, 2026

NERdME: a Named Entity Recognition Dataset for Indexing Research Artifacts in Code Repositories

Existing scholarly information extraction (SIE) datasets focus on scientific papers and overlook implementation-level details in code repositories. README files describe datasets, source code, and other implementation-level artifacts, however, their free-form Markdown offers little semantic structure, making automatic information extraction difficult. To address this gap, NERdME is introduced: 200 manually annotated README files with over 10,000 labeled spans and 10 entity types. Baseline results using large language models and fine-tuned transformers show clear differences between paperlevel and implementation-level entities, indicating the value of extending SIE benchmarks with entity types available in README files. A downstream entity-linking experiment was conducted to demonstrate that entities derived from READMEs can support artifact discovery and metadata integration.

Genet Asefa Gesese, Zongxiong Chen, Shufan Jiang +4

FIZ Karlsruhe Eggenstein-Leopoldshafen, Germany · Karlsruhe Institute of Technology Karlsruhe, Germany · Fraunhofer FOKUS Berlin, Germany +3

Document Information Extraction Named Entity Recognition

Mar 3, 2026

Learning to Generate and Extract: A Multi-Agent Collaboration Framework For Zero-shot Document-level Event Arguments Extraction

Document-level event argument extraction (DEAE) is essential for knowledge acquisition, aiming to extract participants of events from documents . In the zero-shot setting, existing methods employ LLMs to generate synthetic data to address the challenge posed by the scarcity of annotated data. However, relying solely on Event-type-only prompts makes it difficult for the generated content to accurately capture the contextual and structural relationships of unseen events. Moreover, ensuring the reliability and usability of synthetic data remains a significant challenge due to the absence of quality evaluation mechanisms. To this end, we introduce a multi-agent collaboration framework for zero-shot document-level event argument extraction (ZS-DEAE), which simulates the human collaborative cognitive process of "Propose-Evaluate-Revise." Specifically, the framework comprises a generation agent and an evaluation agent. The generation agent synthesizes data for unseen events by leveraging knowledge from seen events, while the evaluation agent extracts arguments from the synthetic data and assesses their semantic consistency with the context. The evaluation results are subsequently converted into reward signals, with event structure constraints incorporated into the reward design to enable iterative optimization of both agents via reinforcement learning.In three zero-shot scenarios constructed from the RAMS and WikiEvents datasets, our method achieves improvements both in data generation quality and argument extraction performance, while the generated data also effectively enhances the zero-shot performance of other DEAE models.

Guangjun Zhang, Hu Zhang, Yazhou Han +4

School of Computer and Information Technology, Shanxi University, Taiyuan, China · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan, China

Document Information Extraction Zero-Shot Learning Synthetic Data Generation for Language Models Multi-Agent LLM Systems

Mar 3, 2026

OCR or Not? Rethinking Document Information Extraction in the MLLMs Era with Real-World Large-Scale Datasets

Multimodal Large Language Models (MLLMs) enhance the potential of natural language processing. However, their actual impact on document information extraction remains unclear. In particular, it is unclear whether an MLLM-only pipeline--while simpler--can truly match the performance of traditional OCR+MLLM setups. In this paper, we conduct a large-scale benchmarking study that evaluates various out-of-the-box MLLMs on business-document information extraction. To examine and explore failure modes, we propose an automated hierarchical error analysis framework that leverages large language models (LLMs) to diagnose error patterns systematically. Our findings suggest that OCR may not be necessary for powerful MLLMs, as image-only input can achieve comparable performance to OCR-enhanced approaches. Moreover, we demonstrate that carefully designed schema, exemplars, and instructions can further enhance MLLMs performance. We hope this work can offer practical guidance and valuable insight for advancing document information extraction.

Jiyuan Shen, Peiyue Yuan, Atin Ghosh +2

SAP · Stanford University

Document Information Extraction Multimodal Large Language Models LLM Evaluation

Mar 2, 2026

CrimeNER: Curated Dataset and Methods for Research and Investigation in Language Intelligence

The extraction of critical information from crime-related documents is a crucial task for law enforcement agencies. The extraction of this information can be interpreted as a Named-Entity Recognition (NER) task. However, there is a considerable lack of adequately annotated data on general real-world crime scenarios. To address this issue, we present CrimeNER, a case study of crime-related NER, and a general crime-related Named-Entity Recognition database (CrimeNER-db), consisting of more than 1.5K annotated documents extracted from public reports of terrorist attacks and the US Department of Justice's press notes. We define 4 coarse types of crime entity and 21 fine-grained entity types. We address the quality of the presented database with fully supervised experiments using flat and nested NER architectures and zero- and few-shot experiments using Large Language Models (LLMs). The database is available on GitHub.

Miguel Lopez-Duran, Julian Fierrez, Aythami Morales +7

BiometricsAI, Universidad Autónoma de Madrid

Named Entity Recognition Document Information Extraction

Mar 2, 2026

LexChronos: An Agentic Framework for Structured Event Timeline Extraction in Indian Jurisprudence

Understanding and predicting judicial outcomes demands nuanced analysis of legal documents. Traditional approaches treat judgments and proceedings as unstructured text, limiting the effectiveness of large language models (LLMs) in tasks such as summarization, argument generation, and judgment prediction. We propose LexChronos, an agentic framework that iteratively extracts structured event timelines from Supreme Court of India judgments. LexChronos employs a dual-agent architecture: a LoRA-instruct-tuned extraction agent identifies candidate events, while a pre-trained feedback agent scores and refines them through a confidence-driven loop. To address the scarcity of Indian legal event datasets, we construct a synthetic corpus of 2000 samples using reverse-engineering techniques with DeepSeek-R1 and GPT-4, generating gold-standard event annotations. Our pipeline achieves a BERT-based F1 score of 0.8751 against this synthetic ground truth. In downstream evaluations on legal text summarization, GPT-4 preferred structured timelines over unstructured baselines in 75% of cases, demonstrating improved comprehension and reasoning in Indian jurisprudence. This work lays a foundation for future legal AI applications in the Indian context, such as precedent mapping, argument synthesis, and predictive judgment modelling, by harnessing structured representations of legal events.

Anka Chandrahas Tummepalli, Preethu Rose Anish

TCS Research

Legal NLP Document Information Extraction LLM Information Extraction

Feb 27, 2026

EDDA-Coordinata: An Annotated Dataset of Historical Geographic Coordinates

This paper introduces a dataset of enriched geographic coordinates retrieved from Diderot and d'Alembert's eighteenth-century Encyclopedie. Automatically recovering geographic coordinates from historical texts is a complex task, as they are expressed in a variety of ways and with varying levels of precision. To improve retrieval of coordinates from similar digitized early modern texts, we have created a gold standard dataset, trained models, published the resulting inferred and normalized coordinate data, and experimented applying these models to new texts. From 74,000 total articles in each of the digitized versions of the Encyclopedie from ARTFL and ENCCRE, we examined 15,278 geographical entries, manually identifying 4,798 containing coordinates, and 10,480 with descriptive but non-numerical references. Leveraging our gold standard annotations, we trained transformer-based models to retrieve and normalize coordinates. The pipeline presented here combines a classifier to identify coordinate-bearing entries and a second model for retrieval, tested across encoder-decoder and decoder architectures. Cross-validation yielded an 86% EM score. On an out-of-domain eighteenth-century Trevoux dictionary (also in French), our fine-tuned model had a 61% EM score, while for the nineteenth-century, 7th edition of the Encyclopaedia Britannica in English, the EM was 77%. These findings highlight the gold standard dataset's usefulness as training data, and our two-step method's cross-lingual, cross-domain generalizability.

Ludovic Moncla, Pierre Nugues, Thierry Joliveau +1

INSA Lyon, CNRS, Lyon 1 Université, LIRIS, UMR5205, 69621 Villeurbanne, France · Lund University, Lund Sweden · Université de Saint-Etienne, CNRS, EVS UMR5600, Saint-Etienne, France +1

Historical Document Processing Document Information Extraction

Feb 26, 2026

IDP Accelerator: Agentic Document Intelligence from Extraction to Compliance Validation

Understanding and extracting structured insights from unstructured documents remains a foundational challenge in industrial NLP. While Large Language Models (LLMs) enable zero-shot extraction, traditional pipelines often fail to handle multi-document packets, complex reasoning, and strict compliance requirements. We present IDP (Intelligent Document Processing) Accelerator, a framework enabling agentic AI for end-to-end document intelligence with four key components: (1) DocSplit, a novel benchmark dataset and multimodal classifier using BIO tagging to segment complex document packets; (2) configurable Extraction Module leveraging multimodal LLMs to transform unstructured content into structured data; (3) Agentic Analytics Module, compliant with the Model Context Protocol (MCP) providing data access through secure, sandboxed code execution; and (4) Rule Validation Module replacing deterministic engines with LLM-driven logic for complex compliance checks. The interactive demonstration enables users to upload document packets, visualize classification results, and explore extracted data through an intuitive web interface. We demonstrate effectiveness across industries, highlighting a production deployment at a leading healthcare provider achieving 98% classification accuracy, 80% reduced processing latency, and 77% lower operational costs over legacy baselines. IDP Accelerator is open-sourced with a live demonstration available to the community.

Md Mofijul Islam, Md Sirajus Salekin, Joe King +8

Amazon Web Services

Document Information Extraction

Feb 24, 2026

Beyond the Star Rating: A Scalable Framework for Aspect-Based Sentiment Analysis Using LLMs and Text Classification

Customer-provided reviews have become an important source of information for business owners and other customers alike. However, effectively analyzing millions of unstructured reviews remains challenging. While large language models (LLMs) show promise for natural language understanding, their application to large-scale review analysis has been limited by computational costs and scalability concerns. This study proposes a hybrid approach that uses LLMs for aspect identification while employing classic machine-learning methods for sentiment classification at scale. Using ChatGPT to analyze sampled restaurant reviews, we identified key aspects of dining experiences and developed sentiment classifiers using human-labeled reviews, which we subsequently applied to 4.7 million reviews collected over 17 years from a major online platform. Regression analysis reveals that our machine-labeled aspects significantly explain variance in overall restaurant ratings across different aspects of dining experiences, cuisines, and geographical regions. Our findings demonstrate that combining LLMs with traditional machine learning approaches can effectively automate aspect-based sentiment analysis of large-scale customer feedback, suggesting a practical framework for both researchers and practitioners in the hospitality industry and potentially, other service sectors.

Vishal Patil, Shree Vaishnavi Bacha, Revanth Yamani +2

University of Southern California, Marina del Rey, 90292, CA, USA.

Aspect-Based Sentiment Analysis Document Information Extraction

Feb 23, 2026

Agentic Framework for Political Biography Extraction

The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.

Yifei Zhu, Songpo Yang, Jiangnan Zhu +1

Ph.D candidate, Department of Politics and Public Administration, The University of Hong Kong. · Boya Postdoctoral Fellow, School of International Studies, Peking University. · Associate Professor, Department of Politics and Public Administration, The University of Hong Kong. +1

Document Information Extraction LLM Information Extraction Computational Social Science Agentic Retrieval Multi-Agent LLMs

Feb 23, 2026

Hyper-KGGen: A Skill-Driven Knowledge Extractor for High-Quality Knowledge Hypergraph Generation

Knowledge hypergraphs surpass traditional binary knowledge graphs by encapsulating complex n-ary atomic facts, providing a more comprehensive paradigm for semantic representation. However, constructing high-quality hypergraphs remains challenging due to the scenario gap: generic extractors struggle to generalize across diverse domains with specific jargon, while existing methods often fail to balance structural skeletons with fine-grained details. To bridge this gap, we propose Hyper-KGGen, a skill-driven framework that reformulates extraction as a dynamic skill-evolving process. First, Hyper-KGGen employs a coarse-to-fine mechanism to systematically decompose documents, ensuring full-dimensional coverage from binary links to complex hyperedges. Crucially, it incorporates an adaptive skill acquisition module that actively distills domain expertise into a Global Skill Library. This is achieved via a stability-based feedback loop, where extraction stability serves as a relative reward signal to induce high-quality skills from unstable traces and missed predictions. Additionally, we present HyperDocRED, a rigorously annotated benchmark for document-level knowledge hypergraph extraction. Experiments demonstrate that Hyper-KGGen significantly outperforms strong baselines, validating that evolved skills provide substantially richer guidance than static few-shot examples in multi-scenario settings.

Rizhuo Huang, Yifan Feng, Rundong Xue +5

Xi’an Jiaotong University State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics Xi’an, China · Tsinghua University Beijing, China · Shanghai University Shanghai, China +1

KG Construction Document Information Extraction

Feb 23, 2026

NovaLAD: A Fast, CPU-Optimized Document Extraction Pipeline for Generative AI and Data Intelligence

Document extraction is an important step before retrieval-augmented generation (RAG), knowledge bases, and downstream generative AI can work. It turns unstructured documents like PDFs and scans into structured text and layout-aware representations. We introduce NovaLAD, a comprehensive document parsing system that integrates two concurrent YOLO object detection models - element detection and layout detection - with rule-based grouping and optional vision-language enhancement. When a page image is sent in, the first thing that happens is that it goes through both models at the same time. The element model finds semantic content like the title, header, text, table, image, and so on, and the layout model finds structural regions like layout_box, column_group, multi_column, row_group, and so on. A key design decision is to first send an image or figure through an image classifier (ViT) that decides whether it is relevant or not. Only useful images are then submitted to the Vision LLM for title, summary, and structured information, which cuts down on noise and costs. NovaLAD is built for speed: it works on CPU, employs parallel execution for detection, classification, OCR, and conversion, and generates several forms, including structured JSON, Markdown, RAG-ready texts, and knowledge graphs. We test on the DP-Bench benchmark (upstage/dp-bench) and get 96.49% TEDS and 98.51% NID, which is better than both commercial and open-source parsers. This paper explains how to extract data, how the architecture works, how data flows, and how to make NovaLAD both accurate and usable without needing a GPU.

Aman Ulla

Document Information Extraction Document Parsing Document Layout Analysis

Feb 20, 2026

Automating Document Intelligence in Statutory City Planning

UK planning authorities face a legislative conflict between the Planning Act, which mandates public access to application documents, and the Data Protection Act, which requires protection of personal information. This situation creates a manually intensive workload for processing large document volumes, diverting planning officers to administrative tasks and creating legal compliance risks. This paper presents an integrated AI system designed to address these challenges. The system automates the identification and redaction of personal information, extracts key metadata from planning documents, and analyzes architectural drawings for specified features. It operates with an AI-in-the-Loop (AI2L) design, presenting all suggestions for review and confirmation by planning officers directly within their existing software; no action is committed without explicit human approval. The system is designed to improve its performance over time by learning from this human oversight through active learning prioritization rather than autoapproval. The system is currently being piloted at four diverse UK local authorities. The paper details the system design, the AI2L workflow, and the evaluation framework used in the pilot. Additionally, it describes a preliminary Return on Investment (ROI) model developed to quantify potential savings and secure partner participation. This work provides a case study on deploying AI to reduce administrative burden and manage compliance risk in a public sector environment.

Lars Malmqvist, Robin Barber

Research and Implementation, Copenhagen, Denmark · Arcus Global, Cambridge, UK

Workflow Automation Document Information Extraction Urban Planning Human-in-the-Loop AI

Feb 18, 2026

Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems

Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study two training strategies for building multilingual OCR systems with Vision-Language Models through the Chitrapathak series. We first follow a popular multimodal approach, pairing a generic vision encoder with a strong multilingual language model and training the system end-to-end for OCR. Alternatively, we explore fine-tuning an existing OCR model, despite not being trained for the target languages. Through extensive evaluation on multilingual Indic OCR benchmarks and deployment-oriented metrics, we find that the second strategy consistently achieves better accuracy-latency trade-offs. Chitrapathak-2 achieves 3-6x speedup over its predecessor with being state-of-the-art (SOTA) in Telugu (6.69 char ANLS) and second best in the rest. In addition, we present Parichay, an independent OCR model series designed specifically for 9 Indian government documents to extract structured key fields, achieving 89.8% Exact Match score with a faster inference. Together, these systems achieve SOTA performance and provide practical guidance for building production-scale OCR pipelines in the Indian context.

Ali Faraz, Raja Kolla, Ashish Kulkarni +1

Krutrim AI, Bangalore, India

Multilingual OCR Optical Character Recognition Document Information Extraction Low-Resource Language Processing Efficient VLM Inference

Feb 16, 2026

ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation

Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-in ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 18 000+ unique schemas across 15 named languages plus a long-tail Other category, with English and Traditional Chinese covering 88% of detected content, each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels (semantic correctness is out of scope, and raw HTML is deferred beyond v1.0). We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support.

William Brach, Francesco Zuppichini, Marco Vinciguerra +1

ScrapeGraphAI, San Francisco, United States

Schema-Constrained Generation LLM Information Extraction Text Generation Document Information Extraction Language Model Generation Evaluation

Jan 26, 2026

Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction

Document-level Information Extraction (DocIE) aims to produce an output template with the entities, relations, and events of interest occurring in the given document. Standard practices include prompting decoder-only LLMs using greedy decoding to avoid output variability. Rather than treating this variability as a limitation, we show that sampling can produce substantially better solutions than greedy decoding, especially when using reasoning models. We thus propose ThinkTwice, a sampling and selection framework in which the LLM generates multiple candidate templates for a given document, and a selection module chooses the most suitable one. We introduce both an unsupervised method that exploits agreement across generated outputs, and a supervised selection method using reward models trained on labeled DocIE data. To address the scarcity of golden reasoning trajectories for DocIE, we propose a rejection-sampling-based method to generate silver training data that pairs output templates with reasoning traces. Our experiments show the validity of unsupervised and supervised ThinkTwice, consistently outperforming greedy baselines and the supervised state-of-the-art.

Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle +1

HiTZ Center - Ixa, University of the Basque Country UPV/EHU

Self-Consistency Decoding Document Information Extraction Best-of-N Sampling LLM Information Extraction

Jan 8, 2026

BioPIE: A Biomedical Protocol Information Extraction Dataset for Experiment Understanding

Understanding biomedical experiments provides a foundation for downstream tasks, e.g., laboratory automation, and facilitates effective cross-disciplinary communication. Two challenges, High Information Density (HID) and Multi-Step Reasoning (MSR), pose unique difficulties for precise experimental understanding. Extracting structured knowledge, e.g., Knowledge Graphs (KGs), is an effective approach to address the HID and MSR. However, existing biomedical datasets for structured knowledge information extraction are limited to a general or coarse-grained level, hindering fine-grained experimental understanding. To address this gap, we introduce Biomedical Protocol Information Extraction Dataset (BioPIE), a dataset providing procedure-centric KGs that capture entities, actions, and relations at a scale sufficient for reasoning across biomedical protocols. We evaluate information extraction methods on BioPIE and implement a question answering system leveraging the dataset for validation, demonstrating improved understanding performance on test sets as well as on the HID and MSR question sets.

Haofei Hou, Shunyi Zhao, Fanxu Meng +3

School of Advanced Manufacturing and Robotics, Peking University · School of Integrated Circuits, Peking University

KG Construction Document Information Extraction Biomedical KGs

Oct 28, 2025

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

Wide access to advanced experimental methods in materials science has given rise to an abundance of procedural knowledge, which is scattered across decades of scientific literature and recorded in unstructured formats that are challenging to analyze systematically. In this work, we present LeMat-Synth Parser, a modular, open-source, and multi-modal extraction toolbox that utilizes large language models (LLMs) and vision language models (VLMs) to automatically structure synthesis protocols and performance metrics extracted from both text and figures of publications. Applying LeMat-Synth Parser to 81K open-access publications, we curate LeMat-Synth, an extensive dataset of 58K synthesis procedures and to our knowledge the largest and most diverse structured inorganic materials synthesis dataset to date, covering 35 synthesis methods and 16 material classes based on a domain-specific ontology. We validate extraction quality against annotations by domain experts and a scalable LLM-as-a-judge framework, and benchmark a suite of models to identify optimal configurations and characterize cross-model biases. To demonstrate the extensibility of LeMat-Synth Parser, we apply it to two distinct domains. First, we link synthesis protocols and catalyst identity to thermocatalytic performance across a corpus of ammonia-decomposition publications. Second, we cross-validate text- and figure-reported critical transition temperatures across 1,384 superconductivity papers, then use the validated pipeline to recover the critical transition temperature for every composition in a sample series. We release LeMat-Synth Parser and the LeMat-Synth dataset openly on GitHub and Hugging Face

Magdalena Lederbauer, Siddharth Betala, Valerie Gentzke +22

Entalpic · ENSA Lyon UMR-MAP Aria · Independent Researcher +8

Document Information Extraction Materials Science LLM Information Extraction

Jul 5, 2025

POSID: Target-Task-Unsupervised Person-Attribute Extraction from Heterogeneous Public-Safety Text

Person descriptions in public-safety text can be compact and regular, or distributed across heterogeneous narratives in which evidence boundaries and person ownership are ambiguous. We present POSID, a target-task-unsupervised framework that decomposes person-attribute extraction into candidate localization, provenance-controlled extraction scope, structured recovery and traversal, and person association, without supervised training or fine-tuning on the target task. We evaluate six localization configurations under two traversal policies across three real-use cohorts and two controlled synthetic diagnostics. On structurally regular descriptions, the strongest configurations reach 1.000/.972 Attr-Only (AO)/Attr-Value (AV) F1 on Compact500. Across the real-use cohorts, every RE-containing configuration benefits from conservative early stopping, while semantic-only retrieval can benefit from continued traversal when localization is less constrained. On the heterogeneous multi-person Challenge14 cohort, the best automatic AO and AV results are .500 and .382; replacing automatic candidates with verified evidence raises the frozen extractor to .522/.410 under early stopping, showing that localization is one of the material bottlenecks. We further introduce provenance-aware scoring for collective and underspecified evidence and a separate person-aware diagnostic for ownership and fragmentation. Together, the results establish a measurable operating regime for inspectable person-attribute extraction and distinguish errors due to localization, traversal, structured recovery, and attribution. Code: https://github.com/ksolaiman/hart-posid.

KMA Solaiman

Department of Computer Science, Purdue University, West Lafayette, IN, USA

Document Information Extraction

May 24, 2025

Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs

Extracting structured procedural knowledge from unstructured business documents is a critical yet unresolved bottleneck in process automation. While prior work has focused on extracting linear action flows from instructional texts, such as recipes, it has insufficiently addressed the complex logical structures, including conditional branching and parallel execution, that are pervasive in real-world regulatory and administrative documents. Furthermore, existing benchmarks are limited by simplistic schemas and shallow logical dependencies, restricting progress toward logic-aware large language models.To bridge this Logic Gap, we introduce BREX, a carefully curated benchmark comprising 409 real-world business documents and 2,855 expert-annotated rules. Unlike prior datasets centered on narrow service scenarios, BREX spans over 30 vertical domains, covering scientific, industrial, administrative, and financial regulations. We further propose ExIde, a structure-aware reasoning framework that investigates five distinct prompting strategies, ranging from implicit semantic alignment to executable grounding via pseudo-code generation. This enables explicit modeling of rule dependencies and provides an out-of-the-box framework for different business customers without finetuning their own large language models. We benchmark ExIde using 13 state-of-the-art large language models. Our extensive evaluation reveals that executable grounding serves as a superior inductive bias, significantly outperforming standard prompts in rule extraction. In addition, reasoning-optimized models demonstrate a distinct advantage in tracing long-range and non-linear rule dependencies compared to standard instruction-tuned models.

Chen Yang, Ruping Xu, Ruizhe Li +2

Zhejiang University of Technology, China1 · Zhejiang Key Laboratory of Visual Information Intelligent Processing, China 2 · University of Aberdeen, UK +1

LLM Grounding LLM Prompting Document Information Extraction Business Process Modeling

Dec 7, 2024

Auto-AdvER: Disentangling Condition, History, and Commercial Offers in Automotive Advertisements

This paper presents a case study on the development of Auto-AdvER, a specialised span-labelling schema, along with its corresponding dataset for the textual car advertisement domain. Developed with industry needs in mind, Auto-AdvER is designed to enhance text mining analytics in this domain and contributes a linguistically unique sequence-labelling dataset. We introduce a schema consisting of three labels: 'Condition', 'Historic', and 'Sales Options'. We outline the guiding principles for annotation, describe the methodology for schema development, and present the results of an annotation study demonstrating an inter-annotator agreement of 78%. Furthermore, we compare the performance of various models, including transformer encoder models (BERT, DeBERTaV3) and transformer decoder-based open- and closed-source large language models: Llama, Qwen, GPT, and Gemini, as well as conditional random fields. Our results show that a fine-tuned DeBERTaV3 encoder achieves the highest F1-score, followed closely by conditional random fields, while the large language models reach the highest precision but lower recall. We present this work as a stepping stone towards more fine-grained analysis and discuss Auto-AdvER's potential impact on advertisement analytics, pricing, and customer insights. Potential applications include the analysis of market dynamics and data-driven predictive maintenance.

Filippos Ventirozos, Ioanna Nteka, Tania Nandy +4

Manchester Metropolitan University, John Dalton tower, Chester St, Manchester M1 5GD, UK · Auto Trader UK, 1 Tony Wilson Place, Manchester M15 4FN, UK

Document Information Extraction Product Attribute Extraction Named Entity Recognition