REGREACT: Self-Correcting Multi-Agent Pipelines for Structured Regulatory Information Extraction

Mohammed Ali; Abdelrahman Abdallah; Adam Jatowt

cs.MAApr 13, 2026

REGREACT: Self-Correcting Multi-Agent Pipelines for Structured Regulatory Information Extraction

Authors: Mohammed Ali, Abdelrahman Abdallah, Adam Jatowt

Organizations: University of Innsbruck, Austria

Abstract

Extracting structured, machine-readable compliance criteria from regulatory documents remains an open challenge. Single-pass language models hallucinate structural elements, lose hierarchical relationships, and fail to resolve inter-document dependencies. We introduce RegReAct, a self-correcting multi-agent framework that decomposes regulatory information extraction into seven specialized stages, each with an Observe--Diagnose--Repair (ODR) loop that validates outputs against the source, correcting not only model hallucinations but also cross-reference errors in the regulations themselves. To ensure structural accuracy, RegReAct constructs a typed criterion graph; to ensure completeness, it resolves external dependencies by retrieving, summarizing, and embedding referenced legal content inline, producing self-contained outputs. Applying RegReAct to three EU Taxonomy Delegated Acts, we construct a dataset comprising 242 activities with over 4,800 hierarchical criteria, thresholds, and enriched source summaries. Evaluation against a GPT-4o single-pass baseline shows that RegReAct outperforms it across all structural and semantic metrics.

Figures & tables

System	Task scope		Extraction approach		Output properties
System	Reg. text	Struct. output	Multi-stage decomp.	Iterative correction	Hier. nesting	Cross-ref. handling	Self- contained
XTRAREG ( Abualhaija et al., 2025 )	✓	✓	✗	✗	✗	$\sim$	✗
Galli et al. ( Galli et al., 2026 )	✓	✓	✓	✗	$\sim$	$\sim$	✗
GraphCompl. ( Chung et al., 2025 )	✓	✓	✓	✗	$\sim$	✓	✗
AgenticIE ( Colakoglu et al., 2025 )	$\sim$	✓	✓	✓	$\sim$	✗	✗
P2T ( Datla et al., 2025 )	✓	✓	✓	✓	✗	✗	✗

Table 1: Comparison of regulatory extraction systems on key problem dimensions. ✓ = full support,

\sim

= partial, ✗ = not addressed. Reg. = primary regulatory text; Struct. = structured output; Decomp. = multi-stage decomposition; Corr. = iterative correction; Hier. = hierarchical nesting; Cross-ref. = cross-reference handling; GraphCompl. = GraphCompliance ( Chung et al., 2025 ) .

Figure 1: Overview of the RegReAct pipeline. Each stage is executed by a specialized agent with an ODR self-correction loop. The shared semantic memory maintains cross-stage consistency, the criterion graph enforces structural invariants, and the RAG sub-pipeline resolves external references inline to produce self-contained output.

Metric	Mean	Std	Median	$n$
Structural & Classification (%)
Structural F1	94.12	3.56	95.2	100
Category Accuracy	98.6	2.62	100.0	100
Applicability Accuracy	97.24	3.16	98.4	100
Evaluation Logic Accuracy	93.4	3.60	94.0	100
Semantic Equivalence (1–5)

Table 2: Evaluation results for RegReAct on

n{=}100

gold-annotated activities. Structural and classification metrics are percentages; semantic equivalence uses a 1–5 GPT-4o judge scale (§5). Activities with all-zero scores are excluded, so

n

varies.

Metric	RegReAct	GPT-4o	$\Delta$
Structural & Classification (%)
Structural F1	94.12	78.6	+15.52
Category Accuracy	98.6	90.2	+8.4
Applicability Accuracy	97.24	85.7	+11.54
Evaluation Logic Accuracy	93.4	80.3	+13.1
Semantic Equivalence (1–5)

Table 3: RegReAct vs. GPT-4o single-pass baseline (

n{=}100

gold-annotated activities). Structural and classification metrics are percentages; semantic equivalence as mean Likert scores (1–5).

Metric	Mean	Std	Median
Faithfulness	4.61	1.13	5
Relevance	4.14	1.02	4
Completeness	4.07	0.85	4
Coverage	4.01	1.08	4

Table 4: RAG summary quality scores (GPT-4o judge, 1–5 Likert).

Iter.	Count (%)	Faith.	Relev.	Compl.	Cover.	Ret. Conf.
1	535 (75.5%)	4.74	4.30	4.17	4.30	0.92
2	88 (12.4%)	4.18	3.80	3.94	3.33	0.86
3	86 (12.1%)	4.28	3.48	3.64	2.93	0.67

Table 5: Quality by ReAct iteration count. Ret. Conf. is the pipeline’s self-assessed retrieval confidence.

Figure 2: Mean quality scores per retrieval confidence bin, averaged over criterion–source pairs. Faithfulness remains stable while relevance, completeness, and coverage increase monotonically.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Role	Primary Output
$s_{1}$	Structural Parser	Criterion hierarchy from HTML
$s_{2}$	Threshold Extractor	Quantitative & temporal thresholds
$s_{3}$	Content Classifier	Category, applicability, eval. logic
$s_{4}$	Reference Extractor	External sources & CELEX IDs
$s_{5}$	Dependency Resolver	Inter-criteria relationships
$s_{6}$	Footnote Processor	Footnote text & anchor linkage

Appendix

Table 6: Stages of the RegReAct pipeline. Each stage receives the cumulative extraction state and the original HTML, applying its specialized competency before passing control to the next stage.

Metric	GPT-4o	$-$ ODR	$-$ Graph	Full
Structural & Classification (%)
Structural F1	78.6	89.5	90.8	94.12
Category Acc.	90.2	96.3	98.1	98.6
Applicability Acc.	85.7	93.6	96.4	97.24
Eval Logic Acc.	80.3	89.2	87.6	93.4
Semantic Equivalence (1–5)

Appendix

Table 7: Ablation results on

n{=}100

gold-annotated activities. Structural and classification metrics are percentages; semantic equivalence uses a 1–5 GPT-4o judge scale.

Dimension	Cohen’s $\kappa$
Hierarchical structure	0.91
Dependencies (incl. cross-reference corrections)	0.86
Threshold extraction	0.84

Appendix

Table 8: Inter-annotator agreement on the shared 15-activity subset.

Explore similar work

ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuCompliance Regulation
The University of Hong Kong · Stellaris AI Limited
Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering
May 28, 2026Yeong-Joon Ju, Seong-Whan LeeCompliance Citations
Department of Artificial Intelligence, Korea University
Code-as-Auditor: Executable Compliance Reasoning via Regulation-to-Code
Sep 16, 2026Jisoo Kim, Taeyoon Kwack, Jinwoo Jang +2Compliance Legal Reasoning Tasks
Sungkyunkwan University Suwon, Republic of Korea

Apr 26, 2026, cs.CLJ/K move · Enter open · S save

ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection

Dongxin Guo, Jikun Wu, Siu Ming Yiu

The University of Hong Kong · Stellaris AI Limited

Compliance Regulation Banking

Open paper AskPDF arXiv

Financial institutions must track over 60,000 regulatory events annually, overwhelming manual compliance teams; the industry has paid over USD 300 billion in fines and settlements since the 2008 financial crisis. We present ComplianceNLP, an end-to-end system that automatically monitors regulatory changes, extracts structured obligations, and identifies compliance gaps against institutional policies. The system integrates three components: (1) a knowledge-graph-augmented RAG pipeline grounding generations in a regulatory knowledge graph of 12,847 provisions across SEC, MiFID II, and Basel III; (2) multi-task obligation extraction combining NER, deontic classification, and cross-reference resolution over a shared LEGAL-BERT encoder; and (3) compliance gap analysis that maps obligations to internal policies with severity-aware scoring. On our benchmark, ComplianceNLP achieves 87.7 F1 on gap detection, outperforming GPT-4o+RAG by +3.5 F1, with 94.2% grounding accuracy (

r=0.83

vs. human judgments) and 83.4 F1 under realistic end-to-end error propagation. Ablations show that knowledge-graph re-ranking contributes the largest marginal gain (+4.6 F1), confirming that structural regulatory knowledge is critical for cross-reference-heavy tasks. Domain-specific knowledge distillation (70B

\to

8B) combined with Medusa speculative decoding yields

2.8\times

inference speedup; regulatory text's low entropy (

H=2.31

bits vs.

3.87

general text) produces 91.3% draft-token acceptance rates. In four months of parallel-run deployment processing 9,847 updates at a financial institution, the system achieved 96.0% estimated recall and 90.7% precision, with a

3.1\times

sustained analyst efficiency gain. We report deployment lessons on trust calibration, GRC integration, and distributional shift monitoring for regulated-domain NLP.

Abstract

Explore similar work

ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection

Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering

Code-as-Auditor: Executable Compliance Reasoning via Regulation-to-Code