AuDQ: Auditable Data Quality through Agentic Rule Discovery

Jacopo Biggiogera; Nicolas Colombo; Krati Saxena; Bhaskarjit Sarmah; Stefano Pasquali

cs.AIOct 19, 2026

AuDQ: Auditable Data Quality through Agentic Rule Discovery

Authors: Jacopo Biggiogera, Nicolas Colombo, Krati Saxena, Bhaskarjit Sarmah, Stefano Pasquali

Organizations: Domyn, Milan, Italy · Domyn, Gurugram, India · Domyn, New York City, USA

Abstract

High-stakes tabular pipelines rely on scattered, static quality checks that are hard to evolve and audit against policy intent. Regulated domains particularly suffer as the LLM-era automation raises expectations for traceability and accountability. We propose a vision for an ontology-grounded, repository-native data quality layer in which rules are first-class artifacts: versioned, provenance-linked to governance texts, and re-playable through immutable execution traces. As a first step toward validating this vision, we present a concrete implementation of its agentic rule-discovery core with two rule acquisition modes (Full-Agentic and Guided) and a human-in-the-loop (HITL) review interface. We define an evaluation framework and run experiments addressing whether our fully autonomous, divide-and-conquer agentic rule discovery produces rules of the same quality and diversity as those specified by a human expert, reporting preliminary results on two real world industry datasets in the finance and retail domains.

Figures & tables

Requirement	What it enforces / example
Syntactic validity	Types, ranges, formats across flat files and joins; referential integrity via keys/views.
Semantic correctness	Ontology-grounded invariants; e.g., notional $>0$ consistent with product type in finance via trade $\to$ instrument $\to$ client, or a medication dose within an age-adjusted safe range in healthcare.
Temporal consistency	No future-dated events relative to ingestion; consistent time-series snapshots across grains.
Cross-table reconciliation	Aggregates align with primaries; e.g., ledger totals vs. front-office trades, or risk vs. accounting views.

Table 1: Core data-quality requirements for analytics-aware rules.

Dataset	Mode	$\hat{p}_{\mathrm{good}}$
Berka	Full agentic	0.890
Berka	Guided	0.880
H&M	Full agentic	0.922
H&M	Guided	0.850
Overall	Full agentic	0.906
Overall	Guided	0.865

Table 2: Mean good-rule rate across runs per dataset and mode.

Matched-scope control
Comparison	HL	90% CI	$r$	Power
Berka	$+0.004$	$[-0.112,+0.135]$	$0$	$<1\%$
H&M	$+0.064$	$[+0.029,+0.129]$	$-1$	$36.5\%$
Overall	$+0.037$	$[-0.008,+0.089]$	$-0.50$	$27.8\%$
Overall	$+0.037$	$[-0.027,+0.094]$	$-0.38$	$3.5$

Table 3: Non-parametric TOST equivalence analysis for RQ1 good-rule rate (Full Agentic

-

Guided); equivalence margin

\Delta=0.0556

. HL: Hodges–Lehmann shift with exact CI from the inverted Mann-Whitney statistic;

r

: rank-biserial effect size (negative favor Full Agentic mode); Power: achieved TOST power. Runs required for 80% power: 28 (Berka), 6 (H&M), 16 (Overall). The matched-scope: rules with column scope shared by all Guided runs.

Subset	$n$	Exact	$\alpha$ [95% CI]	AC2 [95% CI]
Overall	541	77.3	0.174 [.074,.279]	0.812 [.770,.851]
Berka	317	76.0	0.165 [.035,.292]	0.810 [.754,.858]
H&M	224	79.0	0.189 [.021,.350]	0.815 [.747,.872]
Agentic	340	76.8	0.061 [ $-$ .056,.187]	0.816 [.764,.864]
Guided	201	78.1	0.326 [.150,.484]	0.805 [.730,.868]

Table 4: Inter-rater agreement over doubly-rated rules. Exact : % rules where both reviewers gave same verdict.

\alpha

: ordinal Krippendorff’s

\alpha

; AC2: Gwet’s AC2; both with

95\%

bootstrap intervals.

\alpha

is deflated relative to AC2 because accept accounts for

84.8\%

of all ratings.

Weight config	$\mathrm{IDS}_{A}$	$\mathrm{IDS}_{G}$
Equal	0.732	0.729
Sem.-heavy	0.648	0.648
Syn.-heavy	0.749	0.750
Scope-heavy	0.798	0.788

Table 5: Mean IDS across datasets (

n_{A}=n_{G}=8

runs per mode).

Weights	HL	90% CI	$r$	Power
Equal	$-0.003$	$[-0.010,+0.012]$	$+0.16$	$57\%$
Semantic	$-0.005$	$[-0.015,+0.008]$	$+0.22$	$44\%$
Syntax	$-0.005$	$[-0.016,+0.010]$	$+0.25$	$71\%$
Scope	$+0.003$	$[-0.004,+0.019]$	$-0.28$	$53\%$

Table 6: Non-parametric TOST equivalence analysis for RQ2 IDS (Full Agentic

-

Guided); equivalence margin

\Delta=0.0202

. HL: Hodges–Lehmann shift with exact CI from the inverted Mann-Whitney statistic;

r

: rank-biserial effect size. Runs required for 80% power: 11 (Equal), 13 (Semantic), 9 (Syntax), 11 (Scope).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Rule type	Finance example	Grounding
Calendar and market hours	No equity trades on Sunday. Trades outside market hours must be flagged or explained.	Market calendar, policy
Temporal consistency	Settlement date must be after trade date. For many products it follows a fixed lag such as T plus 2, with exceptions.	Policy, product spec
Cross field meaning	Notional should match quantity times price within tolerance and currency scaling.	Domain math, evidence
Entity state	Counterparty must be active on trade date. Trades for blocked accounts must be rejected.	Reference data, policy
Cross table reconciliation	Trade totals must reconcile with positions and ledger aggregates at the portfolio level.	Reporting definition

Appendix

Table 7: Examples of policy grounded semantic rules that go beyond traditional checks.

Agent	Descriptive	Diagnostic	Predictive	Prescriptive
Ontology Gen.	Grounds columns into entities, measures, and units.	Adds semantic context for anomaly explanation.	Marks forecastable measures and derived metrics.	Enables domain-aware imputation and unit normalization.
Rule Discovery	Drafts checks with clear summaries and coverage metrics.	Encodes conditional/composite rules for diagnosis.	Proposes drift-sensitive and stability-aware rules.	Suggests adaptive thresholds and remediation-aware rules.
Implementer	Translates approved rules into executable code.	Generates diagnostic checks capturing failure evidence.	Produces code artifacts that accumulate execution history for forecasting.	Emits validated, controlled fixes to apply safeguards/remediation.
Human review	Confirms expected vs. concerning signals.	Confirms or rejects proposed causes.	Decides which predicted risks are actionable.	Approves, edits, or blocks remediation strategies.
EDRA	Executes checks and visualizes quality metrics and trends.	Captures failure evidence and supports drill-down/lineage views.	Collects execution history and displays risk scores and drift forecasts.	Presents recommended actions for review.
RCADA	Consumes descriptive trace summaries.	Produces ranked RCA hypotheses with evidence.	Forecasts future failures from recurring patterns.	Suggests targeted interventions and rule updates.

Appendix

Table 8: How agents and human review support descriptive, diagnostic, predictive, and prescriptive behavior.

Node type	Purpose
Dataset	Represents a table, file, or view as a whole.
Entity	Represents a real-world object described by rows.
Attribute	Represents a column or derived field.
Relationship	Represents row-to-row or table-to-table linkage.
Vocabulary	Represents a value space or code system.

Appendix

Table 9: Exhaustive ontology node types.

Field	Meaning
node_id	Stable unique identifier.
node_type	One of the five exhaustive types.
name	Human-readable label.
semantic_role	Open-ended semantic classification.
confidence	Belief score in correctness, from 0 to 1.
sources	Provenance of semantic evidence.

Appendix

Table 10: Field semantics for the minimal ontology node schema.

Module	What it outputs
Structural	Schema facts such as column names, physical types, counts, declared keys, partitions.
Distributional	Global numeric and categorical summaries from full scan or distributed aggregation.
Temporal	Frequency, gaps, weekday or holiday patterns, rolling rates with confidence.
Relationship	Dependencies, candidate foreign keys, correlations, ordering constraints.
Quality signals	Null rates, duplicates, declared constraint counts, drift indicators.

Appendix

Table 11: Profiler modules and their evidence outputs.

Figure 2: Full envisioned system overview: all components sharing a central Artifacts Repository across the full DQ lifecycle.

Category	Applied to	Share
Range / statistical	numeric bounds, thresholds, sign, outlier bounds	24.2%
Key / uniqueness	primary-key or unique-column assertions	23.2%
Categorical domain / vocabulary	enumerated valid-value list	22.1%
Syntactic format	regex, length, type, encoding, calendar-validity of a date string	10.5%
Completeness / NOT NULL	presence assertions	9.5%
Functional dependency	$X\to Y$ within a table	6.3%

Appendix

Table 12: Taxonomy of AI-mined rule types, tagged on a random 25% sample (

n=85

) of all AI-mined rules.

Figure 3: Good-rule rate $\hat{p}_{\mathrm{good}}$ per mode and dataset (dots = individual runs, bar = mean, error bars = 95% $CI$ -interval on mean).

Figure 4: IDS distributions per mode across weight configurations (

n_{1}=n_{2}=8

runs per mode, pre-review rules).

Dataset	Mode	$N_{\mathrm{rules}}$	$N_{\mathrm{ratings}}$	Accept	Modify	Reject
Berka	Full agentic	216	432	363 (84.0%)	42 (9.7%)	27 (6.2%)
Berka	Guided	101	202	168 (83.2%)	18 (8.9%)	16 (7.9%)
H&M	Full agentic	124	248	222 (89.5%)	13 (5.2%)	13 (5.2%)
H&M	Guided	100	200	164 (82.0%)	12 (6.0%)	24 (12.0%)
Overall	Full agentic	340	680	585 (86.0%)	55 (8.1%)	40 (5.9%)
Overall	Guided	201	402	332 (82.6%)	30 (7.5%)	40 (10.0%)

Appendix

Table 13: HITL review outcomes per mode and dataset (pooled across

8

runs).

N_{\mathrm{rules}}

: rules proposed;

N_{\mathrm{ratings}}=2N_{\mathrm{rules}}

, since the blinded block design has every rule rated by exactly two reviewers who did not author it (§ C ). Accept/Modify/Reject are counts over ratings, with row percentages. Good-rule rates are reported in Table 2 as the mean across good rule rates of reviewer runs. The Full Agentic mode proposes more rules than Guided because the orchestrator enforces a per-column coverage criterion, whereas a human expert might target a narrower set of constraints. The matched-scope control in § 5.4 re-runs the RQ1 analysis with this volume asymmetry removed.

Data Quality Rule Generation with LLMs
Sep 5, 2026Anna-Christina Glock, Thomas Hütter, Johannes Fürnkranz +3
Software Competence Center Hagenberg, Austria · Johannes Kepler University Linz, Austria · Austrian Post Vienna, Austria +1
Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement
Oct 21, 2026Kardelen Subasi, Jarosław A. ChudziakLLM Auditing LLM Hallucination Mitigation
Warsaw University of Technology, Warsaw, Poland
An Agentic Retrieval Framework for Autonomous Context-Aware Data Quality Assessment
May 15, 2026Hadi Fadlallah, Ibrahim Dhaini, Fatima Mubarak +1
University of Sciences and Arts in Lebanon, Beirut, Lebanon · Saint-Joseph University of Beirut2026, Beirut, Lebanon

Sep 5, 2026J/K move · Enter open · S save

Data Quality Rule Generation with LLMs

Anna-Christina Glock, Thomas Hütter, Johannes Fürnkranz, Wolfram Wöß+2

Software Competence Center Hagenberg, Austria · Johannes Kepler University Linz, Austria · Austrian Post Vienna, Austria+1

Open paper AskPDF arXiv

The validation of data, such as customer and employee data, is an important task in many organizations. Errors in data can have severe consequences. For example, a wrong drug unit in a patient record can lead to life-threatening medication errors, and a missing street number in an address to failed deliveries. Companies often employ rule-based enterprise data quality (DQ) tools, which allow domain experts to specify rules to validate the data over time. While rule-based DQ tools are computationally efficient and provide explainable reports, maintaining a comprehensive rule set manually is challenging, as domain experts often overlook essential rules, especially in complex domains and large data volumes. Hence, closing these gaps remains an open problem in practice. In this paper, we address the challenge of automated DQ rule generation. For this, we formalize a generalizable generate-filter framework and introduce LeDQeR, an LLM-based DQ rule generation approach. First, a large language model (LLM) generates candidates rules from an observed dirty data tuple for a given rule-based DQ tool syntax. Second, we apply four filter techniques that ensure the (i) executability, (ii) correctness, and (iii) generalizability, and avoid (iv) redundancy of the generated rules. An extensive experimental evaluation suggests that LeDQeR is able to produce effective and compact rule sets for various datasets and error types.

Abstract

Explore similar work

Data Quality Rule Generation with LLMs

Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement

An Agentic Retrieval Framework for Autonomous Context-Aware Data Quality Assessment