Constraint Dependency Graph Retrieval for Compliance Detection on Long SKILL Documents
Authors: Shuaitao Zhao, Feng Ni, Lichao Ma, Yuchen Wei, Shuaike Li, Jiaye Lin, Fei Han, Yang Wei, Lu Pan
Organizations: Meituan, Beijing, China · Peking University, Beijing, China · Zhejiang University, Hangzhou, China · University of Science and Technology of China, Hefei, China
Large Language Model (LLM) agents carry out business operations under enterprise documents we refer to as SKILL documents, which take the form of standard operating procedures (SOPs) that state the conditions to verify and the actions to take before an operation proceeds. Long, densely cross-referenced SKILL documents exceed the context budget of small deployed auditors. Similarity-based retrieval then brings in irrelevant rules while omitting mandatory prerequisites, and the two-stage audit policy provides no trainable signal for the retrieval stage. We present SkillCDG, a compliance detection framework that reformulates each SKILL document as a constraint dependency graph, shifting rule retrieval from similarity-based matching to closure-based selection under the deployment's context budget. Each atomic rule is anchored to its source text span as a graph node, and mandatory prerequisite relations form the sole edge type. Policy-generated retrieval keywords activate seed rules, whose dependency closure yields a compact and conditionally complete rule context within that budget. The same graph also supports a joint optimization scheme. The retrieval stage is trained with group-relative policy optimization using a verifiable reward that combines output-format validity with downstream verdict correctness, with closure non-emptiness serving as a validity gate, while selective on-policy distillation concentrates teacher supervision on high-entropy reasoning tokens, with verdict and termination tokens always included, to improve verdict accuracy. Experiments on three enterprise datasets and two public benchmarks show that SkillCDG improves accuracy and F1 over the full-document and retrieval-based baselines.
Figures & tables
Figure 1. Identical responses may result from compliant and non-compliant procedures. Reliable auditing requires the applicable rule and its complete set of policy prerequisites. Motivation of SkillCDG
Figure 2. The overview of SkillCDG framework. Overview of the SkillCDG framework, showing the constraint dependency graph construction, closure-based compliance judgment, and the joint RL training procedure.
Method
Fulfillment
AfterSales
Merchant Support
CompliBench
LongBench
Overall
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
Full-SKILL
Full-SKILL
42.80
41.50
48.00
44.70
46.10
46.10
21.60
23.10
6.28
4.94
32.96
32.07
Retrieval-based
BM25-RAG
36.20
32.80
41.50
37.60
39.80
40.20
18.60
20.40
19.30
21.10
31.08
30.42
Dense-RAG
39.80
36.10
44.20
40.30
42.60
43.10
20.90
22.70
21.60
23.40
33.82
33.12
Table 1. Main results across five test datasets. Columns report accuracy (ACC) and F1. Rows Qwen3-4B through DeepSeek-V4-Pro are backbone baselines; SkillCDG (4B) is the full method. The best and second-best results are highlighted through boldface and underline , respectively, and the best results are also marked with a shaded cell.
Dataset
Total
Compliant
Violating
Avg. SKILL len
Fulfillment
1,442
955
487
10,669
AfterSales
1,536
898
638
7,037
Merchant Support
870
630
240
5,031
CompliBench
2,710
827
1,883
739
LongBench
2152
1360
792
24,674
Table 2. Dataset statistics. Avg. SKILL len is mean tokens per SKILL document.
Fulfillment
LongBench
Variant
ACC
F1
ACC
F1
SkillCDG (full)
71.08
72.13
39.83
32.18
w/o dependency closure (seeds only)
21.64
21.53
10.13
20.63
w/o retrieval RL (frozen retrieval)
53.19
43.03
25.50
23.73
w/o entropy gate (uniform OPD)
69.41
70.32
38.24
33.04
w/o teacher (joint RL only)
58.34
54.31
32.45
26.71
Table 3. Ablation of SkillCDG components.
Figure 3. Accuracy as the SKILL context length grows from 4K to 36K tokens on the length-controlled evaluation set.
Fulfillment
LongBench
Method
Input (k)
Output (k)
Lat.
Input (k)
Output (k)
Lat.
Full-SKILL
35.0
/
1.0
71.2
0.12
1.0
BM25-RAG
21.8
0.38
1.8
37.3
0.34
1.7
Dense-RAG
19.3
0.25
1.9
35.2
0.34
1.8
GraphRAG
15.3
0.28
1.5
29.3
0.31
1.5
LightRAG
14.7
0.31
1.5
20.5
0.31
1.5
Table 4. Online cost and latency per instance on Fulfillment and LongBench. Input and output tokens are mean prompt and completion tokens in thousands; Lat. is judgment latency, reported relative to Full-SKILL ( 1.00× ).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Rule
Condition
Action
Requires
C032
Merchant responsibility and refund explanation
State the refund reason and cite responsibility evidence
C014
C033
Merchant responsibility and refund explanation
State the deduction upper bound
C050
C034
Merchant responsibility and refund explanation
Provide an improvement recommendation
C007
C014
Determining the refund reason or responsible party
Read both from system signals
–
C050
Using the merchant-responsibility refund script
State merchant responsibility and the deduction limit
–
C007
The refund is caused by the merchant
Explain honestly and provide corrective guidance
–
Appendix
Table 5. Extracted rules and require edges for the Merchant Support case. The right column gives the prerequisite of each rule.
Policy context
Verdict
Reason
Full-SKILL
Compliant
The response explains the refund and provides a city-manager route without exceeding the SKILL scope.
Closure
Non-compliant
Rules C033 and C034 require the 8.66 deduction limit and improvement advice. The response gives a review and city-manager routing instead.
Appendix
Table 6. Verdicts for the same instance under the two policy representations. The closure verdict cites the rules whose requirements the response does not meet.
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.