Auditing the Hand-Written Inputs of Learned Access Control
Authors: Arsalan Jalil Sarghaleh, Mehdi Hazrati
Organizations: Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran · Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC, Canada
Learned access control is typically deployed behind two hand-built artefacts: a hard-deny set enforced by a small trusted shield, and a region map that decides which requests mined rules answer and which the learner answers. Verifying the shield or measuring the learner validates neither. We audit both on the five public ABAC Lab policies. First, we formalise hard-deny soundness and check it exhaustively over relations of up to 794,250 tuples. Clauses that faithfully transcribe an explicit restriction are unsound at rates of 0.50-0.70 across four elicitors - an author and three blind language-model elicitors - because prose scopes restrictions to a section while the grant relation is a disjunction over all rules: 97% of counterexamples are granted by other rules. Shipped unchecked, these deny sets would withhold 16-78% of everything the policies grant; certified, they withhold nothing. Vacuity, a second silent failure, also occurs. Second, hybrid routing needs a two-class, abstaining rule set that ABAC miners do not emit, and ranking a mixed-effect candidate pool under a cap is one-class in all 24 cells for the two commonest ranking keys. Third, for additive losses the hybrid-versus-learner effect factorises into covered mass times a rule-learner risk contrast that must be reported on both false-non-grant and false-permit loss: on two real policies, most selection rules give opposite verdicts on the two losses, and the one favourable on both holds for one policy, one learner configuration and a band of noise levels. A released artefact regenerates and checks every reported number.
Figures & tables
Figure 1: The composed architecture, with the two hand-built artefacts this paper audits marked. An untrusted learner proposes an action; a deterministic shield admits no grant on the named hard-deny set H and is the identity outside it. The shield’s behaviour on H is a refinement property and is not in doubt (Lemma 1 ). What is in doubt is which requests are in H (Section 4 ) and which requests reach the learner at all (Sections 5 – 6 ). Neither is produced or checked by any verified component of the stack.
component
status
protects
failure consequence
learner L
untrusted
—
none on H ; bounded by σ
shield σ
safety TCB
conformance on H
unbounded grants
hard-deny clauses
safety TCB
soundness of H
wrong denials (§ 4 )
attested store
safety TCB
attribute integrity
εH−>0
actuator path
safety TCB
complete mediation
shield bypass
statistical gate
guarantee only
marginal FP rate
bound void; H unaffected
Table 1: Components and trust status. The hard-deny clause set is in the safety TCB, and no amount of verification of the shield or of an authorization language detects an error in it. Section 4 audits exactly this row.
Figure 2: Hard-deny soundness as a first-class check. A clause read off the policy’s prose is admitted to H only if it covers no request the policy grants, verified exhaustively against the full grant relation rather than a sample. Because policy descriptions state local restrictions while the grant relation is a global disjunction, prose-derived restriction clauses over-claim by default (Table 2 ). An admitted clause must additionally be checked for non-vacuity (Section 4.8 ), which the same enumeration pass supplies at no extra cost.
policy
enum. tuples
expl.
other
expl. c/ex
E-Document [ 52 ]
600,000
12/14
1/2
495–6,489
Workforce [ 51 ]
794,250
4/9
0/1
20–7,665
University [ 2 ]
6,732
—
3/8
—
Project Mgmt [ 2 ]
3,040
—
2/6
—
Healthcare [ 2 ]
1,008
—
7/8
—
total
16/23
13/25
20–7,665
Table 2: Hard-deny soundness audit across all five public ABAC Lab policies, stratified by clause provenance.
provenance
n
unsound
rate [95% CI]
explicit-restriction (mechanism)
23
16
0.696 [0.471,0.868]
inferred-exclusivity
12
7
0.583 [0.277,0.848]
inferred-necessity
11
5
0.455 [0.168,0.766]
silence
2
1
0.500 [0.013,0.987]
elicitation-error strata (pooled)
25
13
0.520 [0.313,0.722]
all strata pooled (uninterpretable)
48
29
0.604 [0.453,0.742]
Table 3: Rejection rate by provenance, with exact Clopper–Pearson intervals (reported rather than a normal approximation because several strata have single-digit denominators). The strata are not pooled: only the first isolates the scope mismatch, and the pooled rate of 0.60 is reported solely to show what a study omitting this stratification would have published.
mechanism stratum
grants denied
elicitor
model
unsound
rate [95% CI]
vac.
E-Doc.
Workf.
R1
author
16/23
0.696 [0.471,0.868]
0
0.516
0.548
L1
Opus 5.5
18/36
0.500 [0.329,0.671]
4
0.159
0.546
L2
Sonnet 5.5
22/38
0.579 [0.408,0.737]
5
0.265
0.593
L3
Haiku 5.5
23/35
0.657 [0.478,0.809]
3
0.702
0.780
L1–L3 pooled
63/109
0.578 [0.480,0.672]
12
Table 4: Author and independent elicitors compared. Mechanism stratum (explicit-restriction) with exact Clopper–Pearson intervals; “vac.” counts clauses that cover no request; the last two columns give the fraction of all granted requests that the elicitor’s unchecked candidate set would deny if shipped as H .
default-deny (total)
open-world (abstain)
policy
rules
permit-F1
m
covered
deny-F1
E-Document
35
0.934
0.000
0.048
0.000
University
10
0.930
0.000
0.022
0.000
Project Mgmt
8
0.865
0.000
0.025
0.000
Workforce †
1
0.006
0.000
0.000
0.000
† L=2 over a 200,000-row sample; see caption.
Table 5: The published-miner reference arm, read three ways.
∗ the model Prop. 2 assumes. † nearly indistinguishable
from symmetric here: the reference set is almost entirely deny-
Table 6: Structured-noise robustness of Proposition 2 ’s two consequences.
selection rule
degen.
rate
minority
inf. mass
support
12/12
1.00
0.0
0.00
confidence (support tb)
12/12
1.00
0.0
0.16
confidence (low-support tb)
5/12
0.42
8.9
0.81
lift
3/12
0.25
152.5
0.31
class-balanced
0/12
0.00
90.6
0.16
Table 7: Selection-rule degeneracy over 4 policies ×3 cap sizes ( 100/300/1000 ), with an independently reimplemented miner. “Degenerate” means the retained rule set is one-class on the covered region; “minority” is the mean number of minority-class rules retained per cell and “inf. mass” the mean informative-region mass. Per-policy configurations: E-Document at conjunction cap L=3 over a uniform 200,000-row sample (min. support 5); University and Project Management at L=3 over the full relation (min. support 3); Workforce at L=2 over a 200,000-row sample (min. support 5); minimum confidence 0.60 throughout. Region and F1 figures are estimated on a 50,000-row subsample where the relation is larger.
cov.-region F1
false-non-grant (on grant subpop.)
false-permit (on non-grant subpop.)
selection rule
deny
perm.
mG
rCG
ℓCG
rCG−ℓCG [95% CI]
mN
rCN
ℓCN
rCN−ℓCN [95% CI]
verdict
support (all-deny)
0.971
0.000
0.000
1.000
0.058
+0.942[+0.932,+0.950]
0.000
0.000
0.303
−0.303[−0.312,−0.294]
metric-dep.
conf. (support tb)
0.990
0.179
0.931
0.870
0.148
+0.721[+0.640,+0.796]
0.839
0.000
0.188
−0.188[−0.197,−0.178]
metric-dep.
conf. (low-supp. tb)
0.971
0.133
0.927
0.917
0.105
+0.812[+0.759,+0.859]
0.930
0.003
0.333
−0.330[−0.348,−0.310]
metric-dep.
lift (all-permit)
0.000
0.766
0.122
0.000
0.048
−0.048[−0.060,−0.038]
0.968
1.000
0.715
+0.285[+0.264,+0.308]
metric-dep.
class-balanced
0.931
0.852
0.257
0.011
0.044
−0.033[−0.049,−0.020]
0.879
0.125
0.240
−0.115[−0.127,−0.104]
rules
Table 8: Direct measurement of the decomposition terms on E-Document, on both additive losses.
Figure 3: The routing verdict is a point in two-loss space, not a sign (E-Document, gbdt class-balanced, 20 seeds, 95% paired-bootstrap intervals; the data of Table 8 ). Routing the covered region to rules is justified only in the shaded quadrant, where both contrasts are negative. The one-class rule sets sit in opposite off-diagonal positions: an all-deny rule set cannot over-grant, so its false-permit contrast is as favourable as arithmetic allows, and an all-permit rule set cannot withhold a grant, so its false-non-grant contrast is. Either can be presented as a positive result by choosing a loss. Only class-aware selection lies in the shaded quadrant; the inset magnifies the two points near the origin and shows that both of its intervals exclude zero.
ε
rule
FNG contrast [95% CI]
FP contrast [95% CI]
0.00
support
+0.976[+0.969,+0.981]
−0.114[−0.119,−0.110]
class-bal.
−0.010[−0.016,−0.006]
+0.004[+0.003,+0.005]
0.02
support
+0.969[+0.964,+0.975]
−0.182[−0.186,−0.178]
class-bal.
−0.017[−0.022,−0.012]
−0.043[−0.058,−0.031]
0.05
support
+0.962[+0.952,+0.970]
−0.221[−0.225,−0.217]
class-bal.
−0.023[−0.033,−0.015]
−0.069[−0.078,−0.061]
Table 9: Sensitivity of the two-loss verdict to the label-noise level (E-Document, gbdt class-balanced, 20 seeds per level; each level draws its own splits, so the ε=0.15 rows differ slightly from Table 8 ). The support rows are metric-dependent at every level, in the direction Table 8 explains. The class-balanced rows favour rules on both losses for 0.02≤ε≤0.15 ; at ε=0 the losses disagree, and at ε=0.25 the false-non-grant interval includes zero.
learner
thr.
F1
FNG
FP contrast [95% CI]
verdict
default
.500
.587
−0.862
+0.125[+.109,+.142]
metric-dep.
balanced
.500
.547
−0.027
−0.108[−.126,−.091]
rules
tuned
.255
.709
−0.180
+0.028[+.010,+.046]
metric-dep.
bal.+tuned
.582
.699
−0.197
+0.034[+.017,+.050]
metric-dep.
Table 10: Is the routing verdict an artefact of an under-tuned learner? E-Document, class-aware selection, ε=0.15 , 20 seeds; the mined rules are identical across rows ( rCG=0.012 throughout), so all variation is the learner’s. “tuned” selects the decision threshold on a held-out validation split (6,000 rows, drawn in addition to the training and test splits, so the balanced row is close to but not identical with the class-balanced row of Table 8 ). F1 is the learner’s macro-F1; FNG is the false-non-grant contrast. Only one of four configurations yields a route-to-rules verdict; on false-non-grant alone every row favours the rules.
selection rule
mG
FNG contrast [95% CI]
FP contrast [95% CI]
support (all-deny)
0.000
+.969[+.962,+.975]
−.198[−.206,−.190]
conf. (support tb)
0.976
+.737[+.662,+.810]
−.149[−.160,−.138]
conf. (low-supp. tb)
0.975
+.738[+.661,+.813]
−.150[−.160,−.139]
lift
0.958
+.348[+.272,+.424]
−.146[−.156,−.134]
class-balanced
0.958
+.348[+.272,+.424]
−.146[−.156,−.134]
Table 11: The two-loss contrast on Workforce ( L=2 , 20 seeds, ε=0.15 , class-balanced gbdt ). Every selection rule’s rules withhold grants that the learner makes (positive false-non-grant contrast) and over-grant less than the learner (negative false-permit contrast), with every interval excluding zero; no selection rule is favourable on both losses. Lift and class-aware selection retain the same rule set here.
E-Document
Workforce
arm
FP
FNG
H -gr.
FP
FNG
H -gr.
learner
0.303
0.058
0.415
0.198
0.031
0.211
+ shield, certified H
0.284
0.058
0.000
0.164
0.031
0.000
+ shield, unchecked H
0.080
0.538
0.000
0.158
0.580
0.000
hybrid
0.288
0.033
0.395
0.187
0.046
0.198
+ shield, certified H
0.271
0.033
0.000
0.155
0.046
0.000
Table 12: The composed system on two real policies (class-balanced gbdt , ε=0.15 , 20 seeds; means). FP is the false-permit rate on non-granted requests, FNG the false-non-grant rate on granted ones, and H -grant the grant rate on requests in the certified set; zeros in that column for shielded arms are by construction. Bootstrap intervals are in the artefact.
arm
FP
deny
esc
C(1)
H -grant
resid.
deny-by-default
0.0000
1.0000
0.0000
1.0000
0.0000
0.000
rules
0.2060
0.2714
0.0017
0.2731
0.2231
0.000
rules+shield
0.0000
0.2198
0.0000
0.2198
0.0000
0.000
learner (raw)
0.0289
0.0836
0.0000
0.0836
0.0238
0.276
learner+shield
0.0160
0.0947
0.0000
0.0947
0.0000
0.276
hybrid
0.2310
0.0507
0.0017
0.0524
0.2405
0.268
Table 13: Key arms on the synthetic policy ( gbdt , 20 seeds; mean per-seed denominators 2908/1092/2781/125 for FP/cost/ H /residual). H -grant of 0.000 for shielded arms is by construction (Lemma 1 ); other zeros are observed. Every shielded and conformal arm in this table has esc=0.0000 , so the ranking among those arms is invariant in λ ; only the selective arms are λ -sensitive.
corruption
εH−
εH+
εsym
honest L
advers. L
abstraction
0.2680
0.0000
0.1862
0.0009
0.2680
missing attributes
0.5976
0.0000
0.4151
0.0279
0.5976
stale policy
0.1477
0.0000
0.1026
0.0292
0.1477
availability (control)
0.0000
0.4421
0.1350
0.0000
0.0000
Table 14: Directional assumption-failure analysis; every column computed on the same 30 seeds and test draws. εH− is the safety-relevant hard-deny false-negative rate, εH+ the availability-side false-positive rate, εsym symmetric disagreement; the last two columns are the realised grant rate on H under an honest learner and under an always-permit (adversarial) learner.
artefact
report
see
hard-deny set
soundness certificate, coverage of each clause, size of the enumerated relation
§ 4
elicitation
protocol, number of elicitors, provenance strata reported apart
§ 4.4 , § 4.6
mining
minconf against ε^ ; support distribution of retained rules
§ 5.2
region map
semantics (total or abstaining), class composition, full operator tuple
§ 5
routing
rC−ℓC on every additive loss, with the learner configuration
§ 6
cost
denial and abstention separately; any price λ stated
§ 7.3
Table 15: Reporting checklist for evaluations of learned access control behind a trusted layer.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
ID
strat.
verdict
cov.
c/ex
clause
HC1
excl.
unsound
68
9
action in [’addItem’] AND resource.type in HR AND NOT user.position in nurse
HC2
nec.
unsound
68
9
action in [’addItem’] AND resource.type in HR AND NOT user.ward = resource.ward
HC3
nec.
unsound
80
4
action in [’addNote’] AND resource.type in HR AND NOT user.uid = resource.patient
HC4
excl.
unsound
240
6
action in [’read’] AND resource.type in HRitem AND NOT user.uid = resource.author
HC5
nec.
unsound
236
8
action in [’read’] AND resource.type in HRitem AND NOT user.specialties > resource.topics
HC6
sil.
unsound
48
4
action in [’read’] AND resource.type in HRitem AND user.position in nurse
Reasoning systems usually treat premise use as a question of relevance: if a fact is available and useful, it may be selected for inference. Authorization imposes a different constraint: a premise may be represented and logically usable but not permitted for a particular local transition. We formalize this distinction as Governed Deduction (GD), with a transition-local admission predicate admit(p, tau, S). From an independently produced RBAC-augmented Spider benchmark, we construct 4,461 matched authorization pairs in which the same query premise and policy state support permitted and denied consuming transitions. An initial joint controller reaches 99.19% held-out accuracy, but a transition-only control reaches 100%, exposing a role-name shortcut. After a frozen, label-independent context-local role permutation removes that shortcut, premise/state-only, transition-only, and joint linear controllers all score exactly 50% on 1,856 held-out edges, while a symbolic policy oracle remains at 100%. The result is a controlled negative finding: the benchmark instantiates policy-grounded authorization beyond relevance, but the frozen linear representation does not recover the relation. Matched one-sided controls and leakage audits are therefore essential for evaluating learned policy-sensitive reasoning.
Wesley Shu, Hsi-Ching Lin
The Institute of Energetic Paradigm · National Center for High-Performance Computing, Taiwan
Large Language Models are increasingly used to turn natural-language requirements into code. In access control, that shortcut is dangerous: a generated policy can compile and read correctly while granting access that no one approved. The difficulty is not only writing policy code. It is fixing what the requirements mean before code is written, and then checking that the final policy actually satisfies that intent. We present AutoCedar, a verifier-guided system that first turns natural-language access-control requirements into a reviewed, checkable target, and then synthesizes Cedar policies against that target. AutoCedar decomposes schema and policy authoring into small intent atoms: reviewable claims about vocabulary and behavior. Once those atoms pass mechanical validation and human intent review, the model proposes a candidate policy, the verifier checks it against the approved target, and each failure is turned into a repair signal that tells the model whether to broaden, narrow, or restructure the policy without changing the target. Because the model's work is split into small problems, each grounded in reviewed intent and backed by verifier feedback, end-to-end policy authoring becomes tractable. AutoCedar converges on all 221 tasks of CedarBench, our benchmark of authorization tasks paired with executable semantic boundaries. Across three requirements-corpus case studies covering healthcare, education, and conference management, AutoCedar converts noisy prose and extracted access-control fragments into reviewed schemas, formal checks, and a globally verified Cedar policy store for each scenario.
As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central safety problem. Guardrails are often placed in front of these actions, yet few evaluations ask whether their decisions follow authorization: blocking unauthorized actions while still allowing the same actions under valid permission. We introduce \bench{}, a benchmark that contrasts authorization evidence for the same operation in its language track. It combines expert-annotated natural-language scenarios with executed and verified operations, plus externally sourced native-policy requests. \bench{} checks each decision for correctness, usability and resulting effect, and tracks each request across guardrail changes. Our experiments show that many blocked operations stem from failed reviews, not correct denials. Repairing the output format restores usable decisions but changes which authorized requests are honored, and adding a second reviewer removes some errors while rejecting valid authorization. Guardrails should therefore be evaluated on both the unauthorized actions they stop and the authorized actions they let through.