Large language model (LLM) judges are increasingly used to monitor the outputs of LLM-based AI agents, yet formal guarantees on the reliability of these judges remain largely absent, limiting confidence in their use for safety-critical oversight. We consider certifying such a judge and define its risk over safety classifications of prompt--response pairs drawn from a deployed generator. We apply a PAC-Bayes framework to this setting, refining existing compression-based methodology with the use of tighter bounds and more efficient coding. We show that standard parameter-efficient fine-tuning can yield non-vacuous and even tight certificates for safety judges, for both classification and Brier risk, without holding out data for validation. We introduce LoRA-GT, which shares a single quantisation scale across layers and achieves the tightest certificates with minimal degradation in predictive performance. Across five guardrail models and four benchmarks, we find that certificate tightness is governed more by adaptation complexity than by base model scale. Finally, we show how far quantisation can be pushed before the resulting increase in risk outweighs the bound tightening gained from shorter descriptions. Together, these results provide a practical route toward certifiable scalable oversight and offer insight into how the complexity of safety adaptations shapes generalisation.
Figures & tables
Figure 1: Compression yields tight, non-vacuous PAC-Bayes guarantees for safety oversight models. Left: LoRA-T with per-tensor versus shared global scaling (LoRA-GT), which removes the per-layer quantisation-scale metadata that dominates PEFT adaptor description lengths under aggressive compression. Right: certified bounds and test risk on BeaverTails under 0 – 1 and Brier losses, where LoRA-GT achieves the tightest certificates among the shown adaptors while retaining competitive test risk.
Figure 2: Tight non-vacuous guarantees for safety oversight models. Certified bounds and test risk for 0 – 1 and Brier losses across models, datasets, and adaptor families, showing the tightest certificate within each family after accounting for the finite selection cost ( Section C.3 ). LoRA, whose bounds are vacuous, is shown at its lowest test risk. Highly compressed adaptors achieve tight non-vacuous guarantees at a small cost in test risk.
Figure 3: Compression strongly influences certifiability in safety oversight models. (a) Bound vs. description length across models and datasets, showing that smaller, more compressed adaptations yield tighter PAC-Bayes guarantees. (b) LoRA-T variants, where tying and global-scale quantisation reduce description length and tighten bounds with only modest changes in test risk.
Figure 4: Compression and distortion under 0 – 1 loss. LoRA-GT across five models and u∈{1,2,4} . (a) Mean Catoni certificates split into pre-quantisation empirical risk, signed distortion and complexity. (b) Absolute distortion against description length at b∈{2,3,4,8} . Colour shows certificate minus test risk and marker size shows test risk. Dashed lines trace the Pareto front.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Generalised Catoni yields the tightest PAC-Bayes bounds. Mean difference in bound tightness relative to the generalised Catoni bound with 95% bootstrap confidence intervals.
Figure 6: Safety fine-tuning and evaluation prompts. Prompts applied for both training and evaluation, shown separately for each of the four safety datasets and corresponding tasks considered in this work.
Dataset
Response generator
n
Egida
Ensemble of ten open-weight LLMs
792
OpenAI Moderation
None (prompts only)
1,344
Aegis 2.0
Mistral-7B-v0.1
28,984
BeaverTails
Alpaca-7B
300,567
Appendix
Table 1: Generators underlying each benchmark. Each certificate is relative to the joint distribution induced by the corresponding generator and prompt distribution. On OpenAI Moderation only prompts are evaluated, so no generator is involved.
b
Error at s∗
Error at searched s
Reduction
Coordinates clipped
2
0.409
0.365
11%
4.0
3
0.177
0.165
7%
1.5
4
0.081
0.079
3%
1.0
8
0.00489
0.00487
0.3%
0.1
Appendix
Table 2: Reconstruction error of LoRA-GT at the no-clipping scale and the searched scale. Relative ℓ2 error of the dequantised vector, mean over the 60 fine-tuned vectors (five models, four benchmarks, u∈{1,2,4} ), with the mean number of coordinates the searched scale clips. Both columns use bfloat16-representable scales, the precision the decoder stores. The search lowers the error materially only at b≤4 .
Figure 7: Training and certification pipeline. Models are trained via SFT, quantised using Hadamard incoherence processing with either GPTQ (LoRA, LoRA-XS) or nearest-integer rounding (LoRA-TN, LoRA-TT, LoRA-GT), encoded using the shorter of raw and Brotli with a one-bit flag, and evaluated using empirical test risk and PAC-Bayes certificates.
Qwen3Guard 0.6B
Llama-Guard-3 1B
Qwen3Guard 4B
Llama-Guard-3 8B
Qwen3Guard 8B
0–1 loss
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
LoRA
.092
.087
1.000
.112
.112
1.000
.087
.087
1.000
.112
.112
1.000
.102
.097
1.000
(r=2, 2-bit)
(r=1, 4-bit)
(r=2, 4-bit)
(r=2, 4-bit)
(r=2, 4-bit)
LoRA-XS
.153
.158
.991
.158
.163
.961
.158
.158
.995
.133
.138
.994
.138
.143
.996
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
LoRA-GT
.173
.168
.367
.194
.204
.395
.179
.173
.361
.148
.143
.382
.153
.158
.360
Appendix
Table 3: Complete results on Egida. Test risk before (FT) and after (Q) quantisation, with Catoni bounds including the selection charge ( β=0.05 ). Selected configurations appear below each row. Underlining marks the lowest test risk and bold the tightest bound per model and loss.
Qwen3Guard 0.6B
Llama-Guard-3 1B
Qwen3Guard 4B
Llama-Guard-3 8B
Qwen3Guard 8B
0–1 loss
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
LoRA
.116
.116
1.000
.113
.110
1.000
.098
.098
1.000
.095
.080
1.000
.104
.107
1.000
(r=2, 4-bit)
(r=1, 2-bit)
(r=1, 2-bit)
(r=1, 2-bit)
(r=1, 2-bit)
LoRA-XS
.155
.146
.933
.238
.250
.885
.134
.134
.964
.101
.104
.939
.122
.137
.965
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
(r=2, 2-bit)
LoRA-GT
.158
.158
.258
.182
.202
.305
.149
.149
.298
.131
.134
.259
.158
.143
.265
Appendix
Table 4: Complete results on OpenAI Moderation. Test risk before (FT) and after (Q) quantisation, with Catoni bounds including the selection charge ( β=0.05 ). Selected configurations appear below each row. Underlining marks the lowest test risk and bold the tightest bound per model and loss.
Qwen3Guard 0.6B
Llama-Guard-3 1B
Qwen3Guard 4B
Llama-Guard-3 8B
Qwen3Guard 8B
0–1 loss
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
LoRA
.117
.118
1.000
.118
.118
1.000
.114
.111
1.000
.109
.106
1.000
.113
.112
1.000
(r=4, 4-bit)
(r=4, 4-bit)
(r=1, 4-bit)
(r=2, 2-bit)
(r=4, 4-bit)
LoRA-XS
.150
.155
.315
.178
.185
.382
.136
.138
.397
.166
.178
.395
.146
.149
.393
(r=2, 2-bit)
(r=4, 2-bit)
(r=2, 4-bit)
(r=2, 2-bit)
(r=2, 4-bit)
LoRA-GT
.169
.177
.186
.268
.282
.322
.142
.139
.227
.174
.176
.235
.146
.151
.178
Appendix
Table 5: Complete results on Aegis. Test risk before (FT) and after (Q) quantisation, with Catoni bounds including the selection charge ( β=0.05 ). Selected configurations appear below each row. Underlining marks the lowest test risk and bold the tightest bound per model and loss.
Qwen3Guard 0.6B
Llama-Guard-3 1B
Qwen3Guard 4B
Llama-Guard-3 8B
Qwen3Guard 8B
0–1 loss
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
FT
Q
Bound
LoRA
.146
.146
1.000
.146
.146
1.000
.141
.141
1.000
.141
.140
1.000
.142
.141
1.000
(r=4, 4-bit)
(r=4, 4-bit)
(r=4, 4-bit)
(r=2, 4-bit)
(r=4, 4-bit)
LoRA-XS
.161
.163
.213
.174
.183
.243
.160
.160
.230
.182
.180
.240
.160
.168
.225
(r=2, 2-bit)
(r=4, 2-bit)
(r=2, 4-bit)
(r=2, 2-bit)
(r=2, 2-bit)
LoRA-GT
.166
.168
.174
.213
.213
.226
.165
.165
.174
.180
.182
.200
.163
.162
.169
Appendix
Table 6: Complete results on BeaverTails. Test risk before (FT) and after (Q) quantisation, with Catoni bounds including the selection charge ( β=0.05 ). Selected configurations appear below each row. Underlining marks the lowest test risk and bold the tightest bound per model and loss.
Figure 8: Complete comparison of LoRA-T variants. Catoni PAC-Bayes certificates under 0 – 1 and Brier loss across all five models and four benchmarks. Hatching shows test risk and dashed lines mark random-risk baselines.
Figure 9: Certification of Nemotron 4B under 0 – 1 loss. Test risk (hatched) and PAC-Bayes certificate across four benchmarks. LoRA-T yields the tightest certificate on each benchmark. The dashed line marks risk 0.5 .
Figure 10: Adaptor description lengths. Compressed description lengths ∣h∣ (log scale) for all adaptor families and quantisation settings, at 2 and 4 bits. LoRA-T settings are given as (r,u) . Across guardrail models, LoRA, LoRA-XS, LoRA-TN, LoRA-TT and LoRA-GT exhibit progressively shorter codes, with tying and global-scale quantisation yielding the smallest description lengths.
Adaptor
Elias- δ
Elias- γ
Difference
LoRA ( r=4 )
1,974,975
1,974,987
+12
LoRA-XS ( r=2 )
2,588
2,593
+5
LoRA-XS ( r=4 )
4,501
4,507
+6
LoRA-TN ( r=4,u=1 )
1,691
1,695
+4
LoRA-TN ( r=4,u=2 )
2,108
2,113
+5
Appendix
Table 7: Coding scheme ablation, Llama-Guard-3 1B (BeaverTails, b=2 ). Elias- δ against Elias- γ coding of the hypothesis description length (bits).
Adaptor
Raw (B)
LZMA-9
gzip-9
Brotli-11
LoRA-TN ( r=4,u=1 )
252
2,304
1,992
1,672
LoRA-TN ( r=4,u=4 )
336
3,072
2,696
2,336
LoRA-XS ( r=2 )
336
3,072
2,712
2,568
LoRA-XS ( r=4 )
672
5,568
5,032
4,480
LoRA ( r=1 )
176,576
830,752
868,672
810,384
LoRA ( r=4 )
704,960
2,036,640
2,371,144
1,974,944
Appendix
Table 8: Compression algorithm comparison. Compressed output size (in bits) under three lossless compressors for LoRA-based adaptor configurations on Llama-Guard-3 1B (BeaverTails, 2-bit quantisation). The raw column is given in bytes. Brotli is shortest in every row.
b
Adaptors
Mean distortion δn
Test risk
Certificate
Δ∣h∣
s∗
Searched
Lower
Higher
Equal
Tighter
Looser
Equal
(bits)
2
60
0.125
0.065
42
18
0
40
19
1
+3.6
3
48
0.037
0.024
26
19
3
33
13
2
0
4
60
0.006
0.004
26
26
8
31
24
5
0
Appendix
Table 9: Effect of scale search on LoRA-GT. Mean distortion is lower with the searched scale at every bit-width. Counts report changes relative to the no-clipping scale s∗ . Δ∣h∣ is the mean change in description length.
Figure 11: Validation-based KL-inversion bound trends. (a) Bound against description length, which does not enter the validation-bound formula. (b) Bound against empirical validation risk is near-linear, since the bound is a deterministic function of validation risk and nval . This contrasts with the PAC-Bayes certificates of Figure 3 , which vary systematically with compressibility.
Figure 12: Validation bounds against PAC-Bayes certificates across held-out fractions. Validation-based KL-inversion bound minus the PAC-Bayes certificate. Positive values indicate a tighter PAC-Bayes certificate, obtained without holding out data.
Certificate
Validation
Dataset
n
(u,b)
Test
Bound
(u,b)
Test
Bound
Best Test
Egida
792
(1, 2)
.204
.394
(4, 4)
.168
.568
.153
OpenAI Moderation
1,344
(1, 3)
.164
.297
(4, 3)
.170
.366
.164
Aegis
28,984
(4, 3)
.287
.316
(4, 4)
.284
.326
.268
BeaverTails
300,567
(4, 4)
.213
.226
(4, 3)
.215
.227
.213
Appendix
Table 10: Selecting LoRA-GT configurations with and without a validation split. Llama-Guard-3 1B over u∈{1,2,4} and b∈{2,3,4} , with ⌈log29⌉=4 selection bits charged to every bound. Entries give the selected (u,b) , its test risk and its certificate. Best Test is the lowest test risk over the grid.
Large language model (LLM)-as-a-judge research uses language models to score, compare, and verify AI outputs as scalable alternatives to human evaluation.