Better Policies, Worse Verifier Audits

Daniel Cieślak; Jacek Łubiński

cs.LGOct 21, 2026

Better Policies, Worse Verifier Audits

Authors: Daniel Cieślak, Jacek Łubiński

Organizations: Gdańsk University of Technology, Gdańsk, Poland

Abstract

Verifier stacks increasingly sit inside post-training and test-time compute loops. We study a specific control loop: a component verifier is evaluated on fresh policy outputs against the full success label and refreshed when that aggregate score falls. For a conjunctive target $y = F \wedge M$ , its on-policy AUC is $(1-q)A_{\mathrm{own}} + qA_{\mathrm{cross}}$ , where $q = \Pr[F=1 \mid y=0]$ . As the policy masters $F$ , $q$ can rise toward one, removing weight from $F$ -failures and rewarding information about $M$ . We call this on-policy audit collapse. We derive a sharp identified set, a complete-erasure threshold, and the exact point at which the audit can prefer a worse component verifier; its population optimum is the full-outcome predictor, not the component oracle. An exact oracle exhibits the collapse, a natural multi-label text control shows inversion can select a head with lower canonical $F$ -AUC at the observed failure mixture, and a prediction-registered ECG study shows that acting on the audit can reduce off-policy $F$ -AUC, transfer the damage to an unseen generator, and increase downstream constraint violations. Preserving own-failure support prevents the damage. Matching label-complexity bounds and a controlled-mixture audit give the corresponding repair: audit the component on its own failures, and keep those failures in the refresh distribution.

Figures & tables

policy	$q$	aggregate AUC	$A_{\rm own}$	$A_{\rm cross}$
1.5B sum	$0.970\pm 0.007$	$0.532\pm 0.004$	$0.966\pm 0.020$	$0.519\pm 0.007$
1.5B hard gate	$0.916\pm 0.014$	$0.535\pm 0.006$	$0.944\pm 0.016$	$0.494\pm 0.001$
7B sum	$0.984\pm 0.005$	$0.494\pm 0.024$	$0.987\pm 0.017$	$0.487\pm 0.025$
7B hard gate	$0.953\pm 0.011$	$0.583\pm 0.007$	$0.962\pm 0.037$	$0.561\pm 0.004$
GSM8K base	$0.16$	$0.919$	$0.997$	$0.507$
GSM8K sum DPO	$0.96$	$0.506\pm 0.004$	$0.995$	$0.489\pm 0.006$

Table 2: Chance-level aggregate AUC can coexist with near-perfect ordering of the component’s own failures.

q

is the share of off-target candidates that already satisfy the scored conjunct. ECG rows are frozen pre-DPO test pools; Gemma’s has no informative target-positive requirement and is omitted rather than imputed.

Figure 1: Audit collapse. Left: aggregate outcome AUC changes systematically with the failure mixture

q

even when the component’s own-bin ordering remains strong. Right: the zero-parameter

A_{\rm cross}=1/2

reconstruction tracks measured AUC. The complete-erasure region of Eq. 5 is shown separately in Appendix P .

Figure 2: (a) The audit prefers entanglement. Heads trained on base data with labels

(1-\lambda)F+\lambda y

: the pure component head (

\lambda=0

), which has canonical

F

-AUC

1.000

, receives the lowest on-policy audit AUC in all four family

\times

class settings (P1). (b) Acting on the audit damages the component. Refresh minus deployed specialist in each of

10

held-out-subject splits and two head classes. Outcome refresh lowers off-policy canonical

F

-AUC in every paired split while its effect on the monitor changes sign across settings; restoring base failures leaves canonical

F

-AUC essentially unchanged.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

policy	supply@16	sum	log-prob	correct only
base	$0.872$	$0.734$	$0.736$	$0.576$
sum DPO	$0.933\pm 0.005$	$0.673\pm 0.014$	$0.669\pm 0.014$	$0.670\pm 0.015$
gated DPO	$0.927\pm 0.006$	$0.685\pm 0.011$	$0.686\pm 0.014$	$0.685\pm 0.015$

Appendix

Table 4: GSM8K,

500

test prompts, three training seeds per trained arm. Upper: policy rates, median within-pool score SD at

K=64

, and correctness-head AUC against the conjunctive target. Lower: supply and answer at

K=16

(first

16

64

samples) under sum, soft log-probability and correctness-only selectors.

Figure 3: Training-side supply and selection geometry. Left: supply against

K

(seed means; bands one sd for packaged arms). Right: per-draw success against composed answer under the gate, one marker per seed.

Figure 4: One design from each cVAE arm’s pool, decoded by the function the solver grades (longitudinal section above, cross-section at mid-length below). The prior, the unconstrained decoder and latent search produce engines with visibly different lattice families; the decoder trained on the critic’s terms produces a block — every radius control pinned at maximum after clipping from far outside the box — that passes the mass-and-printability criterion and that the critic scored feasible at $0.9999$ before clipping.

literature	closest object	distinction in this work
ROC / case mix ( Sadatsafavi et al., 2022 )	how population composition changes ROC behavior	inverse identification from semantic failure bins; complete-erasure threshold and component-specific audit target
finite-mixture identification ( Henry et al., 2014 )	identification as mixture weights vary while components stay fixed	our two-mixture rank condition is a special linear case; novelty is the verifier-audit interpretation and its operational consequences
rubric RL / policy-aware criteria ( Gunjal et al., 2026 ; Tyagi et al., 2026 )	criterion-level judgments are aggregated for on-policy RL; criterion saturation changes with the policy	establishes the modern component/aggregate setting; we analyze outcome-audit identification and maintenance rather than reward weighting
RM evaluation ( Wen et al., 2025 ; Kim et al., 2025 ; Razin et al., 2025 ; Shum et al., 2026 )	static verifier/RM metrics need not determine downstream optimized-policy quality	a specific structural reason: outcome AUC of a component verifier becomes a moving semantic mixture and can prefer the wrong verifier
verifier-induced support reshaping ( Wei et al., 2026 )	optimizing one verifier can remove policy behaviors needed for later objectives	support loss is not claimed as new; we show that the audit itself loses component identification and can trigger a refresh that forgets the missing failure mode
verifier-loop regression ( Lin, 2026 )	a verifier that is useful on one task can regress a learner after task shift	our failure occurs even without a bad verifier: an exact component oracle can look worse solely because policy success changes the failure mixture

Appendix

Table 9: Closest literatures and the distinction relevant to this paper. We separate ingredients that are classical from the new verifier-audit consequence.

Figure 5: One on-target design from each policy on the same requirement, decoded by the function the solver grades: longitudinal section (bore and wall) and mid-length cross-section through the lattice. The policies propose engines with different lattice families and wall profiles, not blocks.

$c$	$0.01$	$0.1$	$1$	$10$	$100$
raw-sum answer@16	$0.544$	$0.312$	$0.185$	$0.162$	$0.160$

Appendix

Table 11: AUC-preserving scale sensitivity on the matched 1.5B sum-DPO pools. Multiplying the feasibility logit by positive

c

preserves its marginal ROC/AUC exactly, yet changes the additive selector’s best-of-16 answer. Candidate pools and critic outputs are fixed.

family	endpoint	difference	requirement bootstrap 95% CI
DPO 1.5B	per draw	$0.1765$	$[0.1630,0.1905]$
DPO 1.5B	supply@16	$0.1707$	$[0.1453,0.1973]$
DPO 1.5B	two-stage answer@16	$0.1620$	$[0.1373,0.1877]$
DPO 7B	per draw	$0.0827$	$[0.0707,0.0951]$
DPO 7B	supply@16	$0.1597$	$[0.1377,0.1820]$
DPO 7B	two-stage answer@16	$0.1477$	$[0.1260,0.1690]$

Appendix

Table 12: Right-minus-sum differences on the same test requirements. DPO right arm: hard-gated ranking; GRPO right arm: log-probability reward. Intervals use 2000 paired requirement bootstrap resamples after averaging the three fixed training seeds within each requirement. They quantify test-requirement variation conditional on those seeds, not population uncertainty over training seeds. Seed SD is reported separately in the training tables.

ranking	feasible	on target	mass error
sum ( $w=1$ )	$0.654/0.015$	$0.0653/0.0018$	$0.495/0.604$
hard gate	$0.565/0.084$	$0.0890/0.0005$	$0.290/0.918$
tuned sum ( $w=0.005$ )	$0.349/0.223$	$0.1008/0.0000$	$0.193/1.007$
log-prob	$0.635/0.019$	$0.0823/0.0008$	$0.436/0.616$
mass-only	$0.279/0.245$	$0.0930/0.0000$	$0.172/1.007$

Appendix

Table 14: Matched train-pair content, chosen/rejected: feasible rate, on-target rate and median absolute oracle log-mass error (parsed sides). Exact triple overlap (requirement, chosen, rejected) is

0.099

between sum and gate,

0.586

between sum and log-prob,

0.210

between gate and

w=0.005

, and

0.463

between

w=0.005

and mass-only.

Figure 6: Label-free pair contrast in critic units over scalar weights. The balanced region is near

w\approx 0.05

–

0.1

at both scales, far from the validation-tuned

w=0.005

. Oracle labels are not used by these axes.

Figure 7: Complete erasure for the two-bin audit. In the shaded region

q\geq\max(A,1-A)

, the sharp identified interval for

A_{\rm own}

is all of

[0,1]

Figure 8: Matched 1.5B pair geometry: tuned

w=0.005

is mass-dominated, the unit sum feasibility-dominated; the pre-specified

w\approx 0.1

passes near the gate.

Figure 9: Post-hoc controls. Left: full-target AUC for learned versus perfect own-conjunct heads on the same pools. Right: GSM8K finite-pool subset-averaged answers; exact format gating does not repair the remaining correctness ranker. Bands show training-seed SD, not confidence intervals.

Figure 10: Forecast against measurement for all

57

target arms. Each point is one target policy or pool. The zero-parameter form tracks the diagonal across the measured range; persistence sits in a band set by the source audit regardless of the target; chance is correct only in the saturated cluster.

predictor	MAE	worst arm	signed bias	$r$ (pred, measured)
chance $\tfrac{1}{2}$	$0.0745$ [ $0.0742,0.0821$ ]	$0.4366$	$-0.0622$	—
persistence	$0.2566$ [ $0.2432,0.2707$ ]	$0.4204$	$+0.2231$	$-0.093$
$(1-q)A_{\rm own}+q/2$	$0.0280$ [ $0.0275,0.0357$ ]	$0.1130$	$-0.0012$	$0.967$
$(1-q)A_{\rm own}+qA_{\rm cross}$	$0.0325$ [ $0.0322,0.0496$ ]	$0.1162$	$+0.0010$	$0.956$

Appendix

Table 16: Arm-level forecast error over all

57

target arms: absolute error of the arm’s mean predicted AUC against its mean measured AUC. The bootstrap resamples requirements jointly across arms and holds the training seeds fixed; it is requirement uncertainty, not population variance over training runs.

source $\rightarrow$ target	arms	chance	persistence	zero-param	transported
1.5B base $\rightarrow$ 1.5B trained	$26$	$0.0305$	$0.2767$	$0.0279$	$0.0374$
7B base $\rightarrow$ 7B trained	$9$	$0.0302$	$0.1868$	$0.0278$	$0.0291$
1.5B base $\rightarrow$ 7B trained	$9$	$0.0252$	$0.2668$	$0.0262$	$0.0223$
1.5B base $\rightarrow$ cVAE pools	$7$	$0.4145$	$0.1365$	$0.0425$	$0.0416$
GSM8K base $\rightarrow$ GSM8K trained	$6$	$0.0088$	$0.3991$	$0.0144$	$0.0213$

Appendix

Table 17: Arm-level forecast MAE by protocol. Chance is accurate exactly where the target has saturated its own conjunct, and fails on the cVAE pools, which have not.

run	$q$ : step 0 $\to$ 250 $\to$ 1500	chance	persistence	zero-param	transported
log-probability	$0.34\to 0.95\to 0.97$	$0.095$	$0.116$	$0.074$	$0.119$
tuned sum $w{=}0.005$	$0.34\to 0.44\to 0.59$	$0.214$	$0.052$	$0.014$	$0.026$
three-conjunct gate	$0.34\to 0.88\to 0.93$	$0.105$	$0.120$	$0.061$	$0.083$
mass-only (null)	$0.34\to 0.30\to 0.33$	$0.282$	$0.030$	$0.020$	$0.015$

Appendix

Table 19: Within-run forecast from the previous checkpoint: arm-level mean absolute error of each predictor over the six transitions of each run (

250

requirements,

K{=}16

, seed 0). The mass-only run is the registered null case whose

q

does not move.

Figure 11: Measured on-policy AUC of the feasibility head (dots),

q

(squares) and the zero-parameter forecast from the previous checkpoint’s pooled own term (crosses) along four DPO runs;

250

requirements,

K{=}16

, seed 0.

generator	feasible	on target	supply@ $64$
prior, random latent	$0.081\pm 0.001$	$0.0074\pm 0.0002$	$0.321\pm 0.013$
decoder, $\lambda=0$	$0.093\pm 0.005$	$0.0086\pm 0.0005$	$0.336\pm 0.003$
decoder, $\lambda=1$	$0.101\pm 0.120$	$0.0090\pm 0.0111$	$0.091\pm 0.056$
decoder, $\lambda=1$ , capped	$0.036\pm 0.038$	$0.0038\pm 0.0038$	$0.072\pm 0.060$
latent search on $\lambda=0$	$0.481\pm 0.047$	$0.3417\pm 0.0294$	$0.937\pm 0.017$

Appendix

Table 20: Generator families on the engine task. Per-draw on-target is flat across the first three families while supply collapses, which is the dissociation of § AE in the generator-training direction rather than the selection direction. Latent search is not compute-matched with the decoders.

pools	$\overline{\|\operatorname{Par}\|}$	singleton	mixed labels	Answer@16 range (monotone $g$ )	range (linear $c$ )
Llama base	2.28	.28	.21	[.33, .54]	[.34, .54]
Gemma base	1.75	.41	.00	[.00, .00]	[.00, .00]
Phi base	2.18	.28	.14	[.46, .60]	[.46, .59]
Phi gate (3 seeds)	2.43–2.53	.19–.24	.06–.09	[.65–.68, .73–.74]	[.65–.68, .73–.74]
Phi inference (3 seeds)	2.27–2.71	.18–.28	.06–.10	[.65–.67, .73–.75]	[.65–.67, .73–.74]
Phi pair (3 seeds)	1.01–1.10	.91–.99	.00–.01	[.51–.54, .51–.55]	[.51–.54, .51–.55]

Appendix

Table 23: Selection ambiguity on ECG test pools (post-hoc). Ranges are per-pool outer bounds: a single transform must serve all pools, so the attainable aggregate range can be narrower.

$m$ labelled negatives	25	50	100	200	400	800
Llama: mean estimate	.896	.896	.895	.894	.895	.896
Llama: CI width / coverage	.19 / .89	.14 / .91	.12 / .96	.10 / .98	.09 / 1.00	.08 / 1.00
Phi: mean estimate	.852	.846	.841	.841	.843	.842
Phi: CI width / coverage	.25 / .91	.21 / .93	.18 / .98	.16 / .99	.15 / 1.00	.14 / 1.00

Appendix

Table 24: Label budget for a direct own-bin audit of the ECG utility head (post-hoc). Target: pair-weighted within-pool

A_{\rm own}

with all labels (

0.895

Llama,

0.842

Phi).

family	rule	seed 0	seed 1	seed 2
Llama	inference	.512 (1.00)	.518 (1.00)	.470 (1.00)
Llama	gate	.512 (.47)	.520 (1.00)	.488 (1.00)
Llama	pair	.000 (.00)	.000 (.00)	.512 (1.00)
Llama	unit sum	.000 (.00)	.002 (.00)	.512 (1.00)
Gemma	inference	.008 (1.00)	.000 (.00)	.006 (.96)
Gemma	gate	.006 (.97)	.000 (.00)	.006 (1.00)

Appendix

Table 25: All ECG DPO runs: Answer@16 (parse rate) per training seed under the common evaluator. Llama pair-weight and unit-sum were trained on identical pair sets.

pools	$K$	distinct $n$	$\|\operatorname{Par}\|$	$H_{n}$	permuted	$\Pr[\|\operatorname{Par}\|{=}1]$	$1/n$	Answer@ $K$ range
Llama base	2	1.79	1.25	1.40	1.39	0.75	0.60	0.058
	4	3.09	1.57	1.83	1.81	0.52	0.35	0.107
	8	5.10	1.88	2.27	2.25	0.39	0.21	0.153
	16	8.32	2.28	2.73	2.68	0.28	0.13	0.212
Phi base	2	1.73	1.28	1.36	1.36	0.72	0.64	0.049
	4	2.79	1.60	1.74	1.73	0.49	0.40	0.084

Appendix

Table 26: Selection ambiguity on ECG test pools as a function of

K

(post hoc).

pool	evaluable	own support	erased	macro $A$	macro $q$
llama base	292	0.87	0.37	0.751	0.611
llama hard gate s1	85	0.00	1.00	0.671	1.000
llama hard gate s2	110	0.00	1.00	0.604	1.000
llama inference weight s1	43	0.00	1.00	0.605	1.000
llama inference weight s2	123	0.00	1.00	0.628	1.000
gemma pair weight s1	19	0.00	1.00	0.211	1.000

Appendix

Table 27: Deployability-head audit on base and post-DPO ECG test pools (post hoc).

		$m=100$			$m=400$
family	$q$	RMSE	scaled	no own failure	RMSE	scaled
Llama	0.00	0.021	0.20	0.000	0.010	0.21
	0.50	0.029	0.20	0.000	0.014	0.20
	0.80	0.046	0.21	0.000	0.023	0.21
	0.90	0.067	0.21	0.000	0.033	0.21
	0.95	0.102	0.23	0.006	0.047	0.21

Appendix

Table 28: Passive audit cost (post hoc). The scaled error

\sqrt{m(1-q)}\cdot

RMSE is constant, as Proposition 3 predicts, until own failures stop appearing.

family	contrast	estimate	95% interval
Llama	gate-inference	$+0.007$	$[-0.026,+0.045]$
Llama	pair-gate	$-0.336$	$[-0.557,-0.005]$
Llama	pair-inference	$-0.329$	$[-0.551,+0.000]$
Gemma	gate-inference	$-0.001$	$[-0.008,+0.006]$
Gemma	pair-gate	$+0.119$	$[-0.002,+0.266]$
Gemma	pair-inference	$+0.118$	$[-0.001,+0.266]$

Appendix

Table 29: Two-level (seed and subject) bootstrap intervals (post hoc).

family	rule	role	overlap	per-draw	$-$ gate	95% (seed+subject)	dist
Llama	gate		1.000	0.512	—	—	0.00
	$w=0.001$		0.938	0.509	$-0.003$	$[-0.009,+0.001]$	0.08
	$w=0.003$	$w_{\rm sel}$	0.923	0.510	$-0.002$	$[-0.007,+0.003]$	0.08
	$w=0.01$	gate-referenced	0.873	0.516	$+0.004$	$[-0.005,+0.015]$	0.13
	$w=0.03$		0.762	0.523	$+0.011$	$[-0.010,+0.036]$	0.36
	$w=0.1$		0.559	0.511	$-0.001$	$[-0.021,+0.025]$	0.92

Appendix

Table 30: Convex DPO reranker on frozen ECG candidate pools, ten seeds per rule (prediction-registered, CPU-only). “dist” is the full-batch GD parameter distance to the gate after

200

steps.

pool	evaluable	own support	I0: $(A,q)$	I1: $+$ scores	I2: $+$ scores, $\hat{q}$
Llama base	292	0.87	0.528 / 0.37	0.486 / 0.34	0.496 / 0.36
Phi base	283	0.64	0.548 / 0.48	0.538 / 0.48	0.538 / 0.48
Phi hard gate s0	168	0.36	0.662 / 0.64	0.662 / 0.64	0.668 / 0.65
Phi hard gate s1	197	0.28	0.720 / 0.72	0.720 / 0.72	0.721 / 0.72
Phi hard gate s2	184	0.33	0.682 / 0.67	0.682 / 0.67	0.684 / 0.67
Phi inference weight s0	158	0.36	0.648 / 0.64	0.648 / 0.64	0.649 / 0.64

Appendix

Table 31: Mean interval width / share of evaluable pools with an uninformative interval (width

\geq 0.999

) under three information structures. Post-DPO Llama pools have no own-bin support; pair-weight arms have almost no evaluable pools (Phi:

0

2

29

family	$w_{\rm sel}$	test at $w_{\rm sel}$	best common	worst common	regret	hard gate
Llama	0.003	0.480	0.500	0.392	0.020	0.486
Phi	0.03	0.542	0.546	0.512	0.004	0.548

Appendix

Table 32: Validation-tuned common weight versus the test range over audit-equivalent common weights (Answer@16, ECG base test pools).

rule	overlap	$b(r)$	endpoint	gate	gap	informative
engine unit sum	0.099	0.174	0.596	0.758	$-0.162$	yes
engine validation-tuned $w{=}$ .005	0.210	0.262	0.571	0.758	$-0.187$	yes
engine contrast-matched $w{=}$ .1	0.150	0.828	0.770	0.758	$+0.012$	yes
engine log-probability	0.140	0.287	0.675	0.758	$-0.083$	yes
engine mass-only	0.262	0.071	0.519	0.758	$-0.239$	yes
Llama inference $w{=}$ 0.003	0.923	0.969	0.500	0.507	$-0.007$	no

Appendix

Table 33: All eleven trained scalar rules (Answer@16 under the common evaluator; three seeds). Engine values from Table 39 and Appendix Q ; ECG from Table 40 and the frozen pair sets.

work	component-level structure	policy-dependent ingredient	interface relevance
Rubrics as Rewards ( Gunjal et al., 2026 )	prompt-specific rubric criteria judged separately	on-policy rubric RL	criterion signals are aggregated into a scalar reward
Reward hacking in rubric RL ( Mahmoud et al., 2026 )	criterion-level training-verifier judgments	exploitation grows during RL	aggregate proxy reward can diverge from broader reference evaluation
POW3R ( Tyagi et al., 2026 )	independently judged rubric criteria	criteria become saturated, dead or mixed under the current policy	shows criterion informativeness changes with policy state
UGDA / iterative RLHF ( Sun et al., 2025 ; Bai et al., 2022 )	learned reward/preference model	fresh policy interactions enter later updates	supplies the on-policy refinement step used in our controller abstraction

Appendix

Table 34: Contemporary systems instantiate the ingredients of the analyzed interface. We do not claim that any cited system implements the exact audit-trigger-refresh controller end to end.

setting	quantity	$\lambda=0$	$0.25$	$0.5$	$0.75$	$1$
Llama GBM	post-DPO audit	0.625	0.686	0.674	0.721	0.728
	base audit	0.855	0.897	0.902	0.905	0.917
	$F$ -AUC (base)	1.000	0.999	0.999	0.997	0.964
Llama MLP	post-DPO audit	0.561	0.654	0.644	0.673	0.692
	base audit	0.747	0.829	0.858	0.874	0.891
	$F$ -AUC (base)	1.000	0.999	0.998	0.997	0.936

Appendix

Table 35: P1: on-policy audit AUC of label-mixture heads

(1-\lambda)F+\lambda y

(mean of three seeds). The pure component head (

\lambda=0

) is lowest in every row labelled “audit”.

head	setting	monitor AUC	$F$ -AUC base	$F$ -AUC unseen	gate $F$ -violation	Answer@16
$R_{y}$	Llama GBM	$-0.050$ (3)	$-0.433$ (0)	$-0.507$ (0)	$+0.055$ (10)	$-0.048$ (0)
$R_{y}$	Llama MLP	$+0.105$ (9)	$-0.117$ (0)	$-0.291$ (0)	$+0.011$ (10)	$-0.020$ (2)
$R_{y}$	Phi GBM	$-0.012$ (4)	$-0.481$ (0)	$-0.481$ (0)	$+0.040$ (10)	$-0.056$ (0)
$R_{y}$	Phi MLP	$+0.042$ (8)	$-0.091$ (0)	$-0.364$ (0)	$+0.018$ (10)	$-0.030$ (0)
$R_{F}$	Llama GBM	$-0.120$ (1)	$-0.235$ (0)	$-0.224$ (0)	$+0.027$ (10)	$-0.017$ (0)
$R_{F}$	Llama MLP	$+0.008$ (8)	$-0.000$ (0)	$-0.002$ (0)	$+0.001$ (2)	$-0.004$ (0)

Appendix

Table 36: Refresh minus deployed specialist, paired by subject split (mean; in parentheses the number of the

10

splits with a positive difference).

$\tau$	triggered	mean $F$ -AUC	gate $F$ -violation	Answer@16
.05	4/4	.719	.179	.491
.10	3/4	.742	.175	.499
.15	3/4	.742	.175	.499
.20	2/4	.772	.172	.504
.25	1/4	.880	.158	.516
no refresh / reference- $F$ trigger	0/4	1.000	.148	.530

Appendix

Table 37: Counterfactual controller on frozen ECG heads. “Triggered” is the number of the four primary family

\times

class settings in which aggregate audit AUC falls by at least

\tau

from base to post-DPO. Triggered settings deploy

R_{y}

; others keep

S

. No-refresh/component-specific control has

F

-AUC

1.000

, violation

.148

, Answer@16

.530

Figure 12: Frozen closed-loop threshold sweep. Left: aggregate-audit triggers deploy the already-trained outcome-refresh head and reduce canonical

F

-AUC; the dotted secondary axis is the number of the four primary settings that trigger. Right: the same controller raises constraint violations and lowers Answer@16. Dashed horizontal lines are the no-refresh/reference-

F

controller.

$\lambda$	canonical $F$ -AUC	$A_{\rm own}$	$A_{\rm cross}$	$q^{*}$ / observed audit
.10	.668	.656	.299	.120 / .470
.50	.619	.623	.564	.113 / .592
1.00	.570	.589	.765	.126 / .681
2.00	.520	.547	.879	.155 / .721
4.00	.483	.512	.919	.183 / .725

Appendix

Table 38: Leakage-strength sweep for the natural annoyance/gratitude control. The specialist is the

\lambda=0

reference: canonical

F

-AUC

.679

and aggregate audit

.442

at observed

q=.522

. Increasing cross-component leakage makes the component classifier worse while making the aggregate audit better.

Figure 13: Natural semantic inversion and a severity sweep. Left: for the post-hoc annoyance/gratitude positive control, the observed

q=.522

lies well past the analytic crossing. Right: increasing cross-component leakage monotonically degrades canonical

F

-AUC while the outcome audit improves over most of the sweep.

Figure 14: Failure-targeted replay is sample-efficient. Starting from only eight own failures, 80 indiscriminate examples add few relevant failures and barely move canonical

F

-AUC; 80 explicit own failures recover substantially more component discrimination. Error bars show SD over 100 deterministic replay draws; the full-support bar uses all 303 development failures.

Figure 15: Controlled natural-language stress test. Left: the same fixed component verifier looks progressively worse as its own-failure bin vanishes, while a verifier contaminated with the other component overtakes it near the predicted

q^{*}

. Right: with correct component labels fixed, held-out

A_{\rm own}

degrades smoothly as own-failure support is removed; the square marks the 8-failure refresh plus 80 re-injected baseline failures. Bands/error bars show one standard deviation across 24 deterministic splits.

Figure 16: (a) With every component AUC fixed, the range of Answer@

K

reachable by audit-equivalent composites on the ECG base test pools grows with

K

(Proposition 6 ;

20

subsamples per pool). (b) Endpoint minus hard gate against exact pair overlap for all eleven trained scalar rules (engine 1.5B and ECG; three seeds each; common evaluator). Filled: families whose gate clears the requirement-blind floor (engine, Phi); open: uninformative (Llama near the blind pipeline, Gemma without supply); this post-hoc split uses only each family’s gate arm, never a rule’s gap (Appendix AB ). All three rules with overlap

\geq 0.79

match the gate within

0.007

(two in uninformative families); five of six informative low-overlap rules miss by

\geq 0.08

, the exception being the contrast-matched

w=0.1

Algorithmic Auditing

Algorithmic auditing evaluates whether automated systems are fair, reliable, robust, compliant, and accountable in their intended settings.

See all 139 papers

Oct 1, 2026

Verify Claims, Not Scores: Evidence-Based Verification of Modular Agents

When developers change one component of an agent, such as its controller, a learned model or its verifier, they usually judge the change by an aggregate task score. That score cannot tell whether improvement was attainable, which component lost value, or what the agent's own checks certify. We introduce a claim-specific verification audit for modular agents that plan, act, check and refine. Instead of scoring the agent, the audit scores the evidence: each conclusion is recorded with the evidence behind it, one of four verdicts (supported, unsupported, unresolved or not evaluated) and the boundary within which it holds. Three tools supply that evidence. Oracle policies measure attainable improvement under an explicitly stated action set, so that a low value can be traced to the evaluation rather than to the environment. Replacing one component at a time with a perfect counterpart locates lost value, with null results read as unresolved whenever a downstream component could mask them. A separate test asks whether the verifier's score identifies the quantity it is read as bounding. Applied to a constrained portfolio-allocation agent in a synthetic market with known hidden regimes, the audit shows that the value of perfect regime information depends on the action set used to measure it, that the scenario generator discards most of the regime signal while better local fidelity does not improve decisions, and that the runtime verifier can be bypassed with no visible change in outcomes. The contribution is the protocol and the evidential distinctions it enforces; the empirical findings are specific to the agent and environment studied.

Ali Atiah Alzahrani

AI Agent Auditing Agent Evaluation

Ask PDF

Aug 17, 2026

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency

Automated checking pipelines increasingly place one language model as the checker and another (or the same one) as the fixer. We ask whether that wiring changes what the checker reports. Measuring false alarms on human-verified-correct ProcessBench traces with the present task held byte-identical, we find that a completed audit -> repair episode already in the model's context lowers false alarms in 15 of 15 model x wording combinations, by 2.8 to 11.5 percentage points against a length-matched non-audit control, a 9 to 25% reduction relative to that control. The direction contradicts what the accumulated-message literature predicts: an episode whose audit reported an error lowers false alarms further still, at all five wordings on the model where that manipulation lands cleanly, though a negativity asymmetry predicts more flagging. Decomposing the episode finds repair content and audit verdict complementary: different components carry the effect on different model families. Signal-detection analysis locates the change in the threshold rather than in discrimination -- the criterion moves in 15 of 15 combinations and survives correction in 13 while d' survives in none, though the d' test is half as sensitive by construction -- and a hand audit of 50 false alarms finds 82% simply wrong, so at this operating point the shift need not be harmful. With reasoning enabled the effect keeps its relative size on both models tested, and the threshold reading holds there too.

Parsa Mazaheri, Kasra Mazaheri

University of California, Santa Cruz · Massachusetts Institute of Technology

Language Model Error Detection Bias in LLM-as-a-Judge LLM Evaluation Language Model Auditing

Ask PDF

Sep 29, 2026

VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers

Repeated verifier calls are useful only when they contribute conditional information. We introduce VStress, an auditable replay contract, and VStress-CA, a correlation-aware allocation policy that estimates the conditional marginal information of an unqueried verifier on a sealed calibration split, discounts uncertainty, normalizes by call cost, and stops or abstains when the next call is not informative. The controller freezes its decision and cost ledger before joining the clean oracle; a dependence-shift alarm disables channel preference and falls back to exact-stop. The controlled audit gives the mechanism boundary: at 35% symmetric corruption, majority-5 improves balanced accuracy from 0.6578 to 0.7739, whereas at 65% it loses 0.1226 points. In the matched fixed-budget comparison, breadth, redundancy, and adaptive allocation obtain balanced accuracies 0.6048, 0.6375, and 0.6538, with 3.4216 calls per item and an RLVR score of 0.6417 for VStress-CA. Dependence diagnostics also increase from same-model repeats to cross-family channels, with conditional marginal gains of 0.0126, 0.0462, and 0.0913. These measurements turn correlation from a post-hoc warning into an auditable allocation decision.

Miaobo Hu, Shuhao Hu, Xiaobo Guo +4

School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Algorithmic Auditing Black-Box Model Auditing

Ask PDF

Abstract

Algorithmic Auditing

Explore similar work

Verify Claims, Not Scores: Evidence-Based Verification of Modular Agents

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency

VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers