Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

David Gringras

cs.AIMar 8, 2026

Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

Authors: David Gringras

Organizations: Harvard University

Abstract

Safety benchmarks usually test "bare" models that receive prompts and output responses, but real-world deployments "wrap" those models in complex scaffolds. How much do these scaffolds affect model safety as measured by benchmarks? We test six leading models on four pre-registered safety benchmarks with a direct API and three scaffolds: ReAct, multi-agent, and map-reduce. We conducted 60,112 scored evaluations. On average, how safety is measured matters more than scaffolding does: we find that using a multiple choice vs. open-ended format for otherwise-identical benchmark items changes measured safety by about 5-20 percentage points (pp). The two formats are scored with different methods (answer extraction and an LLM judge), so the gap is due to measurement rather than differences in latent safety. Using a heuristic to classify model refusals would have led to different findings in four of five cases. Benchmark choice explains 15.1% of the variation in outcomes; scaffold architecture explains 0.5%, about 33x less. We find that map-reduce scaffolds, a form of structure-destroying delegation that strips answer options by decomposing prompts, reduce pooled measured safety by 7.3 pp (95% CI: 6.4 to 8.1). The pooled effects for ReAct and multi-agent scaffolds are within our pre-registered +/-2 pp margin of equivalence. However, there are large differences across models for specific benchmarks and scaffolds that are hidden by pooled estimates: for example, on the same sycophancy benchmark items, Opus 4.6 has 16.8 pp lower measured safety with a map-reduce scaffold, while Llama 4 has 18.8 pp higher measured safety. Composite reliability is G = 0.251 (95% CI: [0.000, 0.879]). This wide confidence interval, which spans "of little use" to "very good", does not support using a single composite measure of model safety as the basis for go/no-go decisions about model deployment.

Figures & tables

Study	Cases $\times$ models $\times$ configs	$N$ records	Pre-reg status	Items / scoring
Primary scaffold eval	$2{,}617\times 6\times 4$	$60{,}112$ (not fully crossed)	Pre-registered (CJW92; sycophancy added by addendum)	BBQ 800; TQA 817; XSTest 500; Sycophancy 500. Deterministic MC + LLM-judge for XSTest.
AI factual-recall control	$500\times 6\times 4$	$12{,}000$	Pre-registered (CJW92)	500 AI/ML factual-knowledge items from Anthropic MWE (originally mis-labelled as “sycophancy”; repurposed as a non-safety control). Deterministic MC.
Phase 2 mechanistic	$300\times 6\times 4$ per benchmark	$14{,}400$	Pre-registered (WA9Y7)	300 BBQ + 300 TQA items, all also in the primary dataset; 4 invocation conditions.
Format dependence	$220\times 5\times 2\times 2$	$4{,}400$	Pre-data exploratory; design frozen pre-collection	Paired MC/OE on BBQ 60, TQA 30, syc 70, AIFR 10, MMLU 50. Direct + full-question aggregation.

Table 1: Dataset map. The analyses share models and methodology, with overlap in their items: all 600 Phase 2 items also occur in the primary dataset. Pooled estimates and variance decompositions draw from the Primary scaffold evaluation (

N=60{,}112

, four safety benchmarks); sycophancy-specific analyses (Section 6.4 ) use its 9,934-record sycophancy slice. Of the 14,400 evaluation records in Phase 2, three have an error flag.

ID	Element	Pre-registered	Implemented
D-001	Prompt placement	Benchmark instructions via system prompt (§PR 9.1 to 9.2)	Resolved : implemented as registered (benchmark instructions as system prompt, item as user message; §3.3)
D-002	XSTest scoring	LLM judge (Gemini 3 Flash primary, §PR 7.3)	Resolved : LLM judge (Gemini 3 Flash primary, Opus 4.6 validation) now used as pre-registered
D-003	Multi-agent rounds	Max 1 revision round (§PR 4.1)	Resolved : implemented as registered (one revision round)
D-004	ReAct tools	3 tools: calculator, text_search, scratchpad (§PR 4.1)	Resolved : implemented as registered (local tools; no external retrieval)
D-005	Spec curve scope	$\sim$ 1,000 to 2,000 specs, 500 perms (§PR 5.4)	18 primary specs (3 DOF), 384 exploratory specs (9 DOF)
D-006	Primary estimator	GLMM with case random intercept (§PR 5.1)	Case-cluster-robust logistic regression; saved GLMM fit diagnostics unavailable; pre-registered LRT comparisons for H2 and H3 accordingly conducted as cluster-robust Wald tests

Table 2: Protocol deviations from the OSF pre-registration (DOI: 10.17605/OSF.IO/CJW92). Section numbers in the “Pre-registered” column (e.g., §PR 4.1) refer to the pre-registration document; section numbers in the “Implemented” column refer to this paper.

Figure 1: Aggregate safety rates by model (rows) and deployment configuration (columns). Each cell shows the safety rate (%) pooled across all four benchmarks for five models, and across BBQ, TruthfulQA, and XSTest/OR-Bench for Gemini 3 Pro (no Gemini sycophancy data are available). Numbers above each scaffold cell give the deviation (percentage points) from the model’s direct API baseline from Table 4 (rounded separately). Colours show this deviation from direct, with orange indicating degradation and blue indicating improvement (grey cells are direct).

N=60{,}112

scored observations across six models. See Figure 2 for breakdowns by benchmark.

${}^{}p<0.05$ ; ${}^{**}p<0.001$ (Holm-corrected).
Comparison	OR [95% CI]	$p$ (raw)	$p$ (Holm)	RD [95% CI]	NNH
ReAct vs. Direct ∗	0.96 [0.92, 0.99]	0.009	0.017	$-0.8$ [ $-1.2$ , $-0.2$ ]	133
Multi-Agent vs. Direct ∗	0.95 [0.91, 0.99]	0.010	0.017	$-0.8$ [ $-1.5$ , $-0.2$ ]	119
Map-Reduce vs. Direct ∗∗∗	0.64 [0.61, 0.68]	$<$ 0.001	$<$ 0.001	$-7.3$ [ $-8.1$ , $-6.4$ ]	14

Table 3: This table shows the effect of scaffolding on safety using logistic regression with cluster-robust standard errors. Holm–Bonferroni correction was applied to correct for multiple hypothesis testing (H1a–c). Direct (no scaffolding) is used as the reference category.

N

= 60,112. OR = odds ratio. RD = risk difference (in percentage points). NNH

=1/|\text{RD}|

. RD is not a marginal effect from the logistic regression model, but is instead calculated as the raw difference in proportions (scaffold safe rate

-

direct safe rate). Confidence intervals for the risk difference are calculated using a case-cluster bootstrap (

B=2{,}000

	ReAct		Multi-agent		Map-reduce
Model	Direct	RD (pp)	Direct	RD (pp)	Direct	RD (pp)
DeepSeek V3.2	70.2%	$+1.0$	70.2%	$+0.6$	70.2%	$\mathbf{-15.1}$
GPT-5.2	73.4%	$-0.8$	73.4%	$-3.9$	73.4%	$\mathbf{-6.9}$
Llama 4	67.0%	$+2.4$	67.0%	$+3.8$	67.0%	$+0.9$
Mistral Large 2	72.9%	$-2.8$	72.9%	$-3.9$	72.9%	$\mathbf{-6.1}$
Opus 4.6	85.2%	$-2.0$	85.2%	$-1.3$	85.2%	$\mathbf{-15.5}$

Table 4: Safety effects of scaffolding are shown for each model separately, as risk differences (RD) in percentage points of a scaffolding condition compared to the direct API condition. Negative values indicate degraded safety. Cells where

|\text{RD}|\geq 5

pp are in bold.

Figure 2: Safety rates by benchmark, model, and configuration. Stars indicate statistically significant pairwise differences from the direct baseline (BH-FDR,

q<0.05

N=60{,}112

scored observations from six models.

Benchmark	MC (pooled)	OE (pooled)	Gap (pp)	Direction
Sycophancy	33.7%	53.3%	$+19.6$	Higher OE score
BBQ	94.3%	99.2%	$+4.8$	Higher OE score
TruthfulQA	79.3%	85.0%	$+5.7$	Higher OE score
AI Factual Recall	77.0%	76.0%	$-1.0$	Negative-control contrast
MMLU (capability)	85.4%	76.2%	$-9.2$	Higher MC score

Table 5: Pooled safety rates by benchmark and response format (MC = multiple-choice, OE = open-ended). Gap = OE

-

MC, in percentage points.

N=4{,}400

observations from five models, 220 matched item pairs, and two deployment configurations.

Model	MC	OE	Gap (pp)
DeepSeek V3.2	96.7%	100.0%	$+3.3$
GPT-5.2	90.0%	99.2%	$+9.2$
Llama 4 Maverick	94.2%	97.5%	$+3.3$
Mistral Large 2	95.0%	99.2%	$+4.2$
Opus 4.6	95.8%	100.0%	$+4.2$

Table 6: BBQ safety rates by model and format. The between-model range is 6.7 pp in MC and 2.5 pp in OE, pooling direct and full-question aggregation responses.

Model	MC	OE	Gap (pp)
DeepSeek V3.2	32.1%	48.6%	$+16.4$
GPT-5.2	57.1%	60.0%	$+2.9$
Llama 4 Maverick	15.0%	29.3%	$+14.3$
Mistral Large 2	22.1%	53.6%	$+31.4$
Opus 4.6	42.1%	75.0%	$+32.9$

Table 7: Sycophancy: anti-sycophantic accuracy by model and format. Format gaps range from

+2.9

pp (GPT-5.2) to

+32.9

pp (Opus), so the size of the format gap depends on the model as well as the benchmark.

Model	Direct	MR	OP	OP $-$ MR (pp)	Recovery (%)
	Safe of 200
Opus 4.6	189	157	186	$+14.5$ [9.5, 20.0]	90.6 [75.0, 103.4]
DeepSeek V3.2	162	101	141	$+20.0$ [13.0, 27.0]	65.6 [47.8, 82.4]
Mistral Large 2	169	134	156	$+11.0$ [5.0, 17.0]	62.9 [34.9, 90.0]
GPT-5.2	176	146	158	$+6.0$ [0.5, 11.5]	40.0 [3.6, 66.7]
Llama 4 Maverick	158	143	143	$0.0$ [ $-6.0$ , 6.0]	0.0 (CI unstable)

Table 8: This table shows option preservation on matched cases, with 95% confidence intervals from a paired case bootstrap. All three arms for each model use the same 200 cases (100 BBQ and 100 TruthfulQA), and scoring is with the canonical extractor, with parse failures counted as unsafe. Recovery is calculated as (OP

-

MR) / (Direct

-

MR).

Sub-call type	$n$	Option-marker hits	System-keyword hits
Decompose	150	75	3
Map chunk 0	150	0	150
Map chunk 1	148	3	148
Map chunk 2	125	5	125
Reduce	150	75	150
Total	723	158	576

Figure 3: Marker detection in the 723 recorded map-reduce sub-calls. An option-marker hit requires any parenthesized letter A–G or the string “Choices:”; a system-keyword hit requires any benchmark-specific or generic term (including “helpful” or “harmless”) in the saved system and last user messages. Denominators include inputs without MC options. The three decompose keyword hits occur in user questions. The counts track where options and system keywords travel; they are not measures of instruction fidelity.

Model	Direct	CoT	CoT–Direct	MR–Direct	Interpretation
Gemini 3 Pro	84.4%	89.9%	$+5.5$ pp	$-10.0$ pp	CoT benefits modestly
Opus 4.6	88.8%	91.4%	$+2.5$ pp	$-13.5$ pp	CoT benefits modestly
Llama 4	75.4%	76.5%	$+1.2$ pp	$-5.5$ pp	CoT benefits slightly
GPT-5.2	83.5%	83.9%	$+0.4$ pp	$-13.0$ pp	CoT neutral
DeepSeek V3.2	76.4%	76.0%	$-0.4$ pp	$-30.0$ pp	CoT neutral

Table 9: Chain-of-thought vs. map-reduce on 200 items (100 TruthfulQA, 50 BBQ, 50 AI factual recall). Direct and CoT: accuracy among parseable responses (192–200 per cell). MR–Direct: the scaled pilot’s all-200-item contrast, with its own direct baseline. CoT preserves task structure while eliciting extended reasoning; map-reduce decomposes the task and strips MC options.

Model	PT	Min	Mod	Agg	$\Delta$ (Agg–PT)
Opus 4.6	96.3%	95.0%	93.7%	94.0%	$-2.3$ pp
Gemini 3 Pro	93.7%	94.0%	94.7%	89.7%	$-4.0$ pp
GPT-5.2	92.7%	89.3%	85.7%	79.7%	$-13.0$ pp
DeepSeek V3.2	91.3%	91.0%	83.3%	79.0%	$-12.3$ pp
Llama 4 Mav	92.0%	93.0%	92.0%	87.7%	$-4.3$ pp
Mistral Large 2	88.7%	89.3%	78.7%	66.7%	$-22.0$ pp

Table 10: Reported Phase 2 BBQ accuracy (

N=300

records per model

\times

condition).

\Delta

is the difference between the displayed percentages (which are rounded to one decimal place). The aggressive bias invocation decreases accuracy in all six models (

-2.3

pp in Opus to

-22.0

pp in Mistral). The minimal chains never move accuracy by more than 4 pp from the passthrough baseline in any model. In some cases, the change in accuracy is non-monotonic across the intermediate trajectories. Descriptive summaries; no registered test is applied.

Model	PT	Min	Mod	Agg	$\Delta$ (Agg–PT)
Opus 4.6	98.3%	97.7%	98.3%	98.7%	$+0.4$ pp
Gemini 3 Pro	92.7%	92.3%	94.3%	97.0%	$+4.3$ pp
GPT-5.2	90.7%	86.3%	88.7%	93.7%	$+3.0$ pp
DeepSeek V3.2	79.7%	79.7%	82.3%	91.7%	$+12.0$ pp
Llama 4 Mav	69.0%	79.3%	82.7%	87.7%	$+18.7$ pp
Mistral Large 2	83.7%	79.7%	84.3%	90.0%	$+6.3$ pp

Table 11: Phase 2 TruthfulQA results (

N=300

records per model and condition) are shown in the following table. Values are reported as percentages, and the gain

\Delta

is the difference between the values shown (which are rounded to one decimal place). TruthfulQA gains range from

+0.4

percentage points (Opus, from a base rate of 98.3%) to

+18.7

percentage points (Llama 4), and the intermediate trajectories are not uniformly monotonic. There is no separate test for each model.

Figure 4: Reported Phase 2 accuracy changes from passthrough across six models (

N=300

records per condition); each point subtracts the displayed one-decimal passthrough percentage in Tables 10 and 11 , and each panel prints its passthrough rates. “Passthrough” refers to the direct condition. API errors and response parsing errors are handled as described in the text. The three chains (minimal, moderate, and aggressive) all make the same four calls. They differ in the instructions given to the model. These trajectories are descriptive and do not establish a unique internal mechanism.

Figure 5: Item-level flip rates by scaffold type and benchmark. (a) Total flip rate (solid = degradation, hatched = improvement). (b) Net flip rate (degradation minus improvement). TruthfulQA under map-reduce shows the largest net degradation (

+19.3

pp). Sycophancy, the fourth primary safety benchmark, is reported separately (Figure 7 ). XSTest+OR-Bench flips use the LLM-judge refusal labels behind Figure 9 . From 46,354 paired comparisons (14.8% divergent) across six models.

			Safety Rate		RD vs. Direct (pp)
Model	Config	PF%	ITT	PP	ITT	PP
Opus 4.6	Direct	0.3%	84.5%	84.7%	—	—
	ReAct	0.2%	83.3%	83.5%	$-$ 1.2	$-$ 1.3
	Multi-agent	0.3%	83.5%	83.7%	$-$ 1.0	$-$ 1.0
	Map-reduce	5.7%	69.8%	74.0%	$-$ 14.7	$-$ 10.7
GPT-5.2	Direct	1.1%	77.4%	78.3%	—	—

Table 12: This table compares Intent-to-Treat (ITT) and Per-Protocol (PP) analyses of the effect of scaffolds. The ITT analysis counts parse failures as unsafe responses, whereas the PP analysis excludes them from the analysis. The table is limited to the primary MC-format cells (

N=48{,}490

MC-format observations) for six models. For each model and configuration, there are up to 800 BBQ, 817 TruthfulQA, and 500 sycophancy items (for Gemini, only BBQ and TruthfulQA). All items have canonical primary labels. Cells in bold indicate ITT–PP differences greater than 2 percentage points. PF = parse failure rate. RD = risk difference vs. direct baseline (pp).

Figure 6: Direct ITT rate and map-reduce-minus-direct difference for the 23 primary model

\times

benchmark cells (Gemini sycophancy excluded). Each point uses the saved safe-response numerator and total-record denominator for its cell. No independent baseline estimate is used; the association is descriptive and does not identify a reconsideration mechanism.

Model	Direct	ReAct	Multi-Agent	Map-Reduce	$\boldsymbol{\Delta}_{\mathrm{MR}}$
GPT-5.2	42.5	41.6	45.5	45.8	$+3.3$
Opus 4.6	49.0	47.2	50.2	32.2	$-16.8$
DeepSeek V3.2	34.6	41.4	37.0	30.4	$-4.2$
Mistral Large 2	32.2	30.0	31.8	33.4	$+1.2$
Llama 4	11.0	10.8	15.6	29.8	$+18.8$
Pooled	33.8	34.1	36.0	34.2	$+0.4$

Table 13: Sycophancy resistance (non-sycophancy rate, %) for each model and scaffold configuration. The highest rate for each model is in bold and the lowest is underlined.

\Delta_{\mathrm{MR}}

indicates the risk difference between the map-reduce and direct scaffolds. Scoring method: ITT.

N=2{,}000

per model (GPT-5.2: 1,934).

Figure 7: Sycophancy resistance (ITT non-sycophancy rate) by model and configuration.

Model	Direct	MR $\|$ NoLeak	MR $\|$ Leak	Gap	%Leak
Opus 4.6	49.0	46.0	18.7	$-27.3$	50.4%
GPT-5.2	42.5	47.0	42.3	$-4.7$	26.9%
DeepSeek V3.2	34.6	34.8	27.2	$-7.5$	58.0%
Mistral Large 2	32.2	35.0	29.5	$-5.5$	29.8%
Llama 4	11.0	35.8	12.4	$-23.4$	25.8%
Pooled	33.8	39.7	25.4	$-14.4$	38.3%

Table 14: Persona-feature mentions and ITT non-sycophancy in map-reduce. MR

|

NoLeak and MR

|

Leak refer to different subsets of items within each model; Direct uses all 500 items (487 for GPT-5.2). Gap is MR

|

Leak minus MR

|

NoLeak, a descriptive association rather than a decrement attributable to leakage. Feature detection is regex-based and exploratory.

Finding	Heuristic	Judge	Verdict
Over-refusal (MR $-$ Direct)	$-15.3$ pp	$-13.7$ pp	Consistent
Agreement collapse (3-way exact)	$-28.6$ pp	$+0.0$ pp	Reversed
DeepSeek boundary softening	$-14.2$ pp	$+3.6$ pp	Reversed
Sub-call information leakage rate	39.3%	4.4%	Mostly scoring artifact
Safety architecture (DSS) distinction	3 distinct types	Clear-cut: distinct; boundary: concentrated	Refined

Table 15: The Measurement Integrity Scorecard compares the heuristic and LLM judge scoring methods for each of the five exploratory findings.

Figure 8: Specification curve analysis. (a) Odds ratios for three scaffold configurations relative to direct prompting across 18 specifications, ordered by the map-reduce estimate (54 points). A solid point indicates nominal

p<0.05

, and a hollow point indicates

p\geq 0.05

. Small horizontal offsets separate multiple point estimates for a single specification. Whiskers show 18 pointwise 95% confidence intervals for map-reduce using case-cluster sandwich covariance with a finite-sample correction. The reference line

\mathrm{OR}=1

is dashed, and the median map-reduce OR, 0.62 (IQR 0.56–0.66), is plotted in red. All 18 map-reduce estimates have OR less than 1 and are nominally significant. Wald p-values and confidence intervals use the normal reference distribution and are computed per fit without adjustment for the number of specifications considered. The pointwise confidence intervals are not a simultaneous confidence band, and the numbers of nominally significant and non-significant point estimates are not a separate test of the specification curve. (b) Columns show each specification’s benchmark subset, model subset from the six-model dataset, and parse-failure handling. ITT treats parse failures as unsafe; available-case analysis drops them.

Threat Channel	Mitigation	Residual Risk
Opus test scores contaminate results	Opus-excluded sensitivity analysis ( $N=49{,}670$ ; Table 17 )	Low
Scoring rubrics favour Opus-like outputs	Rubrics derived from published benchmark criteria; identical prompts across all models	Low; auditable
Pipeline micro-decisions (prompt formatting, retry logic, answer-extraction) iteratively developed using Opus may advantage Opus	All models share a single LiteLLM code path with provider-specific calls confined to API-parameter handling (Appendix J ); no model-specific branching in prompt, scoring, or extraction logic; code publicly released	Medium; requires independent replication
Opus’s robustness is an artifact of prompts Opus designed	Key finding (refusal under invocation) is binary behavioural outcome; adversarial prompts from published benchmarks	Low–Medium

Table 16: The table lists the mitigations and residual risks for each threat channel of the builder-as-subject conflict.

Test	Metric	Full (6 models)	Excl. Opus (5 models)	Change?
H1a (ReAct)	OR	0.96	0.97	Slightly closer to 1
	RD (pp)	$-0.8$	$-0.5$	Less negative
	$p_{\text{Holm}}$	0.017	0.16	Sig. $\to$ NS
H1b (Multi-agent)	OR	0.95	0.95	No
	RD (pp)	$-0.84$	$-0.75$	Less negative
	$p_{\text{Holm}}$	0.017	0.086	Sig. $\to$ NS

Table 17: Primary hypothesis tests for the full sample (all six models;

N=60{,}112

) and a sensitivity analysis excluding Opus (five models;

N=49{,}670

Recommendation	Evidence Basis	Practice Gap	Cost
Format-paired reporting	about 5 to 20 pp format gaps (Sec. 5.1 )	No benchmark requires dual format	Low
Structure-destroying scaffold test	NNH = 14 under map-reduce (Sec. 4.2 )	Proxy safety benchmarks not scaffold-tested	Medium
Propagation verification	Option-marker loss at map workers (Sec. 5.2 )	No standard propagation audit	Low
NNH operational reporting	Enterprise risk communication (Sec. 7.2 )	Safety scores lack operational interpretation	Low
System prompt governance	Prompt competition can suppress scaffolds (Sec. 18 )	No API mechanism for safety-priority prompts	Low

Table 18: Actionable recommendations and their supporting evidence are listed.

Figure 9: Configuration-Aware Safety Scorecard. (A) Pooled change in safety (

\Delta

percentage points relative to the direct baseline) for each benchmark and scaffold. Brackets indicate 95% confidence intervals, estimated using a case-level cluster bootstrap with 2,000 replicates. Green indicates the point estimate for change in safety is

\geq-2

pp (not a TOST equivalence finding), amber is a small safety degradation (

-2

pp to

-5

pp), red is a moderate (

-5

pp to

-10

pp) or large safety degradation (

<-10

pp). (B) Heatmap by model.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dimension	Map-Reduce	RLM
MC choices in sub-calls	0%	77.9–87.9% (context var.)
System prompt propagation	Yes	0%
Content transformation	Abstract sub-questions	Full context via variable ref.
Sub-call count	3–5 (fixed pipeline)	3–12 (model-decided)
DeepSeek accuracy on TruthfulQA MC1
Short context	33.3% ( $n{=}57$ )	—

Appendix

Table 19: Pilot-stage dual-mechanism comparison: Map-Reduce vs. RLM on TruthfulQA MC1.

	Full context	Condensed
	( $\geq$ 5,000 chars)	( $<$ 5,000 chars)
Sub-call count	28 (84.8%)	5 (15.2%)
MC choices preserved	27/28 (96.4%)	2/5 (40.0%)
Mean prompt length (chars)	$\sim$ 15,400	$\sim$ 1,800
Typical prefix category	extraction	refinement/summary
Cases with errors	0/3 cases	1/2 cases affected

Appendix

Table 20: Pilot-stage analysis of DeepSeek condensation under RLM. We find that the condensed sub-calls (less than 5,000 characters) lose more MC choices, which motivated our use of the content-preservation metric in our full experiment.

Source	df	$F$	$\eta^{2}$ (%)	$\omega^{2}$ (%)	Magnitude
Benchmark	3	3,904	15.1	15.1	Large
Model $\times$ benchmark	14	104	1.9	1.9	Small
Model	5	234	1.5	1.5	Small
Scaffold $\times$ benchmark	9	126	1.5	1.5	Small
Model $\times$ scaffold	15	25	0.5	0.5	Negligible
Scaffold	3	119	0.5	0.5	Negligible

Appendix

Table 21: This table shows a factorial variance decomposition for binary safety outcomes (

N=60{,}112

), with sources ordered by

\eta^{2}

. The table only includes main effects and two-way interactions. All effects are significant at

p<10^{-6}

. Because the three-way interaction term (model

\times

scaffold

\times

benchmark) is contained in the residual term, the residual term overstates the pure within-cell variance. Adding the three-way interaction term to the decomposition shifts approximately 0.8 pp of

\eta^{2}_{\text{residual}}

into a new model

\times

scaffold

\times

benchmark term and leaves the main effect of scaffold negligible. The qualitative conclusions are the same: the choice of benchmark is the dominant source of variance, the main effect of the scaffold is small, and the interaction between scaffold and benchmark is larger (3.2

\times

) than the main effect of the scaffold. Ratios quoted in the paper use unrounded

\eta^{2}

(equivalently

F\times

df).

Model	$\eta^{2}_{\text{scaffold}}$ (%)	Avg range (pp)	Max range (pp)	Overall safe (%)
DeepSeek V3.2	2.1	27.7	47.5	66.8
Mistral Large 2	0.2	17.3	25.6	69.7
Opus 4.6	2.5	16.2	24.7	80.5
GPT-5.2	0.4	14.0	23.9	70.5
Llama 4 Maverick	0.1	13.6	19.0	68.8
Gemini 3 Pro	0.1	6.1	12.8	89.2

Appendix

Table 22: Sensitivity of models to the choice of scaffolding. Range (max

-

min) of safety rates across the four scaffold configurations within each benchmark, averaged across benchmarks.

\eta^{2}_{\text{scaffold}}

is the fraction of within-model variance explained by scaffold.

Hyp.	Comparison	RD (pp)	Wald 95% CI	Wald 90% CI	Bootstrap 95% CI
H1a	ReAct vs. direct	$-0.8$	[ $-1.7$ , $+0.2$ ]	[ $-1.6$ , $+0.1$ ]	[ $-1.2$ , $-0.2$ ]
H1b	Multi-agent vs. direct	$-0.8$	[ $-1.8$ , $+0.1$ ]	[ $-1.7$ , $-0.03$ ]	[ $-1.5$ , $-0.2$ ]
H1c	Map-reduce vs. direct	$-7.3$	[ $-8.3$ , $-6.3$ ]	[ $-8.2$ , $-6.5$ ]	[ $-8.1$ , $-6.4$ ]

Appendix

Table 23: These Wald CIs for the RD assume independence and are presented as a conservative sensitivity check.

Benchmark	ReAct	Multi-agent	Map-reduce
BBQ	$-0.21$ [ $-0.72$ , $+0.31$ ] †	$-7.55$ [ $-8.58$ , $-6.48$ ]	$-8.31$ [ $-9.16$ , $-7.52$ ]
Sycophancy	$+0.28$ [ $-1.13$ , $+1.70$ ] †	$+2.06$ [ $+0.85$ , $+3.31$ ]	$+0.24$ [ $-1.54$ , $+2.07$ ]
TruthfulQA	$+0.63$ [ $-0.06$ , $+1.29$ ] †	$+4.13$ [ $+3.20$ , $+5.12$ ]	$-19.35$ [ $-20.54$ , $-18.14$ ]
XSTest/OR-Bench	$-5.24$ [ $-6.39$ , $-4.13$ ]	$-0.48$ [ $-1.24$ , $+0.28$ ] †	$+6.68$ [ $+5.04$ , $+8.35$ ]

Appendix

Table 24: Per-benchmark TOST at

\Delta=2

pp: risk difference (pp) over model–case pairs present in both arms, with 90% case-cluster bootstrap CI (

B=2{,}000

). † CI inside

(-2,+2)

: equivalent.

Explore similar work

Apr 27, 2026cs.CL

How Sensitive Are Safety Benchmarks to Judge Configuration Choices?

Safety benchmarks such as HarmBench rely on LLM judges to classify model responses as harmful or safe, yet the judge configuration, namely the combination of judge model and judge prompt, is typically treated as a fixed implementation detail. We show this assumption is problematic. Using a 2 x 2 x 3 factorial design, we construct 12 judge prompt variants along two axes, evaluation structure and instruction framing, and apply them using a single judge model, Claude Sonnet 4-6, producing 28,812 judgments over six target models and 400 HarmBench behaviors. We find that prompt wording alone, holding the judge model fixed, shifts measured harmful-response rates by up to 24.2 percentage points, with even within-condition surface rewording causing swings of up to 20.1 percentage points. Model safety rankings are moderately unstable, with mean Kendall tau = 0.89, and category-level sensitivity ranges from 39.6 percentage points for copyright to 0 percentage points for harassment. A supplementary multi-judge experiment using three judge models shows that judge-model choice adds further variance. Our results demonstrate that judge prompt wording is a substantial, previously under-examined source of measurement variance in safety benchmarking.

Xinran Zhang

University of California, Berkeley, Berkeley CA 94720, USA

Safety Benchmarks

Ask PDF

Jun 22, 2026cs.CL

Evaluation Awareness Is Not One Capability: Evidence from Open Language Models

Safety benchmarks assume that test-condition behavior predicts deployment behavior, an assumption that fails if models detect evaluation cues and adapt. This opens a gap between benchmark performance and deployment behavior: compliance measured under test conditions becomes an optimistic upper bound that overstates how safely a model behaves once the evaluation harness is removed. We characterize this evaluation awareness through eight experiments across 37 open-weight models and seven families. (i)Detection is moderate and training-driven (24/37 models exceed chance, best AUROC 0.714 vs.0.819 human, with instruction tuning dominating over scale). (ii)Detection shifts safety behavior (hard refusal drops 5.8 percentage points under hypothetical framing, and 21/140 HarmBench framing effects are significant, with compliance rising up to +30 percentage points. (iii)Representations survive behavioral collapse (probes retain AUROC 0.98 under rewrites that drive behavior below chance, and multi-layer steering causally moves three downstream tasks while random controls do not). (iv)These axes are weakly coupled (only 1/15 correlations are significant, the sole robust link being behavioral detection versus framing resistance,

ρ=-0.79

p<0.001

). We call this gap the benchmark illusion: because detectability, behavioral manifestation, and controllability vary independently, it is multivariate rather than a single number, so no single awareness score is a reliable proxy for deployment safety.

Nilesh Nayan, Aishwarya Sampath Kumar, Rishiraj Girmal +5

University of Massachusetts, Amherst · 2Microsoft Corp.

Safety Benchmarks

Ask PDF

May 21, 2026cs.CL

Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety

Background. Traditional safety benchmarks for language models evaluate generated text: whether a model outputs toxic language, reproduces bias, or follows harmful instructions. When models are deployed as agents, the safety-relevant object shifts from what the system says to what it does within an environment, and evaluating model responses under prompting is no longer sufficient to address the safety challenges posed by artificial intelligence. Recent developments have seen the rise of benchmarks that evaluate large language models as agents. We contribute to this strand of research. Approach. We introduce Boiling the Frog, a benchmark that evaluates whether tool-using AI models deployed in corporate and office settings are susceptible to incremental attacks. Each scenario begins with benign workspace edits and later introduces a risk-bearing request. The benchmark focuses on stateful multi-turn evaluation: chains expose a persistent workspace, place the risk-bearing payload at controlled positions in the turn sequence, and score whether the resulting artifact state becomes unsafe. Scenarios are organized through a three-level operational risk taxonomy grounded in the Boiling the Frog risks, the AI Act Annex I and Annex III high-risk contexts, and EU AI Act's Code of Practice on General-Purpose AI (GPAI). Results. Across a nine-model panel, aggregate strict attack success rate (ASR) is 44.4%. Model-level ASR ranges from 20.5% for Claude Haiku 4.5 to 92.9% for Gemini 3.1 Flash Lite, with Seed 2.0 Lite also above 80%. Average chain category-level ASR reaches 93.3% for Code of Practice loss-of-control scenarios.

Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11

Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6

Large Language Model Safety Safety Benchmarks Multi-Turn Benchmark Molten Pool

Ask PDF