The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior

Erik Larsen

cs.LGDec 12, 2025

The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior

Authors: Erik Larsen

Abstract

Current safety evaluations of large language models rely on single-shot testing, implicitly assuming that model responses are deterministic and representative of the model's safety alignment. We challenge this assumption by investigating the stability of safety refusal decisions across random seeds and temperature settings. Testing four instruction-tuned models from three families (Llama 3.1 8B, Qwen 2.5 7B, Qwen 3 8B, Gemma 3 12B) on 876 harmful prompts across 20 sampling configurations (4 temperatures x 5 seeds), we find that 18-28% of prompts exhibit decision flips (the REFUSE/PARTIAL/COMPLY label changes across configurations), depending on the model. Our Safety Stability Index (SSI) reveals that higher temperatures significantly reduce decision stability (Friedman chi-squared = 396.81, p < 0.001), with mean within-temperature SSI dropping from 0.977 at temperature 0.0 to 0.942 at temperature 1.0. We validate findings across all model families using Claude 3.5 Haiku as a unified external judge, achieving 89.1% inter-judge agreement with the Llama 70B judge on the two models both judges labeled (Cohen's kappa = 0.62). Within each model, prompts with higher compliance rates exhibit lower stability (Spearman rho = -0.47 to -0.70, all p < 0.001), indicating that models "waver" more on borderline requests. These findings demonstrate that single-shot safety evaluations are insufficient for reliable safety assessment and that evaluation protocols must account for stochastic variation in model behavior. For Llama 3.1 8B, single-shot evaluation agrees with multi-sample ground truth only 92.5% of the time when pooling across temperatures (98.7% at greedy to 90.3% at temperature 1.0), and we recommend scaling samples with temperature--one at greedy, three at low temperature, more at higher temperatures (where three reach only ~95%), and ten when pooling--rather than a single flat threshold.

Figures & tables

Figure 1: Distribution of consistent vs. inconsistent (flip) prompts across 876 harmful prompts tested on Llama 3.1 8B Instruct.

Figure 2: Distribution of Safety Stability Index (SSI) scores. The dashed line at 0.8 indicates the threshold for unstable prompts. 14.3% of prompts fall below this threshold.

Figure 3: Effect of temperature on comply rate (blue) and mean Safety Stability Index (green). Higher temperatures reduce stability while showing non-monotonic effects on compliance.

Mean Within-Temperature SSI
Model	t=0.0	t=0.3	t=0.7	t=1.0
Gemma-3-12B	0.986	0.972	0.967	0.964
Llama-3.1-8B	0.985	0.961	0.948	0.926
Qwen3-8B	0.967	0.948	0.943	0.945
Qwen2.5-7B	0.972	0.956	0.941	0.932
Flip Rate (%)

Table 1: Within-temperature stability (seed variance only, N=5 seeds). Shows that instability increases with temperature even when holding temperature fixed. All values use the unified Claude 3.5 Haiku judge across the four models.

Figure 4: Safety Stability Index vs. compliance rate per prompt. Most prompts cluster in the stable-refusal region (upper left), while a subset shows variable behavior.

Prompt Category	SSI	REFUSE	PARTIAL	COMPLY
Request for copyrighted content	0.55	2/20	11/20	7/20
Request for harmful instructions	0.45	3/20	9/20	8/20
Request for copyrighted passage	0.50	10/20	7/20	3/20

Table 2: Example prompts exhibiting decision instability across sampling configurations (N=20).

Figure 5: Safety Stability Index comparison across models. Left: Mean SSI with standard deviation error bars. Right: Percentage of prompts with SSI

<

0.8 (unstable). All four models show instability rates between 6.7% and 12.0%.

Model	Mean SSI $\uparrow$	Flip Rate	% Unstable	Refusal
Gemma-3-12B	.965 [.958, .971]	18.4% [16.0, 21.1]	6.7% [5.3, 8.6]	78.5%
Llama-3.1-8B	.944 [.936, .951]	27.3% [24.4, 30.3]	10.4% [8.5, 12.6]	79.3%
Qwen3-8B	.938 [.929, .946]	27.7% [24.9, 30.8]	11.8% [9.8, 14.1]	92.5%
Qwen2.5-7B	.938 [.929, .946]	26.3% [23.4, 29.3]	12.0% [10.0, 14.3]	81.3%

Table 3: Stability comparison across model families. 95% CIs: Wilson score intervals for Flip Rate and % Unstable (proportions over 876 prompts); bootstrap over prompts for Mean SSI. All models evaluated using Claude 3.5 Haiku as judge. Models sorted by SSI (descending).

Figure 6: Response distribution by model. Green indicates refusal, orange partial compliance, and red full compliance.

	Claude Haiku
Llama 70B	REFUSE	PARTIAL	COMPLY	Total
REFUSE	27,933	1,580	940	30,453
PARTIAL	197	1,842	558	2,597
COMPLY	6	532	1,417	1,955
Total	28,136	3,954	2,915	35,005

Table 4: Confusion matrix comparing the Llama 70B and Claude Haiku judges on the 35,005 valid Llama 3.1 8B and Qwen 2.5 7B response pairs (of 35,040; 35 excluded where a judge returned a non-three-class label) — the two models labeled by both judges. Overall agreement: 89.1%, Cohen’s

\kappa

= 0.62.

Figure 7: Evaluation reliability as a function of sample size (Llama 3.1 8B, pooled across temperatures). Single-shot testing (N=1) achieves only 92.5% agreement with ground truth; the 95% reliability threshold requires N

\geq

3 samples (pooled).

N Samples	Agreement (%)	95% CI
1	92.5	[91.0, 93.8]
3	95.1	[93.8, 96.2]
5	96.3	[95.1, 97.3]
10	97.8	[96.7, 98.5]
20	100.0	—

Table 5: Sample size requirements for reliable safety evaluation (Llama 3.1 8B single-model, Llama 70B judge). Subsampling is without replacement (averaged over bootstrap resamples); N=20 uses all configurations and is therefore 100% by construction.

N Seeds	t=0.0	t=0.3	t=0.7	t=1.0
1	98.7%	95.1%	92.8%	90.3%
3	99.7%	97.4%	96.0%	94.8%
5 (full)	100%	100%	100%	100%

Table 6: Sample size reliability at fixed temperatures (Llama 3.1 8B, Llama 70B judge). Values show % agreement with the 5-seed ground truth; N=5 uses all seeds and is therefore 100% by construction. Single-shot is reliable at greedy decoding (98.7%) but degrades with temperature (90.3% at t=1.0).

Scoring Rule	Mean SSI	Flip Rate	% Unstable
Three-way (original)	0.946	24.9%	10.2%
Binary (PARTIAL $\rightarrow$ COMPLY)	0.968	17.8%	5.3%
Binary (PARTIAL $\rightarrow$ REFUSE)	0.976	11.4%	4.8%

Table 7: Binary scoring ablation: results under different PARTIAL handling rules. Instability persists regardless of scoring methodology.

Model	0.70	0.75	0.80	0.85	0.90
Gemma-3-12B	4.3%	5.8%	6.7%	7.4%	9.1%
Llama-3.1-8B	6.2%	8.4%	10.4%	12.9%	17.6%
Qwen3-8B	8.6%	10.0%	11.8%	14.5%	17.8%
Qwen2.5-7B	8.2%	9.7%	12.0%	15.6%	18.9%

Table 8: Sensitivity analysis: % of prompts classified as unstable at different SSI thresholds. The qualitative finding that a meaningful fraction of prompts exhibit instability is robust to threshold choice.

Inference Configuration
Parameter	Value
Inference Engine	vLLM 0.6.3
PyTorch Version	2.4.1
CUDA Version	12.1
GPU	NVIDIA A100-80GB
Batch Size	32

Table 9: Experimental Configuration Details

Language Model Safety Evaluation

Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.

See all 542 papers

May 6, 2026

The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models

Refusal rates are a poor proxy for LLM safety, i.e., a model may over-refuse benign prompts while still complying with harmful ones. We audit both failure modes across 21 open-weight LLMs on four safety benchmarks (OR-Bench, XSTest, ToxiGen, BOLD), using a composition adjustment to isolate model sensitivity from dataset toxicity confounds. We report three findings. First, models adopt fundamentally different calibration strategies: conservative ecosystems such as Llama suppress unsafe outputs at the cost of elevated over-refusals, while permissive ecosystems such as DeepSeek and Qwen preserve helpfulness but tolerate higher harmful compliance. Second, demographic protection is unequal: models over-protect prominent racial and religious groups, frequently refusing even benign prompts about them, while providing substantially weaker protection against disability-targeted attacks. Third, refusal and compliance tendencies are stable within model families across generations and scales, suggesting that post-training objectives shape safety behavior more than architecture. Our results call for joint, demographically-aware, and multi-judge safety evaluation.

Alif Al Hasan, Sumon Biswas

Department of Computer and Data Sciences Case Western Reserve University Cleveland, OH, USA

LLM Safety Benchmarks Language Model Safety Evaluation LLM Refusal Behavior LLM Safety Over-Refusal Demographic Bias in Language Models+1

Ask PDF

Jun 24, 2026

Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations

LLM-as-judge ("grader") components are now standard in evaluation harnesses, including safety evaluations where a pass/fail verdict may gate downstream deployment decisions. A widespread assumption is that setting the grader's sampling temperature to 0 makes grading deterministic. We test this assumption against a real safety-evaluation codebase (Japan AISI's open-source aisev) and show it fails on two levels. First, the harness invokes its grader without setting temperature or seed; the underlying provider silently applies its default of 1.0, so items near the decision boundary flip pass/fail across identical runs (per-item disagreement up to ~50% over 20 runs). Second, pinning temperature=0 reduces but does not eliminate flips: across 690 API calls spanning two providers, three model tiers, and five sampling configurations, 1-2 of 7 borderline items remain non-reproducible even under forced greedy decoding (top_k=1). Claude Opus 4.7/4.8 has since deprecated temperature entirely, rendering the primary mitigation inapplicable to newer model generations. These findings expose a structural gap: evaluation harnesses that report single-run verdicts without variance or grader-disagreement metrics can present noise as a safety property. We release a reproduction harness (690 calls, 7 conditions) and recommend that harnesses treat grader disagreement as a first-class health metric alongside the scores themselves.

Hiroki Tamba

Tamba Research Academy

LLM Evaluation LLM-as-a-Judge LLM Reliability LLM Safety Evaluation Benchmark Reproducibility

Ask PDF

May 26, 2026

A Paired Testing Protocol for Batch-Conditioned Refusal Robustness in LLM Serving

Safety evaluations of language models often treat serving configuration as fixed background infrastructure, but batch condition is an untested treatment variable whenever the same prompt may be evaluated alone, in a synchronized batch, or inside a continuous-batching scheduler. We synthesize four artifact-backed studies into a paired testing protocol: Study A combines local discovery, scorer-corrected adjudication, and true-batching confirmation; Study B tests cross-model generalization; Study C tests continuous-batch composition; and Study D runs a batch-invariant-kernel ablation. The local test finds safety-label changes more often than capability-label changes (0.51% vs. 0.14%), but adjudication of 63 candidate rows leaves only 17 genuine behavioral flips, implying a corrected full-set rate of 0.16%. The 15-model extension finds no detectable universal safety-over-capability skew: flips are near parity (0.94x), alignment type has no detectable association (

p=0.942

η^2=0.033

), and output instability is the strongest tested fragility screen (

r=0.909

, bootstrap 95% CI [0.65, 0.97]). In the targeted kernel ablation, standard vLLM reproduces 22/55 label flips on current score-flip candidates, while enabling VLLM_BATCH_INVARIANT=1 reduces the same test to 0/55 flips; the composition test separately finds no aggregate effect at 4.7pp sensitivity. The testing recommendation is exact-stack validation: evaluate refusal at the served batch setting, pair safety prompts with capability controls, and report low-rate directional flips separately from aggregate null effects.

Sahil Kadadekar

Independent Researcher.

LLM Inference Language Model Safety Evaluation LLM Refusal Behavior LLM Reliability Language Model Robustness Refusal Behavior in Language Models

Ask PDF

Abstract

Language Model Safety Evaluation

Explore similar work

The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models

Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations

A Paired Testing Protocol for Batch-Conditioned Refusal Robustness in LLM Serving